While on a recent business trip I was discussing some potential alerting platforms with a few colleagues of mine when one of them brought up a customer request he had for sending Zerto alerts to their Syslog servers. For those of you who don’t know what Zerto is, I recommend checking out the website for information. We began discussing the request further and what information customers might value from the alerts the Zerto Virtual Manager generates today.
After looking through the Alerts and API documentation we found that it was possible to pull the relevant data from the ZVM, we just needed a way to be able to send it to a Syslog server. I started looking into doing this with PowerShell and after doing some research and reading through some forums I came across the Posh-Syslog on GitHub and this looked like it would do the trick. There are different tools that can be used with PowerShell to send messages to Syslog, if you decide to choose one of those you will just need to update the script accordingly. For those who decide to use the Posh-Syslog module it will need to be installed on the host the script is running on before script execution.
I installed the module in my lab and began asking how Syslog functioned and one of my colleagues (Wes Carroll) pointed me to Syslog RFC 5424. The RFC defines what the protocol expects when receiving data and we found the following information Syslog requires:
Taking a look at the Zerto Alerts,Alarms, and Events Guide we were able to find all of the information needed for Syslog was provided by the alerts. Let’s walk through the example script and how it works. First you will need to define the variables listed below to execute the script. I’ve left the password in plain text for now just to be able to easily demonstrate the script functionality, but an example with secure password can be added:
In my lab to keep things simple I tested the script with default UDP port 514. Once the variables are complete we will create a logging directory to create a transcript file:
I’m going to skip over the SSL certification handling and the API authentication part of the script and get right into the functionality.One of the first hurdles Wes and I discussed was how to ensure each time the script pulled the ZVM’s Alerts API it didn’t gather the same alerts and keep sending them to a Syslog server. The Zerto alerts API will let you provide it a start time and an end time for the alerts you’re looking for, but you need to provide these to it. Within the script the functionality needed to be added to keep track of the time of the latest alert received from the API and sent to the Syslog server. Next time the API was queried only alerts newer than that last alert would be gathered to ensure duplicate alerts aren’t sent to the Syslog server.
This was where Wes told me about the concept of a book mark file. On the first time the script was run it would gather all the alerts, and then write the time stamp of the newest alert to the book mark file. The next time the script would run it would check to see if the book mark file exists,and if so add a millisecond to it. This would be used in the API query as the start time.
In the else statement you can see that if the book mark doesn’t exist the script will take the date the Zerto directory was created in the C:\Program File directory and use this as a starting time. If your installation of Zerto is under a different drive you’ll want to make sure to change this accordingly.
Now we just need to put together the appropriate strings for the Alerts API. In the $peersListApiURL you’ll see the $startTime variable which comes from either the book mark file or Zerto folder date we talked about previously.
The remaining commands in the script are nested within an IF statement so we’ll walk through them one section at a time. First we check IF there are any alerts because if there aren’t there is no reason to proceed since there’s nothing to do. After that we want to set the latest alert for our book mark file so the next iteration the script will know where we left off.
Once the latest alert is captured the $alertListJSON needs to be re-ordered before being resent. Earlier the ZVM alerts API was queried and the data returned had the most recent alert returned first, and the oldest alert is last. This needs to be reversed so that when the data is received by the Syslog server the oldest alert is received first and then the newer alerts.
One thing I always heard when doing deployments with customers is that they’d like to see any licensing or upgrade alerts to be classified as “informational” and not as warnings. I decided to add the functionality into the script specifically for those licensing and upgrade alerts. After some initial testing I found that there was an unnecessary html tag that was being presented in the message info for licensing alerts. With the $alertInfo variable I have removed this tag from the alert message if its present.
Now that the data is in the correct order it’s time to use the Send-SyslogMessage command from the Posh-Syslog module. The alert information, time stamp of the alert, level, and alert ID received from the alerts API line up with the required information for Syslog messages,timestamp, severity, and messageid. The process ID is left with a ‘-‘ otherwise based on the help documentation for the command it will instead send “PowerShell.exe” to the Syslog server. For the application name I chose Zerto Virtual Replication, but this string value can be changed.
The script itself is written to be run once, if you’d like it to function on a loop there are a few options. First you can add logic to the script to have it run on an interval of your choosing. The other option is to use Windows Task Manager. If you’d like the script to run more often I came across steps in this technetforum that outline how you can get Task Manager to run a task every minute. Before running the script it is important to check the default ports your Syslog application is listening on. I tested this with Solar Winds, Nagios, and Splunk and found with Nagios that it’s default port was not 514. Make sure to check your Syslog applications configuration, and then adjust the script as needed.
Once you have the script configured and run it you will begin to see Zerto alerts in your Syslog server.
Let me know if you have any questions or any comments / feedback. I’m also working on a part two of the script to add some additional functionality. Be on the lookout for a blog post about that as well as other automation discussions in the near future.
A copy of the script can be found here on Github: Send-ZVMSyslog