PRTG API to add maintenance window

I recently had a need to provide capability for adding one-time maintenance windows in PRTG for a specific set of objects.

I found this post on the PRTG forums as a starting point. I also needed to learn how to authenticate an API request in PowerShell, which Paessler has provided a KB article on.

Part of my requirements were conditional logic, to say “Pause these sensors, and maybe pause these other ones too if desired”. I used a Switch parameter in my PowerShell script to accomplish this.

One of the remaining downsides of this script is that it requires pre-knowledge of the exact object IDs from PRTG. These are easy to find, by navigating to the object you desire, and looking at the URL which will display it.

I also want to be able to call this script from a website with user-specified parameters, but that will be for a future post.

Here’s my script, which can be called like this:

$start = get-date
$end = (get-date).AddMinutes(5)
.\PrtgMaintenanceWindow.ps1 -MaintStartTime $start -MaintEndTime $end -IncludProdWebServers -IncludeTestWebServers

Full Script:

param(
    [Parameter(Mandatory = $true)] [datetime]$MaintStartTime,
    [Parameter(Mandatory = $true)] [datetime]$MaintEndTime,
    [Switch]$IncludeProdWebServers,
    [Switch]$IncludeTestWebServers
)
 
# Use $Global parameters so they can be used inside the Function without repeating
$Global:prtgAuth = 'username=PRTGUSERNAME&passhash=GENERATEDHASHVALUE'
$Global:prtgServer = 'https://FQDN.OF.PRTG'
$Global:MaintStart = '{0:yyyy-MM-dd-HH-mm-ss}' -f $MaintStartTime
$Global:MaintEnd = '{0:yyyy-MM-dd-HH-mm-ss}' -f $MaintEndTime
 
$ServicesID = @("OBJECTID") # Group containing devices & sensors that I want to control
$ProdWebServersID = @("13152", "13153", "13149", "13150") # Individual Devices to conditionally apply a maintenance window to
$TestWebServersID = @("13219", "13221", "13220", "13222")
 
# Function that can be called multiple times, after passing in an ObjectID.
function ApplyMaintenanceWindow {
    param(
        [int]$objectid
    )
    # Apply Start Time of Maintenance Window
    $startattempt = Invoke-WebRequest "$prtgServer/api/setobjectproperty.htm?id=$objectid&name=maintstart&value=$MaintStart&$prtgAuth" -UseBasicParsing
 
    # Display the output as successful if HTTP200 response code received. Using Out-String for future integration purposes with website. 
    if ($startattempt.StatusCode -eq "200") {
        $message = "Object ID: $objectid - Maintenance window set to start at $MaintStart"
        $message | out-string
    }
    # Apply End Time of Maintenance Window
    $endattempt = Invoke-WebRequest "$prtgServer/api/setobjectproperty.htm?id=$objectid&name=maintend&value=$MaintEnd&$prtgAuth" -UseBasicParsing
    if ($endattempt.StatusCode -eq "200") {
        $message = "Object ID: $objectid - Maintenance window set to end at $MaintEnd"
        $message | out-string
    }
    # Enable Maintenance Window for the object
    $enableattempt = Invoke-WebRequest "$prtgServer/api/setobjectproperty.htm?id=$objectid&name=maintenable&value=1&$prtgAuth" -UseBasicParsing
    if ($enableattempt.StatusCode -eq "200") {
        $message = "Object ID: $objectid - Maintenance window enabled"
        $message | out-string
    }
}
 
# Add maintenance Window for Client Services
# Do this always, with the parameters supplied
foreach ($id in $ClientServicesID) {
    ApplyMaintenanceWindow -objectid $id
}
 
#If necessary, add maintenance window for ProdWebServers
# Do this conditionally, if the switch is provided
if ($IncludeProdWebServers.IsPresent) {
    foreach ($id in $ClientProdWebServersID) {
        ApplyMaintenanceWindow -objectid $id
    }
}
 
#If necessary, add maintenance window for TestWebServers
# Do this conditionally, if the switch is provided
if ($IncludeTestWebServers.IsPresent) {
    foreach ($id in $ClientTestWebServersID) {
        ApplyMaintenanceWindow -objectid $id
    }
}

 

 

Hyper-V replica health notifications

I set up Hyper-V replica between two clusters in two offices, and needed a way of keeping track of replica health. I used a PowerShell script running from a utility server on a scheduled task to accomplish this.

This script runs a “get-vmreplication” command on each cluster node, and sends an email if any is found in warning or critical state.

One issue I needed to solve was the permissions required to run this from a utility server. I’m sure there are many other (and better) ways to accomplish this such as group managed service accounts, but there are certain limitations in my environment.

First I created an account in AD to act as a service account, as a standard user. I used the “ConvertFrom-SecureString” cmdlet as demonstrated in this article, as the SYSTEM account on my utility server, to produce a file for building a credential object in my PowerShell script. To run this as SYSTEM I used “psexec -s -i powershell.exe”.

Then I created a scheduled task, set to run as SYSTEM when not logged on, at 12 hour intervals, with the action of running the script below:

 

$password = Get-Content C:\scripts\MaintenanceChecks\SecureString.txt | ConvertTo-SecureString
$Credential = New-Object System.Management.Automation.PSCredential("svc.clusterhealth@domain.com",$password)	
 
$Servers = @("ClusterHost1","ClusterHost2","ClusterHost3","ClusterHost4")
 
Foreach ($server in $Servers)
{
	Invoke-Command -credential $credential -computername $server -scriptblock {
 
		$resultsWarn = get-vmreplication | where-object {$_.Health -like "Warning"} 
		if ($resultsWarn)
		{
			$smtpServer = "relay.domain.com"
			$port = "25"
			$message = New-Object System.Net.Mail.MailMessage
			$message.From = "fromaddress@domain.com"
			$message.Sender = "fromaddress@domain.com"
			$message.To.Add( "toaddress@domain.com" )
			$message.Subject = "Replication Health Warning - $($Using:server)"
			$message.IsBodyHtml = $true
			$message.Body = "Replication health was found to be in warning state for the following VMs:  $($resultsWarn.Name)"
			$Client = New-Object System.Net.Mail.SmtpClient( $smtpServer , $port )
			$Client.Send( $message )
		}
 
		$resultsCrit = get-vmreplication | where-object {$_.Health -like "Critical"} 
		if ($resultsCrit)
		{
			$smtpServer = "relay.domain.com"
			$port = "25"
			$message = New-Object System.Net.Mail.MailMessage
			$message.From = "fromaddress@domain.com"
			$message.Sender = "fromaddress@domain.com"
			$message.To.Add( "toaddress@domain.com" )
			$message.Subject = "Replication Health Critical - $($Using:server)"
			$message.IsBodyHtml = $true
			$message.Body = "Replication health was found to be in CRITICAL state for the following VMs: $($resultsCrit.Name)"
			$Client = New-Object System.Net.Mail.SmtpClient( $smtpServer , $port )
			$Client.Send( $message )
		}
	}
}

 

 

 

Azure update configuration – dynamic group workaround

I put my Azure Update Management into full testing recently, using the deployment script I shared last week.

I quickly encountered a problem where despite the VMs communicating with Azure Automation properly, and showing as “ready” for Update Management, they would no longer appear as selected when using Dynamic Groups.

I suspect this may have something to do with the proxy setup in this particular environment, but I didn’t have time to troubleshoot and needed to have my scheduled update configurations statically select my VMs instead of relying upon dynamic groups (they are after all still in preview).

However, I still wanted to use Tags in order to select my VMs. In order to do this, I removed the “Targets” object from my JSON body entirely, and used the “azureVirtualMachines” object instead. I needed to put the logic of VM selection based on tags into this object.

I recognize that this removes the “dynamic” nature, in that it would only be updated when I re-run the PowerShell script to update the scheduled update configuration, but since this is going to be run once-per-month in order to change the date (Microsoft, add day offset support!) that isn’t a large problem.

The only real change is that selection of the VMs by tag can be done like this:

$selectedvms = Get-AzureRmVM | Where-Object {$_.Tags['MaintenanceWindow'] -eq $MaintenanceWindow} | select-object id
$virtualmachines = $selectedvms.id | ConvertTo-JSON

Multiple Tag definitions can be added into the “Where-Object” logic of that command.

Full Example

 
# API Reference: https://docs.microsoft.com/en-us/rest/api/automation/softwareupdateconfigurations/create#updateconfiguration
 
### Monthly Parameters ###
  $deploymentName = "SUG_Thurs-2am-MST-4hours"
  $MaintenanceWindow = "Thurs-2am-MST-4hours"
  $starttime = "2018-11-15T02:00:00+00:00"
 
# Scope Parameters
  $clientsubscription = "<subscription_id_to_target>"
  Select-AzureRmSubscription -Subscription $clientsubscription
  ## Schedule Parameters
  # Populate an array with the full ID of all VMs to apply this schedule to:
  $selectedvms = Get-AzureRmVM | Where-Object {$_.Tags['MaintenanceWindow'] -eq $MaintenanceWindow} | select-object id
  $virtualmachines = $selectedvms.id | ConvertTo-JSON
 
# Static Schedule Parameters
  $AutomationRG = "test-rg" # Resource Group the Automation Account resides in
  $automationAccountName = "test-automation"
  $automationSubscription = "<subscriptionId>" # Subscription that the Automation Account resides in
 
  $duration = "PT4H0M" # This equals maintenance window - Put in the format PT2H0M, changing the numbers for hours and minutes
  $rebootSetting = "IfRequired" # Options are Never, IfRequired
  $includedUpdateClassifications = "Critical,UpdateRollup,Security,Updates" # List of options here: https://docs.microsoft.com/en-us/rest/api/automation/softwareupdateconfigurations/create#windowsupdateclasses
  $timeZone = "America/Edmonton" # List from ??
  $frequency = "OneTime" # Valid values: https://docs.microsoft.com/en-us/rest/api/automation/softwareupdateconfigurations/create#schedulefrequency
  #$interval = "1" # How often to recur based on the frequency (i.e. if frequency = hourly, and interval = 2, then its every 2 hours)
 
 
### These values below shouldn't need to change
  Select-AzureRmSubscription -Subscription "$automationSubscription"
  # Get the access token from a cached PowerShell session
  . .\Get-AzureRmCachedAccessToken.ps1 # Source = https://gallery.technet.microsoft.com/scriptcenter/Easily-obtain-AccessToken-3ba6e593
  $BearerToken = ('Bearer {0}' -f (Get-AzureRmCachedAccessToken))
  $RequestHeader = @{
    "Content-Type" = "application/json";
    "Authorization" = "$BearerToken"
  }
 
# JSON formatting to define our required settings
$Body = @"
{
  "properties": {
    "updateConfiguration": {
	  "operatingSystem": "Windows",
      "duration": "$duration",
      "windows": {
        "excludedKbNumbers": [],
        "includedUpdateClassifications": "$includedUpdateClassifications",
        "rebootSetting": "$rebootSetting"
      },
      "azureVirtualMachines": $virtualmachines,
    },
    "scheduleInfo": {
      "frequency": "$frequency",
      "startTime": "$starttime",
      "timeZone": "$timeZone",
      "interval": $interval,
	  "isEnabled": true
    }
  }
}
"@
 
# Build the URI string to call with a PUT
$URI = "https://management.azure.com/subscriptions/$($automationSubscription)/" `
     +"resourceGroups/$($AutomationRG)/providers/Microsoft.Automation/" `
     +"automationAccounts/$($automationaccountname)/softwareUpdateConfigurations/$($deploymentName)?api-version=2017-05-15-preview"
 
# use the API to add the deployment
$Response = Invoke-RestMethod -Uri $URI -Method Put -body $body -header $RequestHeader

Parameterized Update Configuration deployment

After getting a functional script working for deploying an Update Configuration in my last post, I began working on making it worthwhile to use in the future.

I wanted to parameterize the JSON, so that it could be re-used in a much more efficient manner without having to scroll through the file to the JSON body and manually edit sections. This would be needed for both the Subscriptions/ResourceGroups (in the Scope object) and the Tags that I wanted to use.

In this example, I wanted a scheduled deployment that ran once (so I could control the date for the first Wednesday after the second Tuesday of the month), with a maintenance window of 4 hours, with dynamic grouping applied against multiple subscriptions and virtual machines with a defined “MaintenanceWindow” tag.

For the Scope definition, I created a PowerShell array containing the resource groups resource IDs. The format of these can be found in the Portal under “Properties” of the resource group.

# Populate this array with the subscription IDs and resource group names that it should apply to
$scopeDefinition = @(
  "/subscriptions/$subscriptionId/resourceGroups/managementVMs"
  ,"/subscriptions/$subscriptionId/resourceGroups/webVMs"
)

The Tag definition was created as a Hashtable, noting the tag value that I wanted to include:

# Populate this Hashtable with the tags and tag values that should it should be applied to
$tagdefinition = @{
  MaintenanceWindow = @("Thurs-2am-MST-4hours")
}

* side note, since Update Management doesn’t operate on the basis of local time of the VM (as SCCM is capable of) I need to build my maintenance windows around discrete time zones, and have multiple update configurations for these time zones even if the local time that I’m targeting is always the same (i.e. 2am).

I can now convert these Powershell variables into JSON:

$applyResourceGroup = $scopeDefinition | ConvertTo-JSON
$applyTags = $tagdefinition | ConvertTo-JSON

and reference it within my JSON body like this:

"targets": 
      {
        "azureQueries": [{
                  "scope": $applyResourceGroup,
                  "tagSettings": {
                      "tags": $applyTags,
                      "filterOperator": "Any"
                  },
                  "locations": null
              }]
      }

Full Example

Here’s a full example that makes parameters of the full contents of the JSON body:

 
# API Reference: https://docs.microsoft.com/en-us/rest/api/automation/softwareupdateconfigurations/create
### Monthly Parameters ###
$deploymentName = "SUG_Thurs-2am-MST-4hours"
$starttime = "2018-11-15T02:00:00+00:00" 
 
# Scope Parameters
$clientsubscription = "<subscription_id_to_target>"
# Populate this array with the subscription IDs and resource group names that it should apply to
$scopeDefinition = @(
  "/subscriptions/$clientsubscription/resourceGroups/managementVMs"
  ,"/subscriptions/$clientsubscription/resourceGroups/webVMs"
)
 
# Static Schedule Parameters
$AutomationRG = "test-rg" # Resource Group the Automation Account resides in
$automationAccountName = "test-automation"
$automationSubscription = "<subscriptionId>" # Subscription that the Automation Account resides in
# Populate this Hashtable with the tags and tag values that should it should be applied to
$tagdefinition = @{
  MaintenanceWindow = @("Thurs-2am-MST-4hours")
}
$duration = "PT4H0M" # This equals maintenance window - Put in the format PT2H0M, changing the numbers for hours and minutes
$rebootSetting = "IfRequired" # Options are Never, IfRequired
$includedUpdateClassifications = "Critical,UpdateRollup,Security,Updates" # List of options here: https://docs.microsoft.com/en-us/rest/api/automation/softwareupdateconfigurations/create#windowsupdateclasses
$timeZone = "America/Edmonton" # List from ??
$frequency = "OneTime" # Valid values: https://docs.microsoft.com/en-us/rest/api/automation/softwareupdateconfigurations/create#schedulefrequency
#$interval = "1" # How often to recur based on the frequency (i.e. if frequency = hourly, and interval = 2, then its every 2 hours)
 
 
### These values below shouldn't need to change
Select-AzureRmSubscription -Subscription "$automationSubscription" # Ensure PowerShell context is targeting the correct subscription
$applyResourceGroup = $scopeDefinition | ConvertTo-JSON
$applyTags = $tagdefinition | ConvertTo-JSON
# Get the access token from a cached PowerShell session
. .\Get-AzureRmCachedAccessToken.ps1 # Source = https://gallery.technet.microsoft.com/scriptcenter/Easily-obtain-AccessToken-3ba6e593
$BearerToken = ('Bearer {0}' -f (Get-AzureRmCachedAccessToken))
$RequestHeader = @{
  "Content-Type" = "application/json";
  "Authorization" = "$BearerToken"
}
 
# JSON formatting to define our required settings
$Body = @"
{
"properties": {
  "updateConfiguration": {
    "operatingSystem": "Windows",
    "duration": "$duration",
    "windows": {
      "excludedKbNumbers": [],
      "includedUpdateClassifications": "$includedUpdateClassifications",
      "rebootSetting": "$rebootSetting"
    },
    "azureVirtualMachines": [],
    "targets": 
      {
        "azureQueries": [{
                  "scope": $applyResourceGroup,
                  "tagSettings": {
                      "tags": $applyTags,
                      "filterOperator": "Any"
                  },
                  "locations": null
              }]
      }
  },
  "scheduleInfo": {
    "frequency": "$frequency",
    "startTime": "$starttime",
    "timeZone": "$timeZone",
    "interval": $interval,
    "isEnabled": true
  }
}
}
"@
 
# Build the URI string to call with a PUT
$URI = "https://management.azure.com/subscriptions/$($automationSubscription)/" `
   +"resourceGroups/$($AutomationRG)/providers/Microsoft.Automation/" `
   +"automationAccounts/$($automationaccountname)/softwareUpdateConfigurations/$($deploymentName)?api-version=2017-05-15-preview"
 
# use the API to add the deployment
$Response = Invoke-RestMethod -Uri $URI -Method Put -body $body -header $RequestHeader

 

Azure Update Management deployments – programmatically

I’ve been working through a method to add Scheduled Deployments with Azure Update Management. The particular problem is that the docs for Update Management don’t reference any kind of programmatic way to add Scheduled Deployments, and while I did find the PowerShell cmdlets (in preview), they don’t allow for use of the new features such as Reboot Control and Dynamic Groups.

After a little bit more investigation, I came across the Azure REST API for “Software Update Configurations”. Using a REST API is new to me but at first glance it seems like it would achieve what I wanted, so I dove in.

GET Configuration

I wanted to test with a GET command first, to make sure that I understood how to use the API correctly and interpret the output.

First I created a simple update deployment in the Portal to test with.

When I began looking at how to make the API call, I found a lot of instruction on creating a Service Principal in Azure Active Directory, in order for my code to authenticate against the Azure REST API and make the calls it needs. In my case I’m building a script to run ad-hoc, not part of an integrated program or code-base that needs repeated and fully automated authentication. My thinking was, “if I’m authenticated to Azure in a PowerShell session (through Login-AzureRMAccount) why can’t I just use that to make the call?”

This exact thing is possible, as I discovered through this PowerShell Gallery submission. After loading and testing with this function, I found that it could successfully use my PowerShell session as the access token for my API call.

Putting that together with the GET example, I ended up with this PowerShell script:

# Define my parameters to the Automation Account
$resourceGroupName = "test-rg"
$automationAccountName = "test-automation"
$SubscriptionId = "subscription id"
$ConfigName = "2018-08-SUG"
 
# Import the function, contained in a file from the same directory
. .\Get-AzureRmCachedAccessToken.ps1
# Use the function to get the Access Token
$BearerToken = ('Bearer {0}' -f (Get-AzureRmCachedAccessToken))
# Add the Access Token into proper format
$RequestHeader = @{
    "Content-Type"  = "application/json";
    "Authorization" = "$BearerToken"
}
# Build the URI referencing my parameters
$URI = "https://management.azure.com/subscriptions/$($SubscriptionID)/"`
    + "resourceGroups/$resourcegroupname/providers/Microsoft.Automation/"`
    + "automationAccounts/$automationaccountname/softwareUpdateConfigurations/$($ConfigName)?api-version=2017-05-15-preview"
 
# Use the URI and the Request Header (with access token) and the method GET
$GetResponse = Invoke-RestMethod -Uri $URI -Method GET -header $RequestHeader

 

This returned output that looked like this:

id         : /subscriptions/f745d13d/resourceGroups/test-rg/providers/Microsoft.Autom
             ation/automationAccounts/test-automation/softwareUpdateConfigurations/2018-08-SUG
name       : 2018-08-SUG
type       :
properties : @{updateConfiguration=; scheduleInfo=; provisioningState=Succeeded; createdBy={scrubbed}; error=; tasks=;
             creationTime=2018-08-16T14:09:34.773+00:00; lastModifiedBy=;
             lastModifiedTime=2018-10-24T03:56:51.02+00:00}

Fantastic! This can be turned into read-able JSON by piping your $GetResponse like this:

 $GetResponse | ConvertTo-JSON

PUT Configuration

Let’s look at what it takes to use the PUT example from the REST API reference. Primarily, the difference is passing in a JSON Body to the “Invoke-RestMethod”, and changing the method to PUT rather than GET.

Before I get to working examples, I ran into problems trying to use the API reference that I want to highlight. I kept getting obscure errors on the “Targets” section of the JSON body that I couldn’t figure out for a while, until I began looking very closely at the output of my GET from an update deployment created in the portal and compared it against what I was trying to do with my JSON.

Something that particularly helped here was piping my “$Get-Response” like this:

$GetResponse | ConvertTo-JSON -Depth 10

This converts the output of the RestMethod into JSON, and ensures that the default depth of 5 is overridden so that it expands all the objects within the Targets array.

What I noticed is that the “Targets” object returns a proper structure with a single nested object (“azureQueries”) which itself is a list (as denoted by the square brackets):

 "targets": {
                "azureQueries": [
                    {
                        "scope": [ ],
                        "tagSettings": {},
                        "locations": null
                    }
                ]
            }

However, this is what the API reference uses as it’s structure:

"targets": [
    {
      "azureQueries": {
        "scope": [ ],
        "tagSettings": { },
        "locations":  null
      }
    }
  ]

Note that the square brackets for the Target object shouldn’t be there, and that they’re missing on the AzureQueries object.

Once this was solved, my very basic “Create” script with static JSON seemed to work.

Working Example

Here’s an example of my full PowerShell script, which creates a recurring schedule for Definition Updates applied against a specific resource group with any VM having the tag = “Client1”.

# Define my parameters to the Automation Account
$resourceGroupName = "test-rg"
$automationAccountName = "test-automation"
$SubscriptionId = "&lt;subscription_id&gt;"
$ConfigName = "MalwareDefinitions"
 
# Import the function, contained in a file from the same directory
. .\Get-AzureRmCachedAccessToken.ps1
# Use the function to get the Access Token
$BearerToken = ('Bearer {0}' -f (Get-AzureRmCachedAccessToken))
# Add the Access Token into proper format
$RequestHeader = @{
    "Content-Type"  = "application/json";
    "Authorization" = "$BearerToken"
}
 
$Body = @"
{
  "properties": {
    "updateConfiguration": {
      "operatingSystem": "Windows",
      "duration": "PT0H30M",
      "windows": {
        "excludedKbNumbers": [    ],
        "includedUpdateClassifications": "Definition",
        "rebootSetting": "Never"
      },
      "azureVirtualMachines": [
 
      ],
      "targets": 
        {
          "azureQueries": [{
                    "scope": [
                        "/subscriptions/$SubscriptionId/resourceGroups/$resourceGroupName"
                    ],
                    "tagSettings": {
                        "tags": {
                            "ClientID": [
                                "Client1"
                            ]
                        },
                        "filterOperator": "Any"
                    },
                    "locations": null
                }]
        }
    },
    "scheduleInfo": {
      "frequency": "Hour",
      "startTime": "2018-10-31T12:22:57+00:00",
      "timeZone": "America/Los_Angeles",
      "interval": 2,
      "isEnabled": true
    }
  }
}
"@
 
$URI = "https://management.azure.com/subscriptions/$($SubscriptionId)/"`
    + "resourceGroups/$($resourcegroupname)/providers/Microsoft.Automation/"`
    + "automationAccounts/$($automationaccountname)/softwareUpdateConfigurations/$($ConfigName)?api-version=2017-05-15-preview"
 
#This creates the update scheduled deployment
$Response = Invoke-RestMethod -Uri $URI -Method Put -body $body -header $RequestHeader