Using PRTG for service monitoring has been fairly effective for me, particularly with HTTP Advanced sensors to monitor a website. However, as more and more Azure resources are utilized, I want to continue to centralize my alerting and notifications within a single platform and that means integrating some Azure resource status into PRTG.

At a high level, here’s what needs to happen:

• Azure Function triggered on HTTP POST to query Azure resources and return data
• PRTG custom sensor template to interpret the results of the Function data
• PRTG custom lookup to establish a default up/down threshold
• PRTG REST sensor to trigger the function, and use the sensor template and custom lookup to properly display results

Azure Function

For my first use-case, I wanted to see the health status of the back-end pool members of an Application Gateway:

The intended goal is if a member becomes unhealthy, PRTG would alert using our normal mechanisms. I could use an Azure Monitor alert to trigger something when this event happens, but in reality it is easier for PRTG to poll rather than Azure Monitor to trigger something in PRTG.

I’m not going to cover the full walk-through of building an Azure Function; instead here is a good starting point.

I’m using a PowerShell function, where the full source can be found here: GitHub link

Here’s a snippet of the part doing the heavy lifting:

#Proceed if all request body parameters are found
if ($appgwname -and $httpsettingname -and $resourcegroupname -and $subscriptionid -and $tenantid) {
    $status = [HttpStatusCode]::OK
    # Make sure we're using the right Subscription
    Select-AzSubscription -SubscriptionID $subscriptionid -TenantID $tenantid
    # Get the health status, using the Expanded Resource parameter
    $healthexpand = Get-AzApplicationGatewayBackendHealth -Name $appgwname -ResourceGroupName $resourcegroupname -ExpandResource "backendhealth/applicationgatewayresource"
    # If serving multiple sites out of one AppGw, use the parameter $httpsettingname to filter so we can better organize in PRTG
    $filtered = $healthexpand.BackEndAddressPools.BackEndhttpsettingscollection | where-object { $_.Backendhttpsettings.Name -eq "$($httpsettingname)-httpsetting" }
    # Return results as boolean integers, either health or not. Could modify this to be additional values if desired
    $items = $filtered.Servers | select-object Address, @{Name = 'Health'; Expression = { if ($_.Health -eq "Healthy") { 1 } else { 0 } } }
    # Add a top-level property so that the PRTG custom sensor template can interpret the results properly
    $body = @{ items = $items }

}

You can test this function using the Azure Functions GUI or Postman, or PowerShell like this:

    $appsvcname = "appsvc.azurewebsites.net"
    $functionName = "Get-AppGw-Health"
    $functionKey = " insert key here "
    $Body = @"
{
    "httpsettingname": " prodint ",
    "resourcegroupname": " rgname ",
    "appgwname": " appgw name ",
    "subscriptionid": " subid ",
    "tenantid": " tenant id "
}
"@
    $URI = "https://$($appsvcname)/api/$($functionName)?code=$functionKey"
    Invoke-RestMethod -Uri $URI -Method Post -body $body -ContentType "application/json"

Expected results would look like this:

You can see the “Function Key” parameter in this code above; I’ve created a function key for our PRTG to authenticate against, rather than making this function part of a private VNET.

PRTG Custom Sensor Template

Now, in order to have PRTG interpret the results of that JSON body, and automatically create channels associated with each Item, we need to use a custom sensor template.

Here’s mine (github link):

{
  "prtg": {
    "description" : {
      "device": "azureapplicationgateway",
      "query": "/api/Get-AppGw-Health?code={key}",
      "comment": "Documentation is in Doc Library"
    },
    "result": [
      {

	"value": {
            #1: $..({ @.Address : @.Health }).*
        },
        "valueLookup": "prtg.customlookups.healthyunhealthy.stateonok",
        "LimitMode":0,
        "unit": "Custom",
      }
    ]
  }
}

The important part here is the “value” properties. The syntax for this isn’t officially documented, but Paessler support has provided a couple examples that I used, such as here and here. The #1 before the first semi-colon sets the channel name, and uses the first argument referenced within the braces (@.Address in this case). What is inside the braces associates the channel name to the value that is returned, which in this case is the boolean integer for “Health” that the Azure Function returns.

The valueLookup property references the custom lookup explained below.

This Sensor Template file needs to exist on the PRTG Probe that will be calling it, in this location:

Program Files (x86)\PRTG Network Monitor\Custom Sensors\rest

PRTG Custom Lookup

I want to be able to control what PRTG detects as “down” based on the values that my Function is returning. To do so, we apply a custom lookup (github link):

<?xml version="1.0" encoding="UTF-8"?>
  <ValueLookup id="prtg.customlookups.healthyunhealthy.stateonok" desiredValue="1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="PaeValueLookup.xsd">
    <Lookups>
      <SingleInt state="Error" value="0">
        Unhealthy
      </SingleInt>
      <SingleInt state="Ok" value="1">
        Healthy
      </SingleInt>
    </Lookups>
  </ValueLookup>

This matches my 0 or 1 return values to the PRTG states that a channel can be in. You can modify this to suit your Function return values.

This lookup file needs to exist on the PRTG Core, in this location:

Program Files (x86)\PRTG Network Monitor\lookups\custom

PRTG REST sensor

So lets put it all together now. Create a PRTG REST Sensor, and apply the following settings to it:

Your PostData should match the parameters you’re receiving into your Azure Function. The REST query directs to the URL where your function is located, and uses the function key that was generated for authentication purposes.

Important to note: the REST sensor depends upon the Device it is created under to generate the hostname for URL. This means you’ll need to have a Device created with a hostname matching your Function App URL, where the sensor’s REST Query references the function name itself.

Make sure the REST Configuration is directed to your custom sensor template; this can only be set on creation.

Then for each channel that gets auto-detected, you will go and modify it’s settings and apply the custom lookup:

With that, you should have a good sensor in PRTG relaying important information collected from Azure!

I’m also using this same method to collect Azure Site Recovery status from Azure and report it within PRTG, using this Function.

Programmatic deployment of Azure Site Recovery for Azure VMs – that’s the target I started with for a project a little while ago.

There is a large amount of information for Azure Site Recovery on Microsoft’s Docs, however the amount of available code online to programmatically deploy a full setup is very sparse! Much of what Microsoft provides is PowerShell, which isn’t idempotent and doesn’t fit with my current tooling (Terraform and declarative infrastructure-as-code). I did consider Azure CLI, but couldn’t find any references for Site Recovery.

While working on this I came across an Azure Quickstart which is a little incomplete and starts with some good variable definition but quickly devolves into hard-coded values from where it came from; I also discovered a blog post from Pratap Bhaskar  which was useful especially for understanding the Loop mechanism in the template, but it didn’t go far enough for my purposes.

So I spun up a few VMs, manually configured ASR, and did an ARM template export. What I received was a huge amount of properties on the resources that I was sure were relevant to runtime only, not creation. This was also a good reference, but not exactly where I needed to be.

The final piece of the puzzle that got me on my way was the REST API docs for Site Recovery. With this in hand, and the other sources I had at my disposal, I had the references I needed to begin putting together an ARM Template that would configure my environment end-to-end including Recovery Plans with automation runbooks.

There are a lot of design decisions I made when building this to fit my environment, some of which won’t make sense without additional context; most of which I can’t provide. That’s ok, as I hope it at least serves as a reference for “what’s possible” to others who come across it. Here is the overall structure:

• Pre-define and create destination resources like resource groups, virtual networks, and subnets with Terraform
• Deploy ASR for a subset of Virtual Machines, targeting the destination resources
  • Include dependent resources like source-side storage account for cache, and azure automation account in the same region and subscription as the ASR resources
• Deploy a Recovery Plan that provides runbook functionality to configure a Test Failover environment
  • This environment was intended to be completely isolated, to ensure there’s no chance of contamination with prod
  • Current design of my web servers has multiple ip configurations; these need to be replaced
  • Access to the environment is provided through a Jump Host, which needs a known-in-advance IP address

I have documented this output on a GitHub repo called arm-azuresiterecovery

The majority of my time was actually spent cleaning up the template into proper parameters and variables for effective re-use, and then solving all the syntax challenges and typos that come along with that.

There are still some loose ends in what I’ve created, around certain manual steps still required. However as I’m sure is common in the industry, it is good enough to deploy and I must move on – fine-tuning comes later.