Author: Jeff Miles

+Jeff Miles

Azure monitoring agents

Jeff Miles Leave a comment

I’ve been having some difficulty sorting out all the monitoring components available within Azure for IaaS resources, and finally sat down to do some dedicated reading on this.

Part of the confusion for me has come from the introduction of the Azure Monitor, Azure Security Center, and the removal of the Operations Management Suite (OMS) branding.

The overview for Azure agents for monitoring describes the following agents as viable for monitoring. I’ve added in additional wording that is used in other areas and ARM templates for these, along with some potential use cases:

Azure Diagnostics Extension

Somewhat a legacy agent at this point (my opinion), it collects Performance counters, System Logs, IIS Logs, and others. These are all stored in a Storage Account. Performance counter information can be sent to Azure Monitor (i.e. Log Analytics), but not system logs and other data sources.

This extension can be deployed with the Set-AzureRmVMDiagnosticsExtension cmdlet.

If using an ARM template, it is referenced as resource name “Microsoft.Insights.VMDiagnosticsSettings” (by default and by recommendation), publisher “Microsoft.Azure.Diagnostics”, and type “IaasDiagnostics”.

Often Microsoft docs refer to this as “guest-level monitoring”.

Frustratingly, this agent is also known by “Microsoft Monitoring Agent Diagnostics“, particularly in Visual Studio.  This nomenclature conflicts with the Log Analytics agent, but doesn’t seem to be very common.

This type of monitoring is needed to enable the full suite of Cloudyn metrics and optimizations, particularly for memory counters. It is called “extended metrics”, and the Microsoft Docs article specifically says that it is not compatible with Log Analytics.

The Azure Diagnostic extension (for VMs) is not to be confused with Azure Monitor Diagnostic logs, which feed data from Azure services into Azure Monitor.

 

Log Analytics Agent

Commonly referred to as the “Microsoft Monitoring Agent” (MMA), this is used to collect data from many different types of sources and enable Azure Monitor solutions in the workspace on IaaS resources. This is the direct integration of a VM into Azure Monitor beyond the default metrics that are provided. It is also used to support the Hybrid Runbook Worker feature of Azure Automation.

The MMA is also a required component for Azure Update Management.

There is a Log Analytics VM extension that installs the Log Analytics agent and registers it with a particular workspace.

If using an ARM template, it is referenced as a resource with publisher “Microsoft.EnterpriseCloud.Monitoring”, type “MicrosoftMonitoringAgent”, and commonly name “OMSExtension”.

If using PowerShell, it can be deployed with this cmdlet: Set-AzureRmVMExtension -ExtensionName “Microsoft.EnterpriseCloud.Monitoring”

Important: the Azure Security Center automatically provisions the Log Analytics agent (MMA) and connects it to a workspace – usually a new one but you can configure it to use an existing one.

 

A question I’m still investigating is whether enabling a VM for Azure Security Center with the MMA will automatically enable appropriate features within Azure Monitor – will the VM become a data source for Log Analytics with the collection of system logs and performance counters?

 

 

 

 

Azure Support Plan discovery

Jeff Miles Leave a comment

I learned some things about Azure Support Plans recently.

A co-worker was tasked with adding an Azure Support Plan to a new subscription that was being created. So they went to portal.azure.com, clicked “Help and Support”, and then from the drop-down selected the new subscription ID.

Then they clicked Change Plan and added a Standard support plan. Upon investigation, this appeared to add a support plan to ALL subscriptions, which was scary. We don’t want to be charged 10x$100USD per month!

Working through a support case with Microsoft, I learned the following about support plans that cleared things up.

A support plan is tied to an Azure Account – the account that subscriptions are created under. The support plan is effectively its own subscription, not tied to individual subscriptions themselves. However, you won’t see it displayed this way in portal.azure.com.

It’s not until you log into https://account.windowsazure.com/Subscriptions that you see it itemized by itself and are able to view billing history for the support plan.

So if I have 10 subscriptions, and they are all created under one Azure Account (you can see this on the “Properties” page of your Subscriptions blade), then only a single support plan is needed.

This also clarifies why Microsoft’s instructions on removing a support plan describe to “Go to the portal, and click ‘cancel subscription'”.

When I thought that this was a per-subscription basis, that made me afraid we’d cancel our actual subscription. Knowing now that this is it’s own subscription, that text makes a lot more sense.

Azure Function learning

Jeff Miles Leave a comment

I’m playing around with an Azure Function that I’m going to eventually call from an Azure Automation runbook (post to come in the near future).

During this process, I learned some pretty key things as I was testing and going along. I’m a little embarrassed to post them publicly, but eventually a non-coder like myself somewhere is going to be trying the same thing and maybe this will help.

Function error-ing out during testing

In my function, I used a ForEach loop to add a string to a List<string> for each instances of a collection. My function would compile just fine but would always error out on this one line, with a really generic error. I only knew it was erroring out on this line because I placed a log output on the next line and it would never reach it.

[Error] Executed 'Functions.HttpTriggerCSharp2'

This was my poor coding skills, not beginning the list properly. I was starting it like this:

List<string> collectedIP = null;
But it really needed to be this to initialize it properly:
List<string> collectedIP = new List<string>();
Without this, adding to the collection isn’t possible. I’m sure that anyone who actually knows C# is shaking their head reading this, but I guess that’s what you get when you learn organically without real training.

Building the query string

I want to pass in a parameter to my function, using the query string. At the same time, I also want to use a function key so that this can’t be run anonymously.

Originally, I was trying this, with the ampersand delineating the second query parameter (after the function key):

Invoke-WebRequest https://appname.azurewebsites.net/api/functionName?code=functionKey&amp;name=www.microsoft.com

In PowerShell, this returned the error:

The ampersand (&amp;) character is not allowed. The &amp; operator is reserved for future use; wrap an ampersand in double quotation marks ("&amp;") to pass it as part of a string.

So I tried this string:

Invoke-WebRequest 'https://appname.azurewebsites.net/api/functionName?code=functionKey"&amp;"name=www.microsoft.com'

But then I received the error:

Invoke-WebRequest : The remote server returned an error: (401) Unauthorized.

Well, I knew it wasn’t unauthorized because it ran properly in the browser. The actual fix was stupidly easy, I hadn’t put any quotes around the whole string – either single or double quotes was fine:

Invoke-WebRequest "https://appname.azurewebsites.net/api/functionName?code=functionKey&amp;name=www.microsoft.com"

TLS Support in PowerShell

When I got to actually testing my function, I tried to call it from PowerShell in this format:

Invoke-WebRequest 'https://appname.azurewebsites.net/api/functionName?code=functionKey'

However, upon doing so I received this error:

iwr : The underlying connection was closed: An unexpected error occurred on a send.

A google search led me to discover that PowerShell by default will attempt to use TLS 1.0 for Invoke-WebRequest, unless you’re using PowerShell Core 6.

My Azure Function uses TLS 1.2 by default and as a minimum. This can be found on the “Platform Settings” page of the Function, under “SSL”:

The solution (at least as a workaround) is to force that session to use TLS 1.2 like this:

[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12

Primary source for this information was here.

 

 

 

Azure Multiple NICs or Static IPs through Terraform and DSC

Jeff Miles Leave a comment

A situation came up where I needed to have two HTTP bindings on port 80 on a web server residing in Azure. This would be 1 binding each on two different IIS sites within a single VM. The creation of this configuration isn’t as simple as one would initially expect, due to some Azure limitations.

There are two options in order to achieve this:

  • Add a secondary virtual network interface to the VM
  • Add a second static IP configuration on the primary virtual network interface of the VM.

For each of these, I wanted to deploy the necessary configuration through Terraform and Desired State Configuration (DSC).

Second IP Configuration

In this scenario, its quite simple to add a second static IP configuration in terraform:

resource "azurerm_network_interface" "testnic" {
  name                = "testnic"
  location            = "${var.location}"
  resource_group_name = "${azurerm_resource_group.test.name}"
  ip_configuration {
    name                          = "ipconfig1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "static"
    private_ip_address            = "10.2.0.5"
    primary                       = true
  }
  ip_configuration {
    name                          = "ipconfig2"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "static"
    private_ip_address            = "10.2.0.6"
  }
}

However, when using static IP addresses like this, Azure requires you to perform configuration within the VM as well. This is necessary for both the primary IP configuration, as well as the secondary.

Here’s how you can configure it within DSC:

Configuration dsctest 
{ 
    Import-DscResource -ModuleName PSDesiredStateConfiguration 
    Import-DscResource -ModuleName xPSDesiredStateConfiguration 
    Import-DscResource -moduleName NetworkingDSC
 
    Node localhost {
        ## Rename VMNic
        NetAdapterName RenameNetAdapter 
        { 
            NewName = "PrimaryNIC"  
            Status = "Up" 
            InterfaceNumber = 1 
        }
 
        DhcpClient DisabledDhcpClient
        {
            State          = 'Disabled'
            InterfaceAlias = 'PrimaryNIC'
            AddressFamily  = 'IPv4'
            DependsOn = "[NetAdapterName]RenameNetAdapter "
        }
 
        IPAddress NewIPv4Address
        {
            #Multiple IPs can be comma delimited like this
            IPAddress      = '10.2.0.5/24','10.2.0.6/24'
            InterfaceAlias = 'PrimaryNIC'
            AddressFamily  = 'IPV4'
            DependsOn = "[NetAdapterName]RenameNetAdapter "
        }
 
        # Skip as source on secondary IP address, in order to prevent DNS registration of this second IP
        IPAddressOption SetSkipAsSource
        {
            IPAddress    = '10.2.0.6'
            SkipAsSource = $true
            DependsOn = "[IPAddress]NewIPv4Address"
        }
        DefaultGatewayAddress SetDefaultGateway 
        { 
            Address = '10.2.0.1' 
            InterfaceAlias = 'PrimaryNIC' 
            AddressFamily = 'IPv4' 
        }  
 
    }
 
}

Both the disable DHCP and DefaultGateway resources are required, otherwise you will lose connectivity to your VM.

Once the “SkipAsSource” runs, this sets the proper priority for the IP addresses, matching the Azure configuration.

 

Second NIC

When adding a second NIC in Terraform, you have to add a property (“primary_network_interface_id”) on the VM resource for specifying the primary NIC.

resource "azurerm_network_interface" "testnic1" {
  name                = "testnic"
  location            = "${var.location}"
  resource_group_name = "${azurerm_resource_group.test.name}"
  ip_configuration {
    name                          = "ipconfig1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "static"
    private_ip_address            = "10.2.0.5"
  }
 
}
resource "azurerm_network_interface" "testnic2" {
  name                = "testnic"
  location            = "${var.location}"
  resource_group_name = "${azurerm_resource_group.test.name}"
  ip_configuration {
    name                          = "ipconfig1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "static"
    private_ip_address            = "10.2.0.6"
  }
 
}
 
resource "azurerm_virtual_machine" "test" {
  name                  = "helloworld"
  location              = "${var.location}"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  network_interface_ids = ["${azurerm_network_interface.testnic1.id}","${azurerm_network_interface.testnic2.id}"]
  primary_network_interface_id = "${azurerm_network_interface.test.id}"
  vm_size               = "Standard_A1"
 
  storage_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2016-Datacenter"
    version   = "latest"
  }
 
  storage_os_disk {
    name          = "myosdisk1"
    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/myosdisk1.vhd"
    caching       = "ReadWrite"
    create_option = "FromImage"
  }
 
  os_profile {
    computer_name  = "helloworld"
    admin_username = "${var.username}"
    admin_password = "${var.password}"
  }
 
  os_profile_windows_config {
    provision_vm_agent = true
    enable_automatic_upgrades = true
  }
 
}

In this scenario, once again Azure requires additional configuration for this to work. The secondary NIC does not get a default gateway and cannot communicate out of it’s subnet.

One way to solve this with DSC is to apply a default route for that interface, with a higher metric than the primary interface. In this case, we don’t need to specify static IP addresses inside the OS through DSC, since there’s only one per virtual NIC.

Configuration dsctest 
{ 
    Import-DscResource -ModuleName PSDesiredStateConfiguration 
    Import-DscResource -ModuleName xPSDesiredStateConfiguration 
    Import-DscResource -moduleName NetworkingDSC
 
    Node localhost {
        ## Rename VMNic(s) 
        NetAdapterName RenameNetAdapter 
        { 
            NewName = "PrimaryNIC" 
            Status = "Up" 
            InterfaceNumber = 1 
            Name = "Ethernet 2"
        }
        NetAdapterName RenameNetAdapter_apps
        { 
            NewName = "SecondaryNIC"  
            Status = "Up" 
            InterfaceNumber = 2
            Name = "Ethernet 3"
        }
 
        Route NetRoute1
        {
            Ensure = 'Present'
            InterfaceAlias = 'SecondaryNIC'
            AddressFamily = 'IPv4'
            DestinationPrefix = '0.0.0.0'
            NextHop = '10.2.0.1'
            RouteMetric = 200
        }
 
    }
 
}

You could also add a Default Gateway resource to the second NIC, although I haven’t specifically tested that.

Generally speaking, I’d rather add a second IP to a single NIC – having two NICs on the same subnet with the same effective default gateway might function, but doesn’t seem to be best practice to me.

Windows 10 insider preview failed to install

Jeff Miles 3 Comments

I’m running Windows 10 insider preview on my basement computer at home, and sometime in the middle of summer it started failing to update to the latest release. I finally made some time to troubleshoot this last night and got it working.

During the install, it would get about 40% of the way through, and then fail with this error:

Windows could not configure one or more system components

After some sleuthing I discovered the log file for the upgrade could be found here: C:\Windows\Panther\NewOs\Panther, and I looked at the “setuperror.log” file.

In this, there were a couple of key errors noted:

0xd0000034 Failed to add user mode driver [%SystemRoot%\system32\DRIVERS\UMDF\uicciso.dll]

Failure while calling IPreApply->PreApply for Plugin={ServerPath="Microsoft-Windows-IIS-RM\iismig.dll"

Generic Command ErrorCode: 80004005 Executable: iissetup.exe ExitCode: 13 Phase: 38 Mode: Install (first install) Component: Microsoft-Windows-IIS-SharedLibraries-GC

I did come across a search hit when looking for the “uicciso.dll” error that spoke about IIS install failing with a Windows 10 update, and those two things seemed to correlate with the errors I was seeing in the logs.

I ran this DISM command to see the additional Win10 features that were installed, and noted a whole bunch that I don’t ever recall having put on manually, and certainly weren’t needed:

 dism /online /get-features /format:table

I collected a bunch, turned it into a removal command, ran them and restarted:

dism /online /disable-feature /FeatureName:SMB1Protocol-Server
dism /online /disable-feature /FeatureName:SMB1Protocol
dism /online /disable-feature /FeatureName:MSMQ-Server
dism /online /disable-feature /FeatureName:MSMQ-Container
dism /online /disable-feature /FeatureName:WCF-Services45
dism /online /disable-feature /FeatureName:WCF-TCP-Activation45 
dism /online /disable-feature /FeatureName:WCF-Pipe-Activation45 
dism /online /disable-feature /FeatureName:WCF-MSMQ-Activation45 
dism /online /disable-feature /FeatureName:WCF-TCP-PortSharing45 
dism /online /disable-feature /FeatureName:WAS-ConfigurationAPI
dism /online /disable-feature /FeatureName:WAS-WindowsActivationService 
dism /online /disable-feature /FeatureName:WAS-ProcessModel 
dism /online /disable-feature /FeatureName:IIS-RequestFiltering
dism /online /disable-feature /FeatureName:IIS-Security
dism /online /disable-feature /FeatureName:IIS-ApplicationDevelopment 
dism /online /disable-feature /FeatureName:IIS-NetFxExtensibility45
dism /online /disable-feature /FeatureName:IIS-WebServerRole 
dism /online /disable-feature /FeatureName:IIS-WebServer 
dism /online /disable-feature /FeatureName:NetFx4-AdvSrvs 
dism /online /disable-feature /FeatureName:NetFx4Extended-ASPNET45

After the restart I let the upgrade run, and it completed successfully!