Update SCCM Maintenance Window through PowerShell

Sometimes trying to stay bleeding edge is tough – today’s example is when you want to install updates through SCCM a day or two after Patch Tuesday, particularly using Maintenance Windows to allow restarts within a specific time frame.

We use automatic deployment rules to update a Software Update Group every Patch Tuesday – scheduling this is easy because its always the second Tuesday of a month.

But I want the updates to install on Wednesday night, or Thursday morning. This ensures strict compliance requirements can be met, but allows 24 hours for testing. Can’t just schedule the install and restarts for “second Wednesday of the month” though, because if the first of the month is a Wednesday (like this month) then our actual install date happens to be the THIRD Wednesday of the month.

Previously we solved this by manually updating Maintenance Window schedules every month, painstaking selecting the right date and hoping we didn’t mess it up.

PowerShell took that risk away:

 

SCCM_UpdateMaintenanceWindowSchedule.ps1 – on GitHub

 

See the comments on the script for details of how it works. As a very brief overview:

I found Tim Curwick’s method of calculating Patch Tuesday, and used that in my script to reliably calculate my Wednesday or Thursday install date.

This script runs as System from an SCCM server on the 1st of every month. It performs the calculation, updates maintenance windows on specific Collections, and outputs a log to file and emails results.

 

Azure Site Recovery – ARM template

Programmatic deployment of Azure Site Recovery for Azure VMs – that’s the target I started with for a project a little while ago.

There is a large amount of information for Azure Site Recovery on Microsoft’s Docs, however the amount of available code online to programmatically deploy a full setup is very sparse! Much of what Microsoft provides is PowerShell, which isn’t idempotent and doesn’t fit with my current tooling (Terraform and declarative infrastructure-as-code). I did consider Azure CLI, but couldn’t find any references for Site Recovery.

While working on this I came across an Azure Quickstart which is a little incomplete and starts with some good variable definition but quickly devolves into hard-coded values from where it came from; I also discovered a blog post from Pratap Bhaskar  which was useful especially for understanding the Loop mechanism in the template, but it didn’t go far enough for my purposes.

So I spun up a few VMs, manually configured ASR, and did an ARM template export. What I received was a huge amount of properties on the resources that I was sure were relevant to runtime only, not creation. This was also a good reference, but not exactly where I needed to be.

The final piece of the puzzle that got me on my way was the REST API docs for Site Recovery. With this in hand, and the other sources I had at my disposal, I had the references I needed to begin putting together an ARM Template that would configure my environment end-to-end including Recovery Plans with automation runbooks.

There are a lot of design decisions I made when building this to fit my environment, some of which won’t make sense without additional context; most of which I can’t provide. That’s ok, as I hope it at least serves as a reference for “what’s possible” to others who come across it. Here is the overall structure:

  • Pre-define and create destination resources like resource groups, virtual networks, and subnets with Terraform
  • Deploy ASR for a subset of Virtual Machines, targeting the destination resources
    • Include dependent resources like source-side storage account for cache, and azure automation account in the same region and subscription as the ASR resources
  • Deploy a Recovery Plan that provides runbook functionality to configure a Test Failover environment
    • This environment was intended to be completely isolated, to ensure there’s no chance of contamination with prod
    • Current design of my web servers has multiple ip configurations; these need to be replaced
    • Access to the environment is provided through a Jump Host, which needs a known-in-advance IP address

 

I have documented this output on a GitHub repo called arm-azuresiterecovery

 

 

The majority of my time was actually spent cleaning up the template into proper parameters and variables for effective re-use, and then solving all the syntax challenges and typos that come along with that.

 

There are still some loose ends in what I’ve created, around certain manual steps still required. However as I’m sure is common in the industry, it is good enough to deploy and I must move on – fine-tuning comes later.

Invoke-AzVMRunCommand log output

While working with the Invoke-AzVmRunCommand cmdlet, I encountered a problem, receiving the following error:

Invoke-AzVMRunCommand : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure w
hen processing extension 'RunCommandWindows'. Error message: "Finished executing command".'

I’m attempting to push a DSC configuration into a VM and run it, and although the PowerShell runs successful when run from inside the VM, I still see this error.

I came across a github issue with a helpful tip: the log output of Invoke-AzVmRunCommand can be viewed from this path:

C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\<version>\Status

This helped me determine what the problem was and how to solve it.

Use Azure Function to start VMs

A use-case came up recently to provide a little bit of self-service to some users with an occasionally used Virtual Machine in Azure. I have configured Auto-Shutdown on the VM, but want an ability for users to turn the VM on at their convenience.

Traditionally, this would be done by using Role-Based Access Control (RBAC) over the resource group or VM to allow users to enter portal.azure.com and start the VM with the GUI.

However I wanted to streamline this process without having to manage individual permissions, due to the low-risk of the resource. To do so, I’m using an Azure Function (v2. PowerShell) to start all the VMs in a resource group.

 

First create your function app (Microsoft Docs link) as a PowerShell app – this is still in preview as a Function V2 stack, but it is effective.

The next thing I did was create a Managed Identity in my directory for this Function app. I wanted to ensure that the code the Function runs is able to communicate with the Azure Resource Manager, but did not want to create and manage a dedicated Service Principal.

Within the Function App Platform Features section, I created a Managed Identity for it to authenticate against my directory to access resources:

Go to “Identity”:

Switch “System Assigned” to ON and click Save:

With the Managed Identity now created, you can go to your Subscription or Resource Group, and add a Role assignment under “Access control (IAM)”:

Lastly, I developed the following code to place into the function (github gist):

using namespace System.Net
 
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
 
# Interact with query parameters or the body of the request.
$rgname = $Request.Query.resourcegroup
if (-not $rgname) {
    $rgname = $Request.Body.resourcegroup
}
$action = $Request.Query.action
if (-not $action) {
    $action = $Request.Body.action
}
$subscriptionid = $Request.Query.subscriptionid
if (-not $subscriptionid) {
    $subscriptionid = $Request.Body.subscriptionid
}
$tenantid = $Request.Query.tenantid
if (-not $tenantid) {
    $tenantid = $Request.Body.tenantid
}
 
#Proceed if all request body parameters are found
if ($rgname -and $action -and $subscriptionid -and $tenantid) {
    $status = [HttpStatusCode]::OK
    Select-AzSubscription -SubscriptionID $subid -TenantID $tenantid
    if ($action -ceq "get"){
        $body = Get-AzVM -ResourceGroupName $rgname -status | select-object Name,PowerState
    }
    if ($action -ceq "start"){
        $body = $action
        $body = Get-AzVM -ResourceGroupName $rgname | Start-AzVM
    }
}
else {
    $status = [HttpStatusCode]::BadRequest
    $body = "Please pass a name on the query string or in the request body."
}
 
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]@{
    StatusCode = $status
    Body = $body
})

To provide secure access, I left my Function app with anonymous authentication, but added a new Function Key which I could use and control when calling this function. This is found under the “Manage” options of the function:

To test, I called the function externally like this, passing in my request body parameters and using the Function Key that was generated. You can grab the URL for your function right near the top “Save” button when you’re editing it in the Portal.

$Body = @"
{
    "resourcegroup": "source-rg",
    "action": "start",
    "subscriptionid": "SUBID",
    "tenantid": "TENANTID"
}
"@
$URI = "https://hostname.azurewebsites.net/api/startVMs?code=FUNCTIONKEY"
Invoke-RestMethod -Uri $URI -Method Post -body $body

 

If I run this with the “get” action, then the Function will return the status of each VM in the resource group:

SCVMM VM Network dependency for Hyper-V replica

Encountered a weird issue today with SCVMM. I can’t provide screenshots since I’d have to white out object names, making it without purpose.

I wanted to delete a VM Network, however when I looked at the dependencies on it, I saw a Virtual Machine.

Looking at the Virtual Machine itself, all of its network configuration was tied to a different VM Network. For a while I really couldn’t figure out where this reference was coming from.

Then I realized that this VM has a Hyper-V Replica. SCVMM will show that replica, but you cannot open the properties of it in the GUI. However, you can query it with PowerShell. I was able to do so with this:

 

$VmName = "vm1"
$VM = Get-SCVirtualMachine -Name $VMName -VMHost "destination hyper-v host"
$NIC = Get-SCVirtualNetworkAdapter -VM $VM | Out-Gridview -PassThru -Title 'Select VMs vNIC'

This allowed me to view the properties of the NIC that was attached to my replica. SCVMM stores network information (not Hyper-V network info) in it’s database, and it appears that the original config was changed on my source VM, but that never made its way into the replica.

I was able to update this by adding the following two PowerShell lines:

# Pop up an OutGrid to choose the VM network you want to attach the NIC to
$VMNetwork = Get-SCVMNetwork | Out-Gridview -PassThru -Title 'Select VM Network'
Set-SCVirtualNetworkAdapter -VirtualNetworkAdapter $VM.VirtualNetworkAdapters[($NIC.SlotID)] -VMNetwork $VMNetwork

Now, I did receive an error from this last Set command, that “VMM failed to allocate a MAC address to network adapter”. However, after re-runnining the “Get-SCVirtualNetworkAdapter” I could confirm the Logical Network and VM Network were updated appropriately, and my dependency was no longer there.

I validated that this did not appear to affect the health of Hyper-V replication either.

Update – after performing the actions above, the dependency on the VMNetwork was still there when I tried to delete (but didn’t appear in the GUI). At the risk of breaking replication, I decided to try updating the VMM database directly, rather than try to figure out the best method.

I figured my Hyper-V replica VM nic was still tied to the VMNetwork in the database, despite the command I used above. So I used SQL Server Management Studio to query the tables like this:

-- Find VM Network ID
DECLARE @var VARCHAR(MAX)
SELECT @var = id FROM [VirtualManagerDB].[dbo].[tbl_NetMan_VMNetwork] WHERE Name = 'vm network name'
-- Use output from previous command to find matching VM nics
SELECT * FROM tbl_WLC_VNetworkAdapter WHERE (vmnetworkid = @var)

This produced two rows, one for my source VM and the other for my replica. I updated the VMSubnetID and VMNetworkID columns on my replica to match the source VM.

This allowed me to successfully delete the VM Network.