Category: Code

Azure DevOps Pipeline or Release

Jeff Miles Leave a comment

After a significant amount of time envisioning a DevOps pipeline, hearing about them, reading about them on Twitter, and watching other’s create them I finally made some time to get some hands on experience. I chose to work my Azure DSC configuration deployment into a pipeline, which I’ll blog about sometime in the near future.

First, I ran into an almost immediate question: what is the difference between a Pipeline, and a Release?

From previous reading, I had learned that Continuous Integration (CI) was related to Build pipelines, which the first item in the menu was previously called. Then subsequently Continuous Deployment (CD) was a Release pipeline and separated to its own menu item.

However when I started playing around this week, what I found was a very confusing similarity between the two.

I could make a Pipeline with YAML or a classic editor, both including tasks and steps for build and releases, and when I went to save it would store a YAML file in my connected repository. This covers CI/CD in full and left me wondering where Releases fit.

I could also create a Release pipeline, in a graphical format only, with an Artifacts and a Stages section that seemingly had no connection to a repository other than creating a runtime artifact from one.

Reading the excellent Azure Docs for “Use Azure Pipelines“, it shed a little bit more light on this:

  1. Define pipelines using YAML syntax
    2. “Your code is now updated, built, tested, and packaged. It can be deployed to any target.”
  2. Define pipelines using the Classic interface
    2. “Use the Azure Pipelines classic editor to create and configure your build and release pipelines”

In short, the concept of build AND release pipelines are now effectively the Classic mode of Azure Pipelines, with the new hotness being a single pipeline, created through YAML and stored as code within a branch in your repository.

I understand why Microsoft has not fully removed the classic method, but it does take some time to understand for someone like me jumping in during the middle of a transition.

As it stands right now on the Feature Availability table, it looks like Deployment Groups and Gates are the only features that that YAML pipelines are not able to do compared to the classic Release pipeline.

Update SCCM Maintenance Window through PowerShell

Jeff Miles Leave a comment

Sometimes trying to stay bleeding edge is tough – today’s example is when you want to install updates through SCCM a day or two after Patch Tuesday, particularly using Maintenance Windows to allow restarts within a specific time frame.

We use automatic deployment rules to update a Software Update Group every Patch Tuesday – scheduling this is easy because its always the second Tuesday of a month.

But I want the updates to install on Wednesday night, or Thursday morning. This ensures strict compliance requirements can be met, but allows 24 hours for testing. Can’t just schedule the install and restarts for “second Wednesday of the month” though, because if the first of the month is a Wednesday (like this month) then our actual install date happens to be the THIRD Wednesday of the month.

Previously we solved this by manually updating Maintenance Window schedules every month, painstaking selecting the right date and hoping we didn’t mess it up.

PowerShell took that risk away:

 

SCCM_UpdateMaintenanceWindowSchedule.ps1 – on GitHub

 

See the comments on the script for details of how it works. As a very brief overview:

I found Tim Curwick’s method of calculating Patch Tuesday, and used that in my script to reliably calculate my Wednesday or Thursday install date.

This script runs as System from an SCCM server on the 1st of every month. It performs the calculation, updates maintenance windows on specific Collections, and outputs a log to file and emails results.

 

Azure Site Recovery – ARM template

Jeff Miles Leave a comment

Programmatic deployment of Azure Site Recovery for Azure VMs – that’s the target I started with for a project a little while ago.

There is a large amount of information for Azure Site Recovery on Microsoft’s Docs, however the amount of available code online to programmatically deploy a full setup is very sparse! Much of what Microsoft provides is PowerShell, which isn’t idempotent and doesn’t fit with my current tooling (Terraform and declarative infrastructure-as-code). I did consider Azure CLI, but couldn’t find any references for Site Recovery.

While working on this I came across an Azure Quickstart which is a little incomplete and starts with some good variable definition but quickly devolves into hard-coded values from where it came from; I also discovered a blog post from Pratap Bhaskar  which was useful especially for understanding the Loop mechanism in the template, but it didn’t go far enough for my purposes.

So I spun up a few VMs, manually configured ASR, and did an ARM template export. What I received was a huge amount of properties on the resources that I was sure were relevant to runtime only, not creation. This was also a good reference, but not exactly where I needed to be.

The final piece of the puzzle that got me on my way was the REST API docs for Site Recovery. With this in hand, and the other sources I had at my disposal, I had the references I needed to begin putting together an ARM Template that would configure my environment end-to-end including Recovery Plans with automation runbooks.

There are a lot of design decisions I made when building this to fit my environment, some of which won’t make sense without additional context; most of which I can’t provide. That’s ok, as I hope it at least serves as a reference for “what’s possible” to others who come across it. Here is the overall structure:

  • Pre-define and create destination resources like resource groups, virtual networks, and subnets with Terraform
  • Deploy ASR for a subset of Virtual Machines, targeting the destination resources
    • Include dependent resources like source-side storage account for cache, and azure automation account in the same region and subscription as the ASR resources
  • Deploy a Recovery Plan that provides runbook functionality to configure a Test Failover environment
    • This environment was intended to be completely isolated, to ensure there’s no chance of contamination with prod
    • Current design of my web servers has multiple ip configurations; these need to be replaced
    • Access to the environment is provided through a Jump Host, which needs a known-in-advance IP address

 

I have documented this output on a GitHub repo called arm-azuresiterecovery

 

 

The majority of my time was actually spent cleaning up the template into proper parameters and variables for effective re-use, and then solving all the syntax challenges and typos that come along with that.

 

There are still some loose ends in what I’ve created, around certain manual steps still required. However as I’m sure is common in the industry, it is good enough to deploy and I must move on – fine-tuning comes later.

Testing an ARM Template function

Jeff Miles Leave a comment

While working on an Azure ARM Template, particularly one which references pre-existing resources rather than ones created within the template itself, I realized I needed to do more to make my code simple with as few parameters as possible.

For example, trying to reference an existing Virtual Machine, I didn’t want to hard-code the  values for it’s name, id, disks, or nics into parameters that have to be supplied as configuration data (in this case, working on a template for Azure Site Recovery).

I knew about the “Reference” function that ARM templates provide, but I was struggling with the thought that I’d have to write up all my references in a functional and deployable template before I could actually validate that I had the syntax correct.

Thankfully I came across this TechNet blog post by Stefan Stranger, which demonstrates a method of testing your functions using an output resource.

I’m not going to repeat his blog post (its a good read!), but I did make some additions to pull out the values of the reference that I wanted. In my example below, I originally started with the “referenceObject” output, reviewed it’s JSON, and then determined the additional attributes to append to get the values that I wanted.

I’ve used the additional ResourceGroup syntax of the reference function in my example because it is likely I’ll be making an ARM deployment to one resource group while referencing resources in another. I also threw in an example of the “resourceId” function, to ensure I was using it properly.

Here’s the PS1 script to use for testing, and below will show the output:

<# Testing Azure ARM Functions https://blogs.technet.microsoft.com/stefan_stranger/2017/08/02/testing-azure-resource-manager-template-functions/ https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-functions-resource#reference #>
 
#region Variables
$ResourceGroupName = 'source-rg'
$Location = 'WestUS2'
$subscription = "3d22393a"
#endregion
 
#region Connect to Azure
# Running in Azure Cloud Shell, don't need this
#Add-AzAccount
 
#Select Azure Subscription
Set-AzContext -SubscriptionId $subscription
#endregion
 
#region create Resource Group to test Azure Template Functions
If (!(Get-AzResourceGroup -name $ResourceGroupName -Location $Location -ErrorAction SilentlyContinue)) {
    New-AzResourceGroup -Name $ResourceGroupName -Location $Location
}
#endregion
 
# region Example for if condition
$template = @'
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "vmName": {
            "type": "string",
            "defaultValue": "testVM1"
      },
      "ResourceGroup": {
        "type": "string",
        "defaultValue": "source-rg"
      }
     },
    "variables": { },
    "resources": [ ],
    "outputs": {
      "referenceObject": {
          "type": "object",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01')]"
      },
      "fullReferenceOutput": {
        "type": "object",
        "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01', 'Full')]"
      },
      "resourceid": {
          "type": "string",
          "value": "[resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName'))]"
      },
      "vmSize": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').hardwareprofile.vmSize]"
      },
      "osType": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.osType]"
      },
      "DataDisk1": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].name]"
      },
      "DataDisk2": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[1].name]"
      },
      "nicId": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').networkProfile.networkInterfaces.id]"
      }
    }
}
'@
#endregion
 
$template | Out-File -File .\template.json -Force
 
#region Test ARM Template
Test-AzResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile .\template.json -OutVariable testarmtemplate
#endregion
 
#region Deploy ARM Template with local Parameter file
$result = (New-AzResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile .\template.json)
$result
 
#endregion
 
#Cleanup
Remove-Item .\template.json -Force

Example reference object output:

Example reference strings output:

I’m keeping a list of the common references I’m touching right now as note-taking:

[resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName'))]
 
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').hardwareprofile.vmSize]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.osType]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.Name]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.managedDisk.id]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.diskSizeGB]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].name]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].managedDisk.id]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].diskSizeGB]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').networkProfile.networkInterfaces.id]
 
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01', 'Full').ResourceGroupName]

 

Validate Azure resource move with REST API

Jeff Miles 6 Comments

If you find yourself needing to move resources in Azure, whether between resource groups or subscriptions, you may find the Portal doesn’t offer a very good validation tool to build a plan for a large migration. While the move itself will perform a validation, if it succeeds it will proceed immediately without further input.

I’m working on a large resource move project, with many inter-connected resources and I needed a method to build a migration plan while having confidence that it will succeed.

A few things to keep in mind about resource moves:

  • You cannot move resources between Tenants (Azure Active Directory). You will need to perform a “Change Directory” operation on your source subscription into the new Tenant first, and then do your move validation.
  • You cannot move resources with dependencies in different resource groups. So if you have your VMs and NICs in resource group A, and your VNET in resource group B, you’re going to have to consolidate them into a single resource group
  • Similarly, if a dependent resource exists but is not moveable, you’re going to have to deal with it. If an Application Gateway exists in a subnet in a VNET, if you try to move the VNET it will fail even if you aren’t trying to move the Application Gateway. It must be deleted instead.

While browsing the Move Resources documentation page, I learned of the “validate move operation” that allows you to make a REST API call to kick-off a validation job, and then another to view the results of that job.

This requires a Bearer access token, so I’m using the same method I did when building my Update Management API call, from this TechNet Gallery post (which has since been updated with Az module support, nice!).

This script assumes that you’re already authenticated to Azure in a PowerShell session, in order to use the method for bearer access token above.

In my example below, I’m gathering all resources within a resource group, and excluding some specific ones from consideration. Instead you could supply individual resource ids (as the Microsoft Doc provides examples of) if you have a simple validation to do.

As the documentation link above describes, you first initiate a validation job, and then you check the status and output of that job afterwards, using the Location URL that is provided in the 202 response. In my example below, I add a sleep timer for 30 seconds, but that may not be enough time to wait.

# https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-move-resources#validate-move
 
# Provide your source subscription id
$subscriptionid = "sourcesubid"
#Provide the Resource Group Name containing your source resources
$resourcegroup = "source resource group name"
# Provide your target subscription id (could be the same as source)
$targetsub = "targetsubid"
# Provide your target resource group name (it must exist in advance)
$targetrgname = "target resource group name"
 
# Select the source subscription, so we can dynamically get a list of resources
Select-AzSubscription $subscriptionid
# Get all resources within the source resource group
$resourceids = get-azresource -resourcegroupname $resourcegroup
# Remove resource types that we know are incompatible with the move operation.
# We will delete these resources before migration, so lets remove them from the object array
$resourceids = $resourceids | Where-Object { $_.ResourceType `
        -notlike "*/configurations" ` # Validation throws errors of 'Not top-level resource', because this is contained within Automation Account
        -and $_.ResourceType -notlike "*/runbooks" ` # Validation throws errors of 'Not top-level resource', because this is contained within Automation Account
        -and $_.ResourceType -notlike "*/extensions*" ` # Validation throws errors of 'Not top-level resource', because this is contained within Virtual Machine
        -and $_.ResourceType -notlike "*/applicationgateways*" `
        -and $_.ResourceType -notlike "*/publicipaddresses*" `
        -and $_.ResourceType -notlike "*/vaults*" `
        -and $_.ResourceType -notlike "*/connections*" ` # Excluding Connections because they're part of the VPN GW which will be deleted
        -and $_.ResourceType -notlike "*/localnetworkgateways*" ` # Excluding Local Network Gateways because they're part of the VPN GW which will be deleted
        -and $_.ResourceType -notlike "*/virtualnetworkgateways*" ` # Excluding VPNGW because it depends on Public IP address resource which cannot be move. 
}
#Convert our array to JSON to pass into the body
$resourceids = $resourceids.resourceid | ConvertTo-JSON
 
# Get our bearer token because we're already signed into Azure
. .\Get-AzCachedAccessToken.ps1 #https://gallery.technet.microsoft.com/scriptcenter/Easily-obtain-AccessToken-3ba6e593
$BearerToken = ('Bearer {0}' -f (Get-AzCachedAccessToken))
$RequestHeader = @{
    "Content-Type"  = "application/json";
    "Authorization" = "$BearerToken"
}
# Build the Body variable, using our $resourceids array
$Body = @"
{
 "resources": $resourceids,
 "targetResourceGroup": "/subscriptions/$targetsub/resourceGroups/$targetrgname"
}
"@
 
$URI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourcegroup/validateMoveResources?api-version=2019-05-10"
$response = Invoke-WebRequest -Uri $URI -Method POST -body $body -header $RequestHeader
 
# Wait a while for the validation job to complete
Sleep 30
 
# From the returned response, find the Location section of the string and extract it
$a = $response.rawcontent
$checkURI = $a.Substring($a.IndexOf('https:'), $a.IndexOf('Retry-After:')-$a.IndexOf('https:'))
$CheckResponse = Invoke-WebRequest -Uri $checkURI -Method Get -header $RequestHeader

Here is what your results might look like:

I haven’t found a good way yet to format this output, as it isn’t contained within the $CheckResponse variable. But you can see in this snippet there’s two errors I’m going to have to deal with:

  • Exceeding Core Quota on my target subscription
  • Azure Backup is in use on one of the VMs

Right now I’m using this script ad-hoc, and manually consuming the results for output in other ways. It would certainly be possible to take the output of results and format it nicely into a report; this would go well with iterating through all resource groups in a Subscription, to automate the validation process if you’re consistently moving a large amount of resources (i.e. a Pay-as-you-go transition to CSP subscriptions).