Category: Azure

While working on an Azure ARM Template, particularly one which references pre-existing resources rather than ones created within the template itself, I realized I needed to do more to make my code simple with as few parameters as possible.

For example, trying to reference an existing Virtual Machine, I didn’t want to hard-code the  values for it’s name, id, disks, or nics into parameters that have to be supplied as configuration data (in this case, working on a template for Azure Site Recovery).

I knew about the “Reference” function that ARM templates provide, but I was struggling with the thought that I’d have to write up all my references in a functional and deployable template before I could actually validate that I had the syntax correct.

Thankfully I came across this TechNet blog post by Stefan Stranger, which demonstrates a method of testing your functions using an output resource.

I’m not going to repeat his blog post (its a good read!), but I did make some additions to pull out the values of the reference that I wanted. In my example below, I originally started with the “referenceObject” output, reviewed it’s JSON, and then determined the additional attributes to append to get the values that I wanted.

I’ve used the additional ResourceGroup syntax of the reference function in my example because it is likely I’ll be making an ARM deployment to one resource group while referencing resources in another. I also threw in an example of the “resourceId” function, to ensure I was using it properly.

Here’s the PS1 script to use for testing, and below will show the output:

<# Testing Azure ARM Functions https://blogs.technet.microsoft.com/stefan_stranger/2017/08/02/testing-azure-resource-manager-template-functions/ https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-functions-resource#reference #>

#region Variables
$ResourceGroupName = 'source-rg'
$Location = 'WestUS2'
$subscription = "3d22393a"
#endregion

#region Connect to Azure
# Running in Azure Cloud Shell, don't need this
#Add-AzAccount

#Select Azure Subscription
Set-AzContext -SubscriptionId $subscription
#endregion

#region create Resource Group to test Azure Template Functions
If (!(Get-AzResourceGroup -name $ResourceGroupName -Location $Location -ErrorAction SilentlyContinue)) {
    New-AzResourceGroup -Name $ResourceGroupName -Location $Location
}
#endregion

# region Example for if condition
$template = @'
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "vmName": {
            "type": "string",
            "defaultValue": "testVM1"
      },
      "ResourceGroup": {
        "type": "string",
        "defaultValue": "source-rg"
      }
     },
    "variables": { },
    "resources": [ ],
    "outputs": {
      "referenceObject": {
          "type": "object",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01')]"
      },
      "fullReferenceOutput": {
        "type": "object",
        "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01', 'Full')]"
      },
      "resourceid": {
          "type": "string",
          "value": "[resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName'))]"
      },
      "vmSize": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').hardwareprofile.vmSize]"
      },
      "osType": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.osType]"
      },
      "DataDisk1": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].name]"
      },
      "DataDisk2": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[1].name]"
      },
      "nicId": {
          "type": "string",
          "value": "[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').networkProfile.networkInterfaces.id]"
      }
    }
}
'@
#endregion

$template | Out-File -File .\template.json -Force

#region Test ARM Template
Test-AzResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile .\template.json -OutVariable testarmtemplate
#endregion

#region Deploy ARM Template with local Parameter file
$result = (New-AzResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile .\template.json)
$result

#endregion

#Cleanup
Remove-Item .\template.json -Force

Example reference object output:

Example reference strings output:

I’m keeping a list of the common references I’m touching right now as note-taking:

[resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName'))]

[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').hardwareprofile.vmSize]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.osType]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.Name]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.managedDisk.id]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.osDisk.diskSizeGB]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].name]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].managedDisk.id]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').storageProfile.datadisks[0].diskSizeGB]
[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01').networkProfile.networkInterfaces.id]

[reference(resourceId(parameters('ResourceGroup'), 'Microsoft.Compute/virtualmachines', parameters('vmName')), '2019-07-01', 'Full').ResourceGroupName]

If you find yourself needing to move resources in Azure, whether between resource groups or subscriptions, you may find the Portal doesn’t offer a very good validation tool to build a plan for a large migration. While the move itself will perform a validation, if it succeeds it will proceed immediately without further input.

I’m working on a large resource move project, with many inter-connected resources and I needed a method to build a migration plan while having confidence that it will succeed.

A few things to keep in mind about resource moves:

• You cannot move resources between Tenants (Azure Active Directory). You will need to perform a “Change Directory” operation on your source subscription into the new Tenant first, and then do your move validation.
• You cannot move resources with dependencies in different resource groups. So if you have your VMs and NICs in resource group A, and your VNET in resource group B, you’re going to have to consolidate them into a single resource group
• Similarly, if a dependent resource exists but is not moveable, you’re going to have to deal with it. If an Application Gateway exists in a subnet in a VNET, if you try to move the VNET it will fail even if you aren’t trying to move the Application Gateway. It must be deleted instead.

While browsing the Move Resources documentation page, I learned of the “validate move operation” that allows you to make a REST API call to kick-off a validation job, and then another to view the results of that job.

This requires a Bearer access token, so I’m using the same method I did when building my Update Management API call, from this TechNet Gallery post (which has since been updated with Az module support, nice!).

This script assumes that you’re already authenticated to Azure in a PowerShell session, in order to use the method for bearer access token above.

In my example below, I’m gathering all resources within a resource group, and excluding some specific ones from consideration. Instead you could supply individual resource ids (as the Microsoft Doc provides examples of) if you have a simple validation to do.

As the documentation link above describes, you first initiate a validation job, and then you check the status and output of that job afterwards, using the Location URL that is provided in the 202 response. In my example below, I add a sleep timer for 30 seconds, but that may not be enough time to wait.

# https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-move-resources#validate-move

# Provide your source subscription id
$subscriptionid = "sourcesubid"
#Provide the Resource Group Name containing your source resources
$resourcegroup = "source resource group name"
# Provide your target subscription id (could be the same as source)
$targetsub = "targetsubid"
# Provide your target resource group name (it must exist in advance)
$targetrgname = "target resource group name"

# Select the source subscription, so we can dynamically get a list of resources
Select-AzSubscription $subscriptionid
# Get all resources within the source resource group
$resourceids = get-azresource -resourcegroupname $resourcegroup
# Remove resource types that we know are incompatible with the move operation.
# We will delete these resources before migration, so lets remove them from the object array
$resourceids = $resourceids | Where-Object { $_.ResourceType `
        -notlike "*/configurations" ` # Validation throws errors of 'Not top-level resource', because this is contained within Automation Account
        -and $_.ResourceType -notlike "*/runbooks" ` # Validation throws errors of 'Not top-level resource', because this is contained within Automation Account
        -and $_.ResourceType -notlike "*/extensions*" ` # Validation throws errors of 'Not top-level resource', because this is contained within Virtual Machine
        -and $_.ResourceType -notlike "*/applicationgateways*" `
        -and $_.ResourceType -notlike "*/publicipaddresses*" `
        -and $_.ResourceType -notlike "*/vaults*" `
        -and $_.ResourceType -notlike "*/connections*" ` # Excluding Connections because they're part of the VPN GW which will be deleted
        -and $_.ResourceType -notlike "*/localnetworkgateways*" ` # Excluding Local Network Gateways because they're part of the VPN GW which will be deleted
        -and $_.ResourceType -notlike "*/virtualnetworkgateways*" ` # Excluding VPNGW because it depends on Public IP address resource which cannot be move. 
}
#Convert our array to JSON to pass into the body
$resourceids = $resourceids.resourceid | ConvertTo-JSON

# Get our bearer token because we're already signed into Azure
. .\Get-AzCachedAccessToken.ps1 #https://gallery.technet.microsoft.com/scriptcenter/Easily-obtain-AccessToken-3ba6e593
$BearerToken = ('Bearer {0}' -f (Get-AzCachedAccessToken))
$RequestHeader = @{
    "Content-Type"  = "application/json";
    "Authorization" = "$BearerToken"
}
# Build the Body variable, using our $resourceids array
$Body = @"
{
 "resources": $resourceids,
 "targetResourceGroup": "/subscriptions/$targetsub/resourceGroups/$targetrgname"
}
"@

$URI = "https://management.azure.com/subscriptions/$subscriptionid/resourceGroups/$resourcegroup/validateMoveResources?api-version=2019-05-10"
$response = Invoke-WebRequest -Uri $URI -Method POST -body $body -header $RequestHeader

# Wait a while for the validation job to complete
Sleep 30

# From the returned response, find the Location section of the string and extract it
$a = $response.rawcontent
$checkURI = $a.Substring($a.IndexOf('https:'), $a.IndexOf('Retry-After:')-$a.IndexOf('https:'))
$CheckResponse = Invoke-WebRequest -Uri $checkURI -Method Get -header $RequestHeader

Here is what your results might look like:

I haven’t found a good way yet to format this output, as it isn’t contained within the $CheckResponse variable. But you can see in this snippet there’s two errors I’m going to have to deal with:

• Exceeding Core Quota on my target subscription
• Azure Backup is in use on one of the VMs

Right now I’m using this script ad-hoc, and manually consuming the results for output in other ways. It would certainly be possible to take the output of results and format it nicely into a report; this would go well with iterating through all resource groups in a Subscription, to automate the validation process if you’re consistently moving a large amount of resources (i.e. a Pay-as-you-go transition to CSP subscriptions).

I learned something recently about calling an Azure Automation Runbook through PowerShell, particularly around how to pass multi-value entries in a single parameter.

The original concept was that I have a runbook which takes input parameters of an Azure resource group, network security group, and DNS name. It takes this information, resolves the DNS name to an IP address, and then creates a rule inside the network security group for it.

Here’s a simplified example of what I started with:

param(
  [parameter(Mandatory=$false)]
  [string]$nsggroup = "*",
  [parameter(Mandatory=$false)]
  [string]$rulename = "source_dest_port",
  [parameter(Mandatory=$false)]
  [string]$rgname = "*-srv-rg",
  [parameter(Mandatory=$false)]
  [string]$endpointfqdn = "endpoint.vendor.com"
)
$dnstoip = [System.Net.Dns]::GetHostAddresses("$endpointfqdn")
$subscriptions = Get-AzSubscription

$nsgs = Get-AzNetworkSecurityGroup -ResourceGroupName "$rgname" -Name "$nsggroup"
ForEach($nsg in $nsgs) {
    $nsgname = $nsg.name
    $nsg | Add-AzNetworkSecurityRuleConfig -Access Allow -DestinationAddressPrefix $dnstoip.IPAddressToString -DestinationPortRange 443 -Direction Outbound -name $rulename -Priority $priority -SourceAddressPrefix * -SourcePortRange * -Protocol * | Set-AzNetworkSecurityGroup | out-null
}

The problem with this is once it took it to a production environment, I needed more flexibility on where the rule got applied, without having to create many instances of the scheduled runbook linked to parameters.

The key was that I wanted to pass in a list of network security group names, and have the runbook do a ForEach on that list.

The primary change in parameters was to make $nsggroup an [array] type like this:

[array]$nsggroup

['nsg1','nsg2']

This syntax and learning originally came from this doc link which states:

• If a runbook parameter takes an [array] or [object] type, then these must be passed in JSON format in the start runbook dialog.  For example:

• A parameter of type [object] that expects a property bag can be passed in the UI with a JSON string formatted like this: {“StringParam”:”Joe”,”IntParam”:42,”BoolParam”:true}.
• A parameter of type [array] can be input with a JSON string formatted like this: [“Joe”,42,true]

In order to programmatically set up the runbook along with linked schedules, I put my input into an array containing PowerShell objects like this (note the “nsggroup” value on each object):

$inputrules= @(
    [pscustomobject]@{  name = "dnsresolve1"; rulename = "source_dest_port"; ports = @(443); endpointfqdn = "endpoint.vendor.com"; nsggroup = @('nsg1', 'nsg2', 'nsg5', 'nsg6') },
    @{  name = "dnsresolve2"; rulename = "source2_dest_port"; ports = @(443); endpointfqdn = "endpoint2.vendor.com"; nsggroup = @('nsg3', 'nsg4') }
)

In my script that does the programmatic setup, I use this array of objects to do something like this (vastly simplified):

foreach ($object in $inputrules) {
Register-AzAutomationScheduledRunbook -Name $runbookname -ScheduleName "6Hours-$($object.name)" -AutomationAccountName $automationAccountName -ResourceGroupName $resourceGroupName `
        -Parameters @{"nsggroup" = $object.nsggroup; "rulename" = $object.rulename; "endpointfqdn" = $object.endpointfqdn; "ports" = $object.ports; "subscriptionid" = $clientsubscriptionid }
}

You can see by using the syntax in my custom object of an array, I can pass it into the scheduled runbook as a simple parameter, and Azure Automation will take that and apply it successfully.

I’ve finally got a use-case for using Modules in Terraform, and so I’m beginning to dip into testing with them. I’ve got a bunch of Azure resources (VNET, subnet, NSG, etc) that already exist in my Terraform config, and I’m looking to basically duplicate them for a disaster recovery purpose.

In reality, I don’t want to duplicate the Terraform config, because if it ever changes or improves, it would not be efficient to track those changes in multiple spots. So instead I can move the resources that build my VNET into a module, add some variables, and then call the module twice from my main Terraform config – just need to pass in different values for the variables.

module "vnet" {
  source = "./network"
  env = "test1"
  location = "CanadaCentral"
}

module "dr-vnet" {
  source = "./network"
  env = "test2"
  location = "CanadaEast"
}

While reading up on modules though, I noticed a note in the docs:

When refactoring an existing configuration to introduce modules, moving resource blocks between modules causes Terraform to see the new location as an entirely separate resource to the old. Always check the execution plan after performing such actions to ensure that no resources are surprisingly deleted.

Yikes! Even for a simple implementation of a VNET, deleting it means deleting a bunch of dependent resources like virtual machines.

However, there does appear to be a (very manual) way to work around this problem when refactoring into Modules: use the terraform mv command to move location of a resource in the state file.

This relies upon the syntax of referencing resources beginning with module.”module name”. For example, lets say I have this original resource:

resource "azurerm_resource_group" "test-rg" {
  name     = "test-rg"
  location = "CanadaCentral"
}

Assuming I call the module just like my example above, I would use the following terraform mv command:

terraform state mv azurerm_resource_group.test-rg module.vnet.azurerm_resource_group.test-rg

Note the sytax of my destination, being module.”module name”.resourcetype.”resource name”

The downside is that I would need to do this for each original resource that is moving into a Module – perhaps a small price to pay in order to have better managed code, and at least I don’t need to go and find the Azure resource ID like you do with the terraform import command.

This resource group is actually a bad example, because terraform apply will fail when it tries to create two different resource groups with the same name in the same subscription. This could be avoided if you pass in an alternate AzureRM Provider to your Module, using a different Azure subscription.