Azure Site Recovery learnings – Azure-To-Azure

A few things I’ve learned about Azure Site Recovery (ASR) recently, while doing an Azure-to-Azure DR design – some quite surprising:

  • Both your Recovery Services Vault AND it’s resource group must be located in a different region than the source
  • You need a cache storage account in the source region, however the staged incremental data will have negligible cost according to Microsoft
  • For source VMs utilizing Managed Disks, ASR will create destination Managed Disks and you will be charged for provisioned storage size, not consumed size. This differs from using unmanaged disks or for on-premises ASR, where consumed size is stored as page blobs in the destination storage account. I can’t find a Microsoft Doc link that specifically outlines this.
  • Egress bandwidth compression is estimated at about 60%, according to this blog post: Know exactly how much it will cost for enabling DR to your Azure VMs
  • VM Extensions are not replicated to a failover VM, and need to be manually installed: Doc Link
  • Secondary IP addresses are not replicated! These will need to be re-added through a Post-Failover task: Doc Link

 

Some things I still need to research and test are:

  • What happens if you perform a failover (or test failover) to a VM that is reporting into Log Analytics and Azure Automation (DSC, Update Management)? Will it seamlessly continue these operations, even when the VM extension no longer exists?
  • What happens to Azure Backup? Will the test failover VM using the Azure Backup Agent try to send backup data cross-region to the source Recovery Services Vault if the schedule time is triggered?

SCVMM VM Network dependency for Hyper-V replica

Encountered a weird issue today with SCVMM. I can’t provide screenshots since I’d have to white out object names, making it without purpose.

I wanted to delete a VM Network, however when I looked at the dependencies on it, I saw a Virtual Machine.

Looking at the Virtual Machine itself, all of its network configuration was tied to a different VM Network. For a while I really couldn’t figure out where this reference was coming from.

Then I realized that this VM has a Hyper-V Replica. SCVMM will show that replica, but you cannot open the properties of it in the GUI. However, you can query it with PowerShell. I was able to do so with this:

 

$VmName = "vm1"
$VM = Get-SCVirtualMachine -Name $VMName -VMHost "destination hyper-v host"
$NIC = Get-SCVirtualNetworkAdapter -VM $VM | Out-Gridview -PassThru -Title 'Select VMs vNIC'

This allowed me to view the properties of the NIC that was attached to my replica. SCVMM stores network information (not Hyper-V network info) in it’s database, and it appears that the original config was changed on my source VM, but that never made its way into the replica.

I was able to update this by adding the following two PowerShell lines:

# Pop up an OutGrid to choose the VM network you want to attach the NIC to
$VMNetwork = Get-SCVMNetwork | Out-Gridview -PassThru -Title 'Select VM Network'
Set-SCVirtualNetworkAdapter -VirtualNetworkAdapter $VM.VirtualNetworkAdapters[($NIC.SlotID)] -VMNetwork $VMNetwork

Now, I did receive an error from this last Set command, that “VMM failed to allocate a MAC address to network adapter”. However, after re-runnining the “Get-SCVirtualNetworkAdapter” I could confirm the Logical Network and VM Network were updated appropriately, and my dependency was no longer there.

I validated that this did not appear to affect the health of Hyper-V replication either.

Update – after performing the actions above, the dependency on the VMNetwork was still there when I tried to delete (but didn’t appear in the GUI). At the risk of breaking replication, I decided to try updating the VMM database directly, rather than try to figure out the best method.

I figured my Hyper-V replica VM nic was still tied to the VMNetwork in the database, despite the command I used above. So I used SQL Server Management Studio to query the tables like this:

-- Find VM Network ID
DECLARE @var VARCHAR(MAX)
SELECT @var = id FROM [VirtualManagerDB].[dbo].[tbl_NetMan_VMNetwork] WHERE Name = 'vm network name'
-- Use output from previous command to find matching VM nics
SELECT * FROM tbl_WLC_VNetworkAdapter WHERE (vmnetworkid = @var)

This produced two rows, one for my source VM and the other for my replica. I updated the VMSubnetID and VMNetworkID columns on my replica to match the source VM.

This allowed me to successfully delete the VM Network.

 

Job I want to have

I was looking through my Google Drive recently, doing some cleanup and pruning. I came across a document I had created in June 2016, called “Job I want to have”.

I don’t remember creating this document at all. It’s contents are a job posting for an “Infrastructure Technology Analyst”, without any kind of reference to the original company.

Here’s a snippet of what it looked like:

In June 2016 I was feeling stagnant; lack of motivation, lack of direction. I looked at this posting and thought that it was a huge stretch, and that it may be so difficult to actually achieve enough skill to be able to fill a position like this.

Now I’m reflecting on this, and realize that I have this job – I do all of these things right now, and it didn’t take a monumental effort. It wasn’t hours and hours of study time, or money for certifications and courses. I’m not saying I didn’t have to work hard to learn, or that it was random chance that put me here. It was certainly time spent learning, but by doing; by embracing the challenges as I faced them and learning how to solve them with the focus of a goal in mind.

What it really required was for me to step outside of where I was comfortable, embrace the fear of uncertainty, and try. Try something new and something different; try a chance that the grass could actually be greener.

I’m glad I came across this because I needed a refresh in my mind of what my goal was and understanding that I have achieved it. I needed a reminder that the core of what I’m doing now is still fun and drives me to have the kind of career I want to have.

Perhaps its nearing time to set my sights on something a little scary again.

Azure NSG Update issue with PowerShell

Today I’m working on creating and updating Azure Network Security Group rules from the Az PowerShell module. However I hit a bit of a bump.

I’m trying to do something simple like add 1 rule to an existing NSG that has other rules:

# Find the security group where it matches a variable
$nsgs = Get-AzNetworkSecurityGroup -ResourceGroupName $rgname | where-object { $_.Name -like "$nsgsuffix" }
# For the selected security group, add a rule, and then apply it
$nsgs | add-aznetworksecurityruleconfig -Access Allow -DestinationAddressPrefix $ips -DestinationPortRange 443 `
 -Direction Outbound -name allowOut-To3rdParty_443 -Priority 400 `
-SourceAddressPrefix * -SourcePortRange * -Protocol * | Set-AzNetworkSecurityGroup

This NSG and other rules were previously deployed through TerraForm, and I want to add an “out-of-band” rule that isn’t tracked in the Terraform state.

When I run these commands, I get this error:

Required security rule parameters are missing for security rule with Id:
Security rule must specify either DestinationPortRange or DestinationPortRanges.

That’s super strange; when I look at that rule in the Portal, it has ports listed:

So I check my $nsgs variable that was previously populated, looking specifically for the SecurityRules property:

$nsgs.SecurityRules

What I find is that for any rule that the portal displays having multiple ports, it lists them just fine (see Destination Port Range property below):

Name                                 : test_test.test
Id                                   : /subscriptions/1/resourceGroups/rg/providers/Microsoft.Network
/networkSecurityGroups/db-nsg/securityRules/test_test.test
Etag                                 : W/"3a98e199-3bf6-4308-806c-8d84bc03723e"
ProvisioningState                    : Succeeded
Description                          :
Protocol                             : *
SourcePortRange                      : {*}
DestinationPortRange                 : {123, 456, 789}
SourceAddressPrefix                  : {10.8.1.}
DestinationAddressPrefix             : {10.8.1.}
SourceApplicationSecurityGroups      : []
DestinationApplicationSecurityGroups : []
Access                               : Allow
Priority                             : 109
Direction                            : Inbound

But for any rule that only has a SINGLE port shown in the portal, it is blank for that property:

Name                                 : web_.db
Id                                   : /subscriptions/1/resourceGroups/rg/providers/Microsoft.Network
/networkSecurityGroups/db-nsg/securityRules/web_.db
Etag                                 : W/"3a98e199-3bf6-4308-806c-8d84bc03723e"
ProvisioningState                    : Succeeded
Description                          :
Protocol                             : *
SourcePortRange                      : {*}
DestinationPortRange                 : {}
SourceAddressPrefix                  : {10.0.0.1, 10.0.0.2}
DestinationAddressPrefix             : {10.1.1.100}
SourceApplicationSecurityGroups      : []
DestinationApplicationSecurityGroups : []
Access                               : Allow
Priority                             : 107
Direction                            : Inbound

My first thought was “how is this possible”?

I think this is because for these single port rules, we’re using the following Terraform syntax:

destination_port_ranges    = ["${var.port-https}"]

Terraform is expecting a list, but we’re only passing a single value. Terraform doesn’t consider this invalid, and it is applied to Azure successfully. However the flaw is revealed when trying to use the “Set-AzNetworkSecurityGroup” cmdlet because that attempts to re-validate ALL rules on the NSG, not just the one that I added or modified.

If I change that Terraform property to destination_port_range (note the singular) then everything appears to work properly when using Az PowerShell afterwards.

Get Azure VM Uptime – sorta

Let’s say you have a few virtual machines in Azure, that should only be running for a limited period of time. You want to ensure that they are stopped after that period of time, but you aren’t the one responsible for the service running on the VMs, and as such, want to empower the owner(s) with both the responsibility and capability to determine when that period of time is finished; perhaps with just a bit of nudging…

Thinking about this scenario had me determine, “what if I could find out when a VM uptime exceeds a set value (say 14 days)” and notify someone that “hey, maybe you forgot to turn this off”.

Turns out there isn’t an accessible “uptime” property for an Azure VM. However, there is a time stamp property from the last state change, and that’s what we can use to enable this logic.

The key is to use “Get-AzVM” with the -Status switch. When you run Get-AzVM, you can either return the “model view” or the “instance view” of a collection of virtual machines, or an individual VM. The Microsoft Doc page for this cmdlet describes using the -Status switch to provide the “instance view” which is where we get the properties we’re after.

If you run it for a collection of VMs you can retrieve the PowerState property:

For an individual VM, you get the “Statuses” property, which contains a collection of items:

The first item within the “Statuses” collection is the “ProvisioningState”, which displays (from what I can gather) the most recent provisioning action taken on the VM (starting it, stopping/deallocating it), along with a Time property. The second item is the current PowerState of the VM.

Using this information, I’ve built the PowerShell below to grab all VMs within a subscription, and for each of them evaluate the status against a point-in-time, outputting results for those which are running and where the successful provisioning exceeds the age of that point-in-time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Select-AzSubscription 
$comparedate = (get-date).AddDays(-14)
$rg = "resourcegroup"
#Get the Instance view of a collection of virtual machines (returns the PowerState property)
$vms = get-azvm -status -resourcegroup $rg
 
#Iterate through the collection
foreach ($vm in $vms)
{
	# only check if the VM is running, because if it's off we don't care
	if ($vm.powerstate -ceq "VM running")
	{
		# Get the instance view of a single virtual machine (returns the "statuses" object)
		$foundvm = get-azvm -resourcegroup $vm.ResourceGroupName -name $vm.Name -status
		    #$foundvm.Statuses.Time
                # check if time since it was provisioned (in Statuses[0]) is greater than a value
		if ($foundvm.Statuses.Time -le $comparedate)
		{
			write-output "$($foundvm.name) : running longer than 14 days"
		}
	}
}

Using this PowerShell, you could modify the results to populate an array, and email it to a recipient. This could be placed into an Azure Automation runbook and ran on a schedule.