Azure DevOps Pipeline or Release

After a significant amount of time envisioning a DevOps pipeline, hearing about them, reading about them on Twitter, and watching other’s create them I finally made some time to get some hands on experience. I chose to work my Azure DSC configuration deployment into a pipeline, which I’ll blog about sometime in the near future.

First, I ran into an almost immediate question: what is the difference between a Pipeline, and a Release?

From previous reading, I had learned that Continuous Integration (CI) was related to Build pipelines, which the first item in the menu was previously called. Then subsequently Continuous Deployment (CD) was a Release pipeline and separated to its own menu item.

However when I started playing around this week, what I found was a very confusing similarity between the two.

I could make a Pipeline with YAML or a classic editor, both including tasks and steps for build and releases, and when I went to save it would store a YAML file in my connected repository. This covers CI/CD in full and left me wondering where Releases fit.

I could also create a Release pipeline, in a graphical format only, with an Artifacts and a Stages section that seemingly had no connection to a repository other than creating a runtime artifact from one.

Reading the excellent Azure Docs for “Use Azure Pipelines“, it shed a little bit more light on this:

  1. Define pipelines using YAML syntax
    1. “Your code is now updated, built, tested, and packaged. It can be deployed to any target.”
  2. Define pipelines using the Classic interface
    1. “Use the Azure Pipelines classic editor to create and configure your build and release pipelines”

In short, the concept of build AND release pipelines are now effectively the Classic mode of Azure Pipelines, with the new hotness being a single pipeline, created through YAML and stored as code within a branch in your repository.

I understand why Microsoft has not fully removed the classic method, but it does take some time to understand for someone like me jumping in during the middle of a transition.

As it stands right now on the Feature Availability table, it looks like Deployment Groups and Gates are the only features that that YAML pipelines are not able to do compared to the classic Release pipeline.

Scott Hanselman Podcast Episode 719

I recently listened to and really enjoyed this podcast episode from Scott Hansleman: “Myself. Its not weird at all

In it, he touches on such a wide range of concepts: discipline and consistency, doing what you love, “Yes, and” instead of “no” as a redirect. He talks about deliberate practice, imposter syndrome, mindfulness, commitment to what is important, and making time.

Some choice quotes that really resonated with me:

  • “Most people live their lives accepting the defaults. If you are deliberate about installing something, you hit custom.”
    • Curiosity has been a major component to my learning, both professionally and personally. I see an “advanced” window on an installer wizard, and I can’t help but click it just to find out what’s hidden there. I feel the same way outside of work too – be curious about why I think the way I think, believe what I believe; be curious about how things work in a physical sense and a conceptual sense.
  • “By simply being mindful, you cannot stop but improving”
    • This thought gives me encouragement that even small intentional actions can have a positive impact. Even if I’m not feeling particularly capable, even if I’m not crushing a project or task, even if I haven’t been dedicated to deep learning, I can still be on an upward trend by being aware, considerate, and mindful of my actions and those around me.
  • “Yes there are a thousand little things where someone can make me question who I am, and am I good; but then I go back and look at what I do, and what I have made and say “no, I did that and I’m going to give myself credit”.”
    • Often times for me, this is as much giving myself permission to be proud of what I have accomplished and reflect on it a little bit, rather than discount my efforts for a variety of reasons that my mind is surely able to conjure up.
  • “Why do I feel like an imposter talking to this person? The reason is because of the road not taken; This person did something I did not do, and I’m intimidated because they stuck with it when I did not.”
    • I frequently question and doubt some of my decisions, particularly when I see people have success on alternate paths. I need to remind myself that their success in no way invalidates my path – if I am confident in who I am and what I do, I can celebrate their success rather than wonder what could have been.

Highly recommended listen.

Update SCCM Maintenance Window through PowerShell

Sometimes trying to stay bleeding edge is tough – today’s example is when you want to install updates through SCCM a day or two after Patch Tuesday, particularly using Maintenance Windows to allow restarts within a specific time frame.

We use automatic deployment rules to update a Software Update Group every Patch Tuesday – scheduling this is easy because its always the second Tuesday of a month.

But I want the updates to install on Wednesday night, or Thursday morning. This ensures strict compliance requirements can be met, but allows 24 hours for testing. Can’t just schedule the install and restarts for “second Wednesday of the month” though, because if the first of the month is a Wednesday (like this month) then our actual install date happens to be the THIRD Wednesday of the month.

Previously we solved this by manually updating Maintenance Window schedules every month, painstaking selecting the right date and hoping we didn’t mess it up.

PowerShell took that risk away:

 

SCCM_UpdateMaintenanceWindowSchedule.ps1 – on GitHub

 

See the comments on the script for details of how it works. As a very brief overview:

I found Tim Curwick’s method of calculating Patch Tuesday, and used that in my script to reliably calculate my Wednesday or Thursday install date.

This script runs as System from an SCCM server on the 1st of every month. It performs the calculation, updates maintenance windows on specific Collections, and outputs a log to file and emails results.

 

Azure Site Recovery – ARM template

Programmatic deployment of Azure Site Recovery for Azure VMs – that’s the target I started with for a project a little while ago.

There is a large amount of information for Azure Site Recovery on Microsoft’s Docs, however the amount of available code online to programmatically deploy a full setup is very sparse! Much of what Microsoft provides is PowerShell, which isn’t idempotent and doesn’t fit with my current tooling (Terraform and declarative infrastructure-as-code). I did consider Azure CLI, but couldn’t find any references for Site Recovery.

While working on this I came across an Azure Quickstart which is a little incomplete and starts with some good variable definition but quickly devolves into hard-coded values from where it came from; I also discovered a blog post from Pratap Bhaskar  which was useful especially for understanding the Loop mechanism in the template, but it didn’t go far enough for my purposes.

So I spun up a few VMs, manually configured ASR, and did an ARM template export. What I received was a huge amount of properties on the resources that I was sure were relevant to runtime only, not creation. This was also a good reference, but not exactly where I needed to be.

The final piece of the puzzle that got me on my way was the REST API docs for Site Recovery. With this in hand, and the other sources I had at my disposal, I had the references I needed to begin putting together an ARM Template that would configure my environment end-to-end including Recovery Plans with automation runbooks.

There are a lot of design decisions I made when building this to fit my environment, some of which won’t make sense without additional context; most of which I can’t provide. That’s ok, as I hope it at least serves as a reference for “what’s possible” to others who come across it. Here is the overall structure:

  • Pre-define and create destination resources like resource groups, virtual networks, and subnets with Terraform
  • Deploy ASR for a subset of Virtual Machines, targeting the destination resources
    • Include dependent resources like source-side storage account for cache, and azure automation account in the same region and subscription as the ASR resources
  • Deploy a Recovery Plan that provides runbook functionality to configure a Test Failover environment
    • This environment was intended to be completely isolated, to ensure there’s no chance of contamination with prod
    • Current design of my web servers has multiple ip configurations; these need to be replaced
    • Access to the environment is provided through a Jump Host, which needs a known-in-advance IP address

 

I have documented this output on a GitHub repo called arm-azuresiterecovery

 

 

The majority of my time was actually spent cleaning up the template into proper parameters and variables for effective re-use, and then solving all the syntax challenges and typos that come along with that.

 

There are still some loose ends in what I’ve created, around certain manual steps still required. However as I’m sure is common in the industry, it is good enough to deploy and I must move on – fine-tuning comes later.

Invoke-AzVMRunCommand log output

While working with the Invoke-AzVmRunCommand cmdlet, I encountered a problem, receiving the following error:

Invoke-AzVMRunCommand : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure w
hen processing extension 'RunCommandWindows'. Error message: "Finished executing command".'

I’m attempting to push a DSC configuration into a VM and run it, and although the PowerShell runs successful when run from inside the VM, I still see this error.

I came across a github issue with a helpful tip: the log output of Invoke-AzVmRunCommand can be viewed from this path:

C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\<version>\Status

This helped me determine what the problem was and how to solve it.