Populate Azure File Share from DevOps Pipeline

Use case: There’s a set of files/scripts/templates that I want to keep in sync on a set of servers, but only on-demand.

A few different ways to solve this, but one way following a pattern I’ve used a few times is to have an Azure DevOps pipeline that populates and Azure File Share, and then a separate script deployed on the servers that can on-demand pull in files from the File Share.

The script below is a YAML pipeline for Azure DevOps, that uses an AzurePowerShell task.

The primary issue I had to work-around with this (at least using the Azure PowerShell module, is that the cmdlet “Set-AzStorageFileContent” requires the parent directory to exist; it won’t auto-create it. And unfortunately “New-AzStorageDirectory” has the same problem, not creating directories recursively.

So the PowerShell script below has two sections: first to create all the folders by ensuring each leaf in the path of each distinct folder gets created, and then populating with files.

variables:
  storageAccountName: "stg123"
  resourcegroupName: "teststorage-rg"
  fileShareName: "firstfileshare"

trigger:
  branches:
    include:
    - main
  paths:
    include: # Only trigger the pipeline on this path in the git repo
    - 'FileTemplates/*'

pool:
    vmImage: 'windows-latest'
steps:

- task: AzurePowerShell@5
  inputs:
    azureSubscription: 'AzureSubConnection' #This is the devops service connection name
    ErrorActionPreference: 'Stop'
    FailOnStandardError: true
    ScriptType: 'inlineScript'
    inline: |
      $accountKey = (Get-AzStorageAccountKey -ResourceGroupName $(resourcegroupName) -Name $(storageAccountName))[0].Value
      $ctx = New-AzStorageContext -StorageAccountName $(StorageAccountName) -StorageAccountKey $accountKey
      $s = Get-AzStorageShare $(fileShareName) -Context $ctx
      # We only want to copy a subset of files in the repo, so we'll set our script location to that path
      Set-Location "$(Build.SourcesDirectory)\FileTemplates"
      $CurrentFolder = (Get-Item .).FullName
      $files = Get-ChildItem -Recurse | Where-Object { $_.GetType().Name -eq "FileInfo"}

      # Get all the unique folders without filenames
      $folders = $files.FullName.Substring($Currentfolder.Length+1).Replace("\","/") | split-path -parent | Get-Unique

      # Create Folders for every possible path
      foreach ($folder in $folders) {
        if ($folder -ne ""){
          $folderpath = ("dbscripts\" + $folder).Replace("\","/") # Create a toplevel folder in front of each path to organize within the Azure Share
          $foldersPathLeafs = $folderpath.Split("/")
          if ($foldersPathLeafs.Count -gt 1) {
            foreach ($index in 0..($foldersPathLeafs.Count - 1)) {
              $desiredfolderpath = [string]::Join("/", $foldersPathLeafs[0..$index])
              try {
                $s.ShareClient.GetDirectoryClient("$desiredfolderpath").CreateIfNotExists()
              }
              catch {
                $message = $_
                Write-Warning "That didn't work: $message"
              }

            }
          }
        }
      }

      # Create each file
      foreach ($file in $files) {
        $path=$file.FullName.Substring($Currentfolder.Length+1).Replace("\","/")
        $path = "scripts/"+$path # Create a toplevel folder in front of each path to organize within the Azure Share
        Write-output "Writing: $($file.FullName)"
        try {
          Set-AzStorageFileContent -Share $s.CloudFileShare -Source $file.FullName -Path $path -Force
        }
        catch {
          $message = $_
          Write-Warning "That didn't work: $message"
        }
      }
    azurePowerShellVersion: 'LatestVersion'
  displayName: "Azure Files Storage Copy"