When version 1.0.0 of this open-sourced (!) product was released, I had expected much more buzz on social media from VMware PowerCLI users. 

This appliance does in fact bring an answer to a wish that many PowerShell/PowerCLI users have had for years: a “Scripting Host“!

This Hitchhikers Guide to SRS 1.0.0 post will show how I build my own customised SRS appliance, and how I use it to run PowerShell/PowerCLI scripts.

Introduction

When you visit the Script Runtime Service for vSphere (SRS) repository, you’ll notice that this open-sourced project comes with extensive documentation.

Always a great characteristic for an open-sourced project!

The installation as a VM  (from the downloadable OVF files) or in a Kubernetes cluster is well documented.

The available documentation in the SRS repo contains instructions on how to build, and customise, your own SRS appliance under the Build and Run page.

This blog post documents how I did exactly that.

I wanted to have a fully automated, self-documenting, and repeatable method, think CI, to roll out my own SRS station.

1. Roll out the Builder station and install the prerequisites.
2. Start the Build of the SRS Appliance on the Builder station
3. Retrieve the OVF/OVA files, created from the SRS Appliance, that came out of the build
4. Use the OVF/OVA to deploy your SRS station

Creating the SRS Builder

The Prerequisites

To create an SRS OVA/OVF yourself you need a “builder” station. And on that station there need to be a number of packages installed. 

I included the installation of these packages in the “build” I describe in the next section.

The buildappliance.sh script, which you will need to run on the builder station, will use the packages to setup and configure an SRS appliance (VMware Photon based), and as the last step will generate an OVA file from the appliance.

That OVA file can then be used to deploy your SRS VM.

This is also the place where you can install extra applications and extra PowerShell modules that will be included in your DIY SRS OVA.

Create the Builder station

Preparation

I used an Ubuntu 18.04 LTS station as the Builder station. As the download of the ISO is also automated, I use the Daily Build.

#region Get the Ubuntu Bionic Cloud Image OVA Daily Build

$uri = 'https://cloud-images.ubuntu.com/bionic/current/'
$ovaName = 'bionic-server-cloudimg-amd64.ova'
$repository = 'D:\Repository\Linux\Ubuntu\'

$currentOVA = Get-ChildItem -Path "$repository\$ovaName" -ErrorAction SilentlyContinue
if (-not $currentOVA -or $currentOVA.LastWriteTime.Date -lt $now.Date) {
  $sWeb = @{
    Uri = "$uri$ovaName"
    OutFile = "$repository$ovaName"
    Verbose = $false
  }
  Invoke-WebRequest @sWeb
  $currentOVA = Get-ChildItem -Path "$repository\$ovaName"
}
#endregion

I also check if the DNS A and PTR records for the Builder station are present in DNS. The Builder station I used has an FQDN of srsbuilder.local.lab with an IP address of 192.168.10.88.

$hostName = 'srsbuilder'
$domain = 'local.lab'
$ipAddress = '192.168.10.88'

$if = Get-Netadapter
$sDns = @{
   AddressFamily = 'IPv4'
   InterfaceIndex = $if.ifIndex
}
$dns = Get-DnsClientServerAddress @sDns
$dnsServer = $dns.ServerAddresses | Get-Random
$ip = $ipAddress.Split('.')
$reverseZone = "$($ip[2]).$($ip[1]).$($ip[0]).in-addr.arpa"

try{
  Write-Verbose (Get-LogText -Text "Check DNS A record for $hostname.$domain with $ipAddress")
  $sQDns = @{
    Name = "$hostName.$domain"
    Server = $dnsServer
    ErrorAction = 'Stop'
    Verbose = $false
  }
  Resolve-DnsName @sQDns  | Out-Null
  Write-Verbose (Get-LogText -Text "DNS A record exists for $hostname.$domain with $ipAddress")
}
catch{
  Write-Verbose (Get-LogText -Text "Create A record for $hostname.$domain with $ipAddress")
  $sADns = @{
    Name = $hostName
    ZoneName = $domain
    IPv4Address = $ipAddress
    A = $true
    CreatePtr = $true
    ComputerName = $dnsServer
  }
  Add-DnsServerResourceRecord @sADns | Out-Null
}
try {
  Write-Verbose (Get-LogText -Text "Check DNS PTR record for $hostname.$domain with $ipAddress")
  $sQdns = @{
    Name = $ipAddress
    Type = 'PTR'
    Server = $dnsServer
    Verbose = $false
    ErrorAction = 'Stop'
  }
  Resolve-DnsName @sQdns | Out-Null
  Write-Verbose (Get-LogText -Text "DNS PTR record exists for $hostname.$domain with $ipAddress")
}
catch {
  Write-Verbose (Get-LogText -Text "Create PTR record for $hostname.$domain with $ipAddress")
  $sADns = @{
    Name = "$($ip[3])"
    PtrDomainName = "$hostname.$domain"
    ZoneName = $reverseZone
    ComputerName = $dnsServer
  }
  Add-DnsServerResourceRecordPtr @sADns | Out-Null
}

Deploy the Builder

For the rollout of this Builder station, I used the Cloud-Init method I described in Cloud-Init – Part 2 – Advanced Ubuntu. I had to make some small changes to the Install-CloudInitVM function from that post, so I was able to specify the memory size and the disk size. The default sizes were insufficient to run the SRS Appliance build.

The updated function now has MemoryGB and DiskGB parameters.

I also had to add a snippet to handle the issue with the $PSScriptRoot parameter when running the code from the Visual Studio Code editor.

function Install-CloudInitVM {
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
#>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [string]$CloudConfig,
    [PSCredential[]]$Credential,
    [int]$MemoryGB,
    [int]$DiskGB
  )

#region Bypass for known issue in VSC ()
# See https://github.com/PowerShell/vscode-powershell/issues/633

  if ($psISE) {
    $dir = Split-Path -Path $psISE.CurrentFile.FullPath
  }
  else {
    if ($profile -match "VScode") {
      $dir = Split-Path $psEditor.GetEditorContext().CurrentFile.Path
    }
    else {
      $dir = $PSScriptRoot
    }
  }
#endregion

  $waitJob = (Get-Command -Name "$dir\Wait-Job.ps1").ScriptBlock
  $userData = Get-Content -Path "$dir\$CloudConfig" -Raw

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue
  if ($vm) {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn') {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile
  $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
  $ovfProp.NetworkMapping.VM_Network.Value = $pgName

  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName | Get-VMHost | Get-Random
    OvfConfiguration = $ovfProp
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  if($MemoryGB){
    $vm = Set-VM -VM $vm -MemoryGB $MemoryGB -Confirm:$false
  }
  if($DiskGB){
    $hd = Get-HardDisk -VM $vm | Set-HardDisk -CapacityGB $DiskGB -Confirm:$false
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"

  Write-Verbose "nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

The call to the Install-CloudInitVM functions is rather straightforward.

$vmName = 'SRSBuilder'

# Get credential for logging on to the guest
# Replace with your secrets manager application when
# running this on PSv6 or higher

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = 'D:\Repository\Linux\Ubuntu\bionic-server-cloudimg-amd64.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfig = 'user-data-srsbuilder.yaml'
  Credential = $cred
  MemoryGB = 4
  DiskGB = 25
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

Notice how I used the VICredentialStore to store and retrieve the credentials for the Builder station. When you run this from a PowerShell version greater than 5.1, you will have to replace that part with calls to the secrets manager of your choice.

The real strength of using this Cloud-Init method is that one can use a YAML file to specify how the target station, the Builder station, in this case, needs to be configured. And also specify which packages shall be installed on the target station as part of the deployment.

The following extract of my YAML file shows the packages that are required. It also includes pulling down the SRS appliance.

# SRS 1.0
# A) dotnet sdk
# 1) add the Microsoft package signing key
- wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
# 2) add the package repository
- dpkg -i packages-microsoft-prod.deb
# 3) donet sdk
- apt-get update
- apt-get install -y apt-transport-https
- apt-get update
- apt-get install -y dotnet-sdk-3.1
# B) docker
- curl -fsSL https://get.docker.com -o get-docker.sh
- sh get-docker.sh
# C) PowerShell
- apt-get install -y wget apt-transport-https software-properties-common
- wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
- dpkg -i packages-microsoft-prod.deb
- apt-get update
- add-apt-repository universe
- apt-get install -y powershell
# D) VMware PowerCLI
- pwsh -C '& {Install-Module -Name VMware.PowerCLI -Scope AllUsers -Force -Confirm:$false -AllowClobber}'
# E) OvfTool
- /root/ovftool-ftp.sh
- chmod 744 /root/VMware-ovftool-4.4.1-16812187-lin.x86_64.bundle
- /root/VMware-ovftool-4.4.1-16812187-lin.x86_64.bundle --eulas-agreed --required --console
# F) packer
- curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add -
- apt-get update
- apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
- apt-get update
- apt-get install packer
# X) Additional modules
- pwsh -C '& {Install-Module -Name ImportExcel -Scope AllUsers -Force -Confirm:$false -AllowClobber}'
- pwsh -C '& {Install-Module -Name Posh-Ssh -Scope AllUsers -Force -Confirm:$false -AllowClobber}'
# Y) Clone SRS repo
- mkdir /root/SRS
- git clone https://github.com/vmware/script-runtime-service-for-vsphere /root/SRS
# End SRS 1.0

Verify the Builder

Once the Builder station is deployed, it is useful to check that everything the creation of the SRS Appliance needs, is present.

A word of warning, the following code includes all versions as hard-coded. This is not ideal, and especially for products that have a daily build, this will require updating the code each time. Since this was, at the time of writing this post, not my top priority, I postponed looking for a more portable solution to a later date.

#region Helper functions
function Get-LogText {
  param([string]$Text)

  $dt = (Get-Date).ToString('yyyyMMdd HH:mm:ss.fff')
  $app = Split-Path -Path $MyInvocation.ScriptName -Leaf
  "$dt - $app - $Text"
}
#endregion

#region Preamble

$vmName = 'SrsBuilder'

$now = Get-Date

$viCredObj = Get-VICredentialStoreItem -Host $vmName
$sObj = @{
  TypeName = 'PSCredential'
  ArgumentList = $viCredObj.User,(ConvertTo-SecureString -String $viCredObj.Password -AsPlainText -Force)
}
$cred = New-Object @sObj

$vm = Get-VM -Name $vmName
#endregion

#region dotnet SDK
$check_dotnet = @'
dotnet --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_dotnet
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if($result.ScriptOutput.Trim("n") -ne '3.1.404'){
  Write-Host (Get-LoGText -Text "Dotnet SDK installation failure") -ForegroundColor red
}
else{
  Write-Host (Get-LogText -Text "Dotnet SDK $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Docker
$check_docker = @'
docker --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_docker
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne 'Docker version 20.10.1, build 831ebea') {
  Write-Host (Get-LoGText -Text "Docker installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "$($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region PowerShell v7
$check_PSv7 = @'
pwsh -C '& {$PSVersionTable.PSVersion.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_PSv7
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '7.1.0') {
  Write-Host (Get-LoGText -Text "PowerShell installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "PowerShell $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region VMware PowerCLI
$check_PowerCLI = @'
pwsh -C '& {(Get-Module -Name VMware.PowerCLI -ListAvailable).Version.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_PowerCLI
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '12.1.0.17009493') {
  Write-Host (Get-LoGText -Text "VMware PowerCLI installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "VMware PowerCLI $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region VMware ovftool
$check_ovftool = @'
ovftool --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_ovftool
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne 'VMware ovftool 4.4.1 (build-16812187)') {
  Write-Host (Get-LoGText -Text "VMware OVFTool installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "$($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Packer
$check_packer = @'
packer --version
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_packer
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '1.6.6') {
  Write-Host (Get-LoGText -Text "Packer installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "Packer $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Additional

#region ImportExcel
$check_ImportExcel = @'
pwsh -C '& {(Get-Module -Name ImportExcel -ListAvailable).Version.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_ImportExcel
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '7.1.1') {
  Write-Host (Get-LoGText -Text "ImportExcel installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "ImportExcel $($result.ScriptOutput.Trim("n")) installation OK") -ForegroundColor green
}
#endregion

#region Posh-Ssh
$check_PoshSSH = @'
pwsh -C '& {(Get-Module -Name Posh-SSH -ListAvailable).Version.ToString()}'
'@
$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptText = $check_PoshSSH
  ScriptType = 'bash'
}
$result = Invoke-VMScript @sInvoke
if ($result.ScriptOutput.Trim("n") -ne '2.3.0') {
  Write-Host (Get-LoGText -Text "Posh-SSH installation failure") -ForegroundColor red
} else {
  Write-Host (Get-LogText -Text "Posh-SSH $($result.ScriptOutput.Trim("`n")) installation OK") -ForegroundColor green
}
#endregion

#endregion

Run the Build

When the Builder station is deployed, and we verified that all prerequisites are in place, we can start the deployment of the SRS Applicance. From which, if all goes well, the OVA file will be generated.

During my experimenting with building the SRS Appliance, I stumbled on two what I suspect are issues.

As a bypass I build the SRS Appliance for now with Photon V3 Rev 2.

I did not succeed in building the SRS Appliance with Photon V3 Rev 3 image. I opened an Issue for that, and I’m curious to know if the issue is a Photon issue or something in my environment.

Another issue, which seems to be a real, known issue, is that you apparently can not use a Distributed Switch Portgroup to build the SRS Appliance. The underlying issue seems to stem from the Packer application, more specifically the VMware-Iso builder. This builder uses VNC to ‘talk’ with the ESXi node on which the SRS Appliance is created.

There are two issues here:

• VNC is not included anymore in ESXi since 6.7
• With the ESXi API you can not interact with a VDS

The VNC issue is resolved by changing some Packer settings. But for the VDS issue, there is no solution besides switching to the vSphere-Iso builder in the Packer step, so I had to build the SRS Appliance on a VSS Portgroup.

I packaged the preparation steps and the start of the SRS Appliance build in a function, named Invoke-SRSBuild.

function Invoke-SRSBuild {
  [CmdletBinding()]
  param(
    [Parameter(Mandatory)]
    [String]$BuilderName,
    [Parameter(Mandatory)]
    [PSCredential]$Credential,
    [Switch]$PrepOnly,
    [Switch]$BuildOnly,
    [Parameter(Mandatory)]
    [ValidateSet('V3Rev2', 'V3Rev3')]
    [String]$PhotonVersion,
    [Parameter(Mandatory)]
    [String]$Portgroup,
    [Switch]$PackerLog
  )

  #region Helper functions
  function Get-LogText {
    param([string]$Text)

    $dt = (Get-Date).ToString('yyyyMMdd HH:mm:ss.fff')
    $app = Split-Path -Path $MyInvocation.ScriptName -Leaf
    "$dt - $app - $Text"
  }

  function Remove-SRSFolder {
    Param(
      [Parameter(Mandatory = $true)]
      [VMware.VimAutomation.ViCore.Types.V1.DatastoreManagement.StorageResource]$Datastore
    )

    $dsBrowser = Get-View -Id $Datastore.ExtensionData.Browser
    $spec = New-Object -TypeName VMware.Vim.HostDatastoreBrowserSearchSpec
    $spec.MatchPattern = 'SRS_Appliance'
    $spec.Query = New-Object -TypeName Vmware.Vim.FolderFileQuery
    $spec.Details = New-Object -TypeName VMware.Vim.FileQueryFlags
    $result = $dsBrowser.SearchDatastore("[$($Datastore.Name)]", $spec)

    if ($result.File.Count -gt 0 -and $result.File[0]) {
      $dc = Get-VMHost -Datastore $Datastore | Get-Datacenter
      $fileMgr = Get-View FileManager
      $fileMgr.DeleteDatastoreFile("[$($Datastore.Name)] $($result.File[0].Path)", $dc.ExtensionData.MoRef)
    }
  }
  #endregion

  #region Bypass for known issue in VSC ()
  # See https://github.com/PowerShell/vscode-powershell/issues/633

  if ($psISE) {
    $dir = Split-Path -Path $psISE.CurrentFile.FullPath
  } else {
    if ($profile -match "VScode") {
      $dir = Split-Path $psEditor.GetEditorContext().CurrentFile.Path
    } else {
      $dir = $PSScriptRoot
    }
  }
  #endregion

  #region Preamble

  $vm = Get-VM -Name $BuilderName
  #endregion

  if (-not $BuildOnly.IsPresent) {

    #region SSH service
    $ssh = Get-VMHostService -VMHost $vm.VMHost | Where-Object { $_.Label -eq 'SSH' }
    if ($ssh.Running) {
      Write-Verbose (Get-LogText -Text "SSH is running on $($vm.VMHost.Name)")
    } else {
      Write-Verbose (Get-LogText -Text "Starting SSH on $($vm.VMHost.Name)")
      $ssh = Start-VMHostService -HostService $ssh -Confirm:$false
      if (-not $ssh.Running) {
        Write-Verbose (Get-LogText -Text "Could not start SSH on $($vm.VMHost.Name)")
        throw "SSH service not started on $($vm.VMHost.Name)"
      }
    }
    #endregion

    #region GuestIPHack
    $esxcli = Get-EsxCli -VMHost $vm.VMHost -V2
    $sOpt = @{
      option = '/Net/GuestIPHack'
    }
    $opt = $esxcli.system.settings.advanced.list.Invoke($sOpt)
    if ($opt.IntValue -eq 1) {
      Write-Verbose (Get-LogText -Text "GuestIPHack setting is correct on $($vm.VMHost.Name)")
    } else {
      Write-Verbose (Get-LogText -Text "GuestIPHack setting is not correct on $($vm.VMHost.Name)")
      $sOpt.Add('intvalue', [long]1)
      if ($esxcli.system.settings.advanced.set.Invoke($sOpt)) {
        Write-Verbose (Get-LogText -Text "GuestIPHack setting corrected on $($vm.VMHost.Name)")
      } else {
        Write-Verbose (Get-LogText -Text "Could not change GuestIPHack setting on $($vm.VMHost.Name)")
        throw "Could net set GuestIPHack"
      }
    }
    #endregion

    #region Make sure folder doesn't exist
    $ds = Get-Datastore -RelatedObject $vm
    Remove-SRSFolder($ds)
    #endregion

    #region Update photon-build.json
    # Suspected issue when using a vdPortgroup

    $jsonFile = New-TemporaryFile
    $photonBuilder = @{
      builder_host = $vm.VMHost.Name
      builder_host_username = $viCredObj.User
      builder_host_password = $viCredObj.Password
      builder_host_datastore = (Get-Datastore -RelatedObject $vm).Name
      builder_host_portgroup = $Portgroup
    }
    $photonBuilder | ConvertTo-Json | Set-Content -Path $jsonFile
    $sCopy = @{
      VM = $vm
      GuestCredential = $Credential
      LocalToGuest = $true
      Source = $jsonFile
      Destination = '/root/SRS/appliance/photon-builder.json'
      Force = $true
      Confirm = $false
    }
    Copy-VMGuestFile @sCopy
    Remove-Item -Path $jsonFile -Confirm:$false
    #endregion

    #region Update photon-version.json
    $photonTab = Get-Content -Path "$dir\Photon-version.json" | ConvertFrom-Json
    $photonSelected = $photonTab.Photon | Where-Object { $_.Label -eq $PhotonVersion }
    $jsonFile = New-TemporaryFile
    $sCopy = @{
      VM = $vm
      GuestCredential = $Credential
      GuestToLocal = $true
      Source = '/root/SRS/appliance/photon-version.json'
      Destination = $jsonFile
      Force = $true
      Confirm = $false
    }
    Copy-VMGuestFile @sCopy
    $photonBuilder = (Get-Content -Path $jsonFile | ConvertFrom-Json)[0]
    if ($photonBuilder.iso_url -ne $photonSelected.Uri -or $photonBuilder.iso_checksum -ne $photonSelected.CheckSum) {
      $photonBuilder.iso_url = $photonSelected.Uri
      $photonBuilder.iso_checksum = $photonSelected.CheckSum
      @($photonBuilder) | ConvertTo-Json | Set-Content -Path $jsonFile
      $sCopy = @{
        VM = $vm
        GuestCredential = $Credential
        LocalToGuest = $true
        Source = $jsonFile
        Destination = '/root/SRS/appliance/photon-version.json'
        Force = $true
        Confirm = $false
      }
      Copy-VMGuestFile @sCopy
      Remove-Item -Path $jsonFile -Confirm:$false

    }
    #endregion

    #region Handle VNC for ESXi 6.7 and later
    if ([Version]$vm.VMHost.Version -ge [Version]'6.7.0') {
      $updateJSON = 'sed' +
      ' -i.bak -e ''/vnc_disable_password/ i \      "vnc_over_websocket": true,''' +
      ' -e ''/vnc_disable_password/ i \      "insecure_connection": true,''' +
      ' -e ''/vnc_disable_password/d'' ~/SRS/appliance/photon.json'

      $sInvoke = @{
        VM = $vm
        GuestCredential = $Credential
        ScriptText = $updateJSON
        ScriptType = 'bash'
      }
      $result = Invoke-VMScript @sInvoke
    }
    #endregion

  }

  if (-not $PrepOnly.IsPresent) {

    #region Get PowerCLI folder path
    $findDir = 'pwsh -C ''& {Split-Path -Path (Split-Path -Path (Get-Module -Name VMware.PowerCLI -ListAvailable).ModuleBase)}'''
    $sInvoke = @{
      VM = $vm
      GuestCredential = $Credential
      ScriptText = $findDir
      ScriptType = 'bash'
    }
    $pcliPath = (Invoke-VMScript @sInvoke).ScriptOutput
    #endregion

    #region Start build
    $sInvoke = @{
      VM = $vm
      GuestCredential = $Credential
      ScriptText = "/root/SRS/build.sh $pcliPath"
      ScriptType = 'bash'
    }
    if ($PackerLog.IsPresent) {
      $packerLogName = 'packer.log'
      $sInvoke.ScriptText = "export PACKER_LOG=1;export PACKER_LOG_PATH='$($packerLogName)';$($sInvoke.ScriptText)"
    }
    $buildResult = Invoke-VMScript @sInvoke
    $buildResult.ScriptOutput > "$dir\Buildlog.txt"
    #endregion

    #region Rettrieve Packer log
    if($PackerLog.IsPresent){
      $sCopy = @{
        VM = $vm
        GuestCredential = $Credential
        GuestToLocal = $true
        Source = "/root/SRS/appliance/$($packerLogName)"
        Destination = "$dir\$($packerLogName)"
        Force = $true
        Confirm = $false
      }
      Copy-VMGuestFile @sCopy
    }
    #endregion

    #region Clean up
    # Packer leaves an orphaned entry in the VCSA behind

    $endpointVM = Get-VM -Name 'SRS_Appliance' -ErrorAction SilentlyContinue
    if($endpointVM){
      while($endpointVM.PowerState -ne 'PoweredOff'){
        sleep2
        $endpointVM = Get-VM -Name 'SRS_Appliance' -ErrorAction SilentlyContinue
      }
      Remove-VM -VM $endpointVM   -DeletePermanently -ErrorAction SilentlyContinue -Confirm:$false
    }
    #endregion
  }
}

The call to the Invoke-SRSBuilder function is again straight-forward.

$vmName = 'SRSBuilder'

#region Credential
# Get the credentials for the SRS Builder station
# Alternative method required when using PSv6 or later

$viCredObj = Get-VICredentialStoreItem -Host $vmName
$sObj = @{
  TypeName = 'PSCredential'
  ArgumentList = $viCredObj.User, (ConvertTo-SecureString -String $viCredObj.Password -AsPlainText -Force)
}
$cred = New-Object @sObj
#endregion

$sBuild = @{
  BuilderName = $vmName
  Credential = $cred
  PhotonVersion = 'V3Rev2'
  Portgroup = 'PG1'
  PackerLog = $true
}
Invoke-SRSBuild @sBuild

The OVA

When the call to the build.sh script completes successfully, there will be OVA/OVF files created on the Builder station.

Since the Builder station is, in my setup, a temporary station, I need to download those files to a more permanent location. The following snippet uses the Get-ScpFile cmdlet from the Posh-Ssh module to do that.

#requires -Modules Posh-Ssh

#region Preamble

$vmName = 'SrsBuilder'

$now = Get-Date

$viCredObj = Get-VICredentialStoreItem -Host $vmName
$sObj = @{
  TypeName = 'PSCredential'
  ArgumentList = $viCredObj.User, (ConvertTo-SecureString -String $viCredObj.Password -AsPlainText -Force)
}
$cred = New-Object @sObj

$vm = Get-VM -Name $vmName

$fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost
#endregion

#region Download OVA
$sSCP = @{
  ComputerName = $fqdn
  Credential = $cred
  RemoteFile = '/root/SRS/appliance/output-vmware-iso/SRS_Appliance_1.0.0.ova'
  LocalFile = 'D:\OVA\SRS\V1.0.0\Rev2-VSS\SRS_Appliance_1.0.0.ova'
  AcceptKey = $true
}
Get-SCPFile @sSCP
#endregion

Deploy SRS

Once we have the OVA available, it is a piece of cake to deploy our SRS VM with the Get-OvfConfiguration and Import-VApp cmdlets.

Note that deploying the SRS OVA to a Distributed Switch Portgroup is perfectly possible. The VDS issue mentioned earlier only comes into play when Packer with the VMware-Iso builder is involved. 

$vmName = 'srs1'
$clusterName = 'cluster'
$numCPU = 2
$memoryGB = 4
$dsName = 'vsanDatastore'
$harddiskGB = 2
$ovaPath = 'D:\OVA\SRS\V1.0.0\Rev2-VSS\SRS_Appliance_1.0.0.ova'
$networkName = 'vdPg1'

$cluster = Get-Cluster -Name $clusterName
$esx = Get-VMHost -Location $cluster | Get-Random
$ds = Get-Datastore -Name $dsName
$ovfProp = Get-OvfConfiguration -Ovf $ovaPath
$ovfProp.Common.guestinfo.hostname.Value = $vmName
$ovfProp.Common.guestinfo.ipaddress.Value = '192.168.10.43'
$ovfProp.Common.guestinfo.netmask.Value = '24'
$ovfProp.Common.guestinfo.gateway.Value = '192.168.10.1'
$ovfProp.Common.guestinfo.root_password.Value = 'Welcome2020!'
$ovfProp.Common.guestinfo.dns.Value = '192.168.10.2'
$ovfProp.Common.guestinfo.domain.Value = 'local.lab'
$ovfProp.Common.srs.vcaddress.Value = 'vcsa7.local.lab'
$ovfProp.Common.srs.vcpassword.Value = 'Welcome2020!'
$ovfProp.Common.srs.vcuser.Value = 'administrator@vsphere.local'
$ovfProp.Common.srs.vcthumbprint.Value = '5bf3246fde90fd3b7ab84144f50105114c2826e1'
$ovfProp.NetworkMapping.PG1.Value = $networkName

$vm = Import-VApp -Name $vmName -Source $ovaPath -VMHost $esx -OvfConfiguration $ovfProp -Datastore $ds
Start-VM -VM $vm -Confirm:$false

Your first test to see if the deployment went ok, is to try and access the swagger page on your SRS VM. The URI is https://<your-SRS-FQDN>/swagger. When all goes well, you will see a page like this.

Using SRS

Now we can start using the SRS server to run our scripts. The way to do that is quite simple. The complete process is described in the Run Scripts section in the SRS repository.

A Simple Sample

My test installation of SRS is done on a VM that is named SRS1.

A basic REST API call to verify that everything is working, is to call the api/about method.

function Invoke-SrsMethod {
  [cmdletbinding()]
  param(
    [Parameter(Mandatory)]
    [String]$FQDN,
    [Parameter(Mandatory)]
    [String]$API,
    [Parameter(Mandatory)]
    [String]$Method
  )

  $sWeb = @{
    Uri = "https://$($FQDN)/$($API)"
    Method = $Method
  }

  try {
    $result = Invoke-WebRequest @sWeb
  }
  catch [System.Net.WebException] {
    switch ($error[0].Exception.Status) {
      ([System.Net.WebExceptionStatus]::TrustFailure) {
        if ($PSVersionTable.PSVersion.Major -lt 6) {
          if (-not ([System.Management.Automation.PSTypeName]"TrustAllCertsPolicy").Type) {
            Add-Type -TypeDefinition @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem)
    {
        return true;
    }
}
"@
          }
          if ([System.Net.ServicePointManager]::CertificatePolicy.ToString() -ne "TrustAllCertsPolicy") {
            [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
          }
        } else {
          $sWeb.Add('SkipCertificateCheck', $true)
        }
        $result = Invoke-WebRequest @sWeb
      }
      Default {
        Write-Error "Unhandled WebException $($error[0].Exception.Status)"
      }
    }
  }
  catch {
    Write-Error "Unhandled exception code $($error[0].Exception.gettype().Name)"
  }

  switch ($result.StatusCode) {
    200 {
      $result
    }
    Default {
      Write-Error "Unhandled StatusCode $($result.StatusCode)"
    }
  }
}

$vmName = 'srs1'

$vm = Get-VM -Name $vmName
$fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost

$about = Invoke-SrsMethod -FQDN $fqdn -API 'api/about' -Method 'Get'
$about.Content | ConvertFrom-Json

You notice that I created a function Invoke-SRSMethod instead of just calling the Invoke-WebRequest cmdlet directly. The reason for creating that function is that I needed to be able to bypass invalid certificates.

With PSv6 that has become easy, since the Invoke-WebRequest cmdlet now has the SkipCertificateCheck switch. 

But I wanted to have a function that would also work in a PSV5.1 environment, hence the function and the logic in there.

When all goes well, that snippet should return the following.

A PowerCLI example

The full sequence for running code on the SRS station is perfectly explained in the Run Scripts section on the SRS repository.

A full, scripted example, excluding the earlier Invoke-SRSMethod function, could look like this.

#region Preamble
$vmName = 'srs1'

$vm = Get-VM -Name $vmName
$fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost

# Get credential for logging on to the guest OS
$viCred = Get-VICredentialStoreItem -Host $global:DefaultVIServer.Name
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$headers = @{
  "accept" = "application/json"
  "content-type" = "application/json"
}
#endregion

#region Login
$sLogon = @{
  FQDN = $fqdn
  API = '/api/auth/login'
  Method = 'Post'
  Credential = $cred
  Headers = $headers
}
$logon = Invoke-SrsMethod @sLogon
$headers.Add('X-SRS-API-KEY', $logon.Headers['X-SRS-API-KEY'])
#endregion

#region Create Runspace
$sRSCreate = @{
  FQDN = $fqdn
  API = '/api/runspaces'
  Method = 'Post'
  Credential = $cred
  Headers = $headers
  Body = @{
    name = 'MyRS'
    run_vc_connection_script = $true
  }
}
$createRS = Invoke-SrsMethod @sRSCreate
$rs = $createRS.Content | ConvertFrom-Json
#endregion

#region Wait till RS is ready
while ($rs.state -eq 'Creating') {
  $sRSGet = @{
    FQDN = $fqdn
    API = "/api/runspaces/$($rs.Id)"
    Method = 'Get'
    Headers = $headers
  }
  $getRS = Invoke-SrsMethod @sRSGet
  $rs = $getRS.Content | ConvertFrom-Json
}
#endregion

#region Run Script
$code = {
  Get-VM
}
$sSCRCreate = @{
  FQDN = $fqdn
  API = '/api/script-executions'
  Method = 'Post'
  Headers = $headers
  Body = @{
    runspace_id = $rs.id
    name = 'MyScript'
    script = $code.ToString()
    script_parameters = @()
  }
}
$createSCR = Invoke-SrsMethod @sSCRCreate
$scr = $createSCR.Content | ConvertFrom-Json
#endregion

#region Wait for Script to end
while($scr.state -eq 'running'){
$sSCRCreate = @{
  FQDN = $fqdn
  API = "/api/script-executions/$($scr.id)"
  Method = 'Get'
  Headers = $headers
}
$getSCR = Invoke-SrsMethod @sSCRCreate
$scr = $getSCR.Content | ConvertFrom-Json
}
#endregion

#region Retrieve Script output
$sSCROut = @{
  FQDN = $fqdn
  API = "/api/script-executions/$($scr.id)/output"
  Method = 'Get'
  Headers = $headers
}
$outSCR = Invoke-SrsMethod @sSCROut
$outSCR.Content | ConvertFrom-Json
#endregion

#region Retrieve Script streams
'information', 'error', 'warning', 'debug', 'verbose' |
ForEach-Object -Process {
  $sSCRStr = @{
    FQDN = $fqdn
    API = "/api/script-executions/$($scr.id)/streams/$($_)"
    Method = 'Get'
    Headers = $headers
  }
  $getStream = Invoke-SrsMethod @sScrStr
  if($getStream.Content -ne '[]'){
    Write-Host "--- $_ ---" -ForegroundColor Green
    $getStream.Content | ConvertFrom-Json | Out-Default
    Write-Host "------------" -ForegroundColor Green
  }
}
#endregion

#region Remove Runspace
$sRSDel = @{
  FQDN = $fqdn
  API = "/api/runspaces/$($rs.id)"
  Method = 'Delete'
  Headers = $headers
}
$rsDel = Invoke-SrsMethod @sRSDel
$rsDel.Content | ConvertFrom-Json
#endregion

#region Logout
$sLogout = @{
  FQDN = $fqdn
  API = "/api/auth/logout"
  Method = 'Post'
  Headers = $headers
}
$srsLogout = Invoke-SrsMethod @sLogout
#endregion

A Function

That turned out to be a whole lot of code to just run a simple Get-VM.

But since we will be using most of the time the same logic, we can easily turn that into a function. That will make submitting code to run on the SRS a lot simpler.

And while we are at it, let’s include an option to run the code from an existing .ps1 file.

Function Invoke-SRSScript {
  [cmdletbinding()]
  param(
    [Parameter(Mandatory = $true)]
    [String]$SRSHost,
    [Parameter(ParameterSetName = 'ScriptBlock', Mandatory = $true)]
    [scriptblock]$Code,
    [Parameter(ParameterSetName = 'ScriptFile', Mandatory = $true)]
    [String]$Path,
    [Parameter(Mandatory = $true)]
    [PSCredential]$VCCredential
  )

  #region Helper functions
  function Invoke-SrsMethod {
    [cmdletbinding()]
    param(
      [Parameter(Mandatory)]
      [String]$FQDN,
      [Parameter(Mandatory)]
      [String]$API,
      [Parameter(Mandatory)]
      [String]$Method,
      [PSCredential]$Credential,
      [PSObject]$Headers,
      [PSObject]$Body
    )

    $sWeb = @{
      Uri = "https://$($FQDN)$($API)"
      Method = $Method
    }
    if ($Credential) {
      $sWeb.Add('Credential', $Credential)
    }
    if ($Headers) {
      $sWeb.Add('Headers', $Headers)
    }
    if ($Body) {
      $sWeb.Add('Body', ($Body | ConvertTo-Json))
    }

    try {
      $result = Invoke-WebRequest @sWeb
    } catch [System.Net.WebException] {
      switch ($error[0].Exception.Status) {
        ([System.Net.WebExceptionStatus]::TrustFailure) {
          if ($PSVersionTable.PSVersion.Major -lt 7) {
            if (-not ([System.Management.Automation.PSTypeName]"TrustAllCertsPolicy").Type) {
              Add-Type -TypeDefinition @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem)
    {
        return true;
    }
}
"@
            }
            if ([System.Net.ServicePointManager]::CertificatePolicy.ToString() -ne "TrustAllCertsPolicy") {
              [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
            }
          } else {
            $sWeb.Add('SkipCertificateCheck', $true)
          }
          $result = Invoke-WebRequest @sWeb
        }
        Default {
          Write-Error "Unhandled WebException $($error[0].Exception.Status)"
        }
      }
    } catch {
      Write-Error "Unhandled exception code $($error[0].Exception.gettype().Name)"
      $error[0].Exception | Format-Custom
    }

    switch ($result.StatusCode) {
      200 {
        $result
      }
      202 {
        $result
      }
      401 {
        Write-Error "Unauthorized $($result.StatusCode)"
      }
      500 {
        Write-Error "Server error $($result.StatusCode)"
      }
      Default {
        Write-Error "Unhandled StatusCode $($result.StatusCode)"
      }
    }
  }
  #endregion

  #region Preamble

  $vm = Get-VM -Name $SRSHost
  $fqdn = (Resolve-DnsName -Name $vm.ExtensionData.Guest.IpAddress).NameHost

  switch ($PSCmdlet.ParameterSetName){
    'ScriptBlock' {
      $strCode = $Code.ToString()
    }
    'ScriptFile' {
      $strCode = Get-Content -Path $Path | Out-String
    }
  }

  $headers = @{
    "accept" = "application/json"
    "content-type" = "application/json"
  }
  #endregion

  #region Login
  $sLogon = @{
    FQDN = $fqdn
    API = '/api/auth/login'
    Method = 'Post'
    Credential = $VCCredential
    Headers = $headers
  }
  $logon = Invoke-SrsMethod @sLogon
  $headers.Add('X-SRS-API-KEY', $logon.Headers['X-SRS-API-KEY'])
  #endregion

  #region Create Runspace
  $sRSCreate = @{
    FQDN = $fqdn
    API = '/api/runspaces'
    Method = 'Post'
    Credential = $cred
    Headers = $headers
    Body = @{
      name = 'MyRS'
      run_vc_connection_script = $true
    }
  }
  $createRS = Invoke-SrsMethod @sRSCreate
  $rs = $createRS.Content | ConvertFrom-Json
  #endregion

  #region Wait till RS is ready
  while ($rs.state -eq 'Creating') {
    $sRSGet = @{
      FQDN = $fqdn
      API = "/api/runspaces/$($rs.Id)"
      Method = 'Get'
      Headers = $headers
    }
    $getRS = Invoke-SrsMethod @sRSGet
    $rs = $getRS.Content | ConvertFrom-Json
  }
  #endregion

  #region Run Script
  $sSCRCreate = @{
    FQDN = $fqdn
    API = '/api/script-executions'
    Method = 'Post'
    Headers = $headers
    Body = @{
      runspace_id = $rs.id
      name = 'MyScript'
      script = $strCode
      script_parameters = @()
    }
  }
  $createSCR = Invoke-SrsMethod @sSCRCreate
  $scr = $createSCR.Content | ConvertFrom-Json
  #endregion

  #region Wait for Script to end
  while ($scr.state -eq 'running') {
    $sSCRCreate = @{
      FQDN = $fqdn
      API = "/api/script-executions/$($scr.id)"
      Method = 'Get'
      Headers = $headers
    }
    $getSCR = Invoke-SrsMethod @sSCRCreate
    $scr = $getSCR.Content | ConvertFrom-Json
  }
  #endregion

  #region Retrieve Script output
  $sSCROut = @{
    FQDN = $fqdn
    API = "/api/script-executions/$($scr.id)/output"
    Method = 'Get'
    Headers = $headers
  }
  $outSCR = Invoke-SrsMethod @sSCROut
  $outSCR.Content | ConvertFrom-Json
  #endregion

  #region Retrieve Script streams
  $streams = 'information','error','warning','debug','verbose'

  $streams | ForEach-Object -Process {
    $sSCRStr = @{
      FQDN = $fqdn
      API = "/api/script-executions/$($scr.id)/streams/$($_)"
      Method = 'Get'
      Headers = $headers
    }
    $getStream = Invoke-SrsMethod @sScrStr
    if ($getStream.Content -ne '[]') {
      $out = ($getStream.Content | ConvertFrom-Json).message | Out-String
      switch($_){
        'information' {
          Write-Information -MessageData $out
        }
        'error' {
          Write-Error -Message $out
        }
        'warning' {
          Write-Warning -Message $out
        }
        'debug' {
          Write-Debug -Message $out
        }
        'verbose' {
          Write-Verbose -Message $out
        }
      }
    }
  }
  #endregion

  #region Remove Runspace
  $sRSDel = @{
    FQDN = $fqdn
    API = "/api/runspaces/$($rs.id)"
    Method = 'Delete'
    Headers = $headers
  }
  $rsDel = Invoke-SrsMethod @sRSDel
  $rsDel.Content | ConvertFrom-Json
  #endregion

  #region Logout
  $sLogout = @{
    FQDN = $fqdn
    API = "/api/auth/logout"
    Method = 'Post'
    Headers = $headers
  }
  $srsLogout = Invoke-SrsMethod @sLogout
  #endregion

}

This Invoke-SRSScript function can be used in two variations (parametersets). With the Code parameter, you pass a ScriptBlock.

$sScript = @{
  SRSHost = 'srs1'
  Code = { Get-VM }
  VCCredential = $cred
}
Invoke-SRSScript @sScript

With the Path parameter, you point to a .ps1 file.

$sScript = @{
  SRSHost = 'srs1'
  Path = 'D:\Git\SRS-Explore\Use\Sample.ps1'
  VCCredential = $cred
}

Some Administration

The SRS comes with a number of preset values, see the Initial Configuration documentation. In some situations, you might want to change one or more of these settings.

The obvious solution is to rebuild your SRS OVA with the desired settings, and re-deploy your SRS VM. 

But you can also change these settings on the fly on an existing and running SRS. The following snippet is just an example where I change the script-runtime-service setting from the default 10 minutes to 20 minutes.

$vmName = 'srs1'

# Get the SRS credentials

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$vm = Get-VM -Name $vmName

#region Change SRS setting

$code = @'
kubectl get cm service-settings -n script-runtime-service -o yaml | sed -e 's/\("MaxRunspaceIdleTimeMinutes": \).*/\120,/' | kubectl apply -f -
'@

$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptType = 'bash'
  ScriptText = $code
}
Invoke-VMScript @sInvoke
#endregion

#region Check new SRS settings

$code = @'
kubectl get cm service-settings -n script-runtime-service -o yaml
'@

$sInvoke = @{
  VM = $vm
  GuestCredential = $cred
  ScriptType = 'bash'
  ScriptText = $code
}
Invoke-VMScript @sInvoke

#endregion

Note that such a change will not be applied immediately, it might take some time (we are talking minutes).

Epilogue

This concludes, for now, my somewhat lengthy Hitchhikers Guide to SRS 1.0.0, which resulted from my playing/testing/exploring the Script Runtime Service for vSphere (SRS) 1.0.0.

Is this something to should have gotten more attention when it was released?

Definitely!

I will surely be doing some more experimenting with the new and shining tool.

Enjoy!

Like often, and one of the VMware PowerCLI features I absolutely like, when a cmdlet is missing a feature, you can fall back on the API to solve the issue.

Introduction

Like so often, the reason I had a closer look at this issue was due to a question in PowerCLI Community on VMTN.

My first step in such cases is always to have a look at the vSphere Web Services API.

But unfortunately, the ReconfigVM_Task method documentation didn’t make me any wiser. There is apparently no object, property, or method to mount an ISO from a Content Library onto a VM.

In the next step I had a look at the vSphere REST API Reference. But in there I could also not find anything specific about Content Library ISOs and VM CD drives.

When all else fails, look at the code!

And with Code Capture we have just that option!

The first thing I noticed while looking at the generated code, there are no ‘special’ object, properties, or method used. Just a plain ReconfigVM_Task call with an edit action on the CD drive device.

The only special feature was the value in the FileName property.

That value definitely pointed to the ISO file in the Content Library. At first I tried to compose that value by hand/script. But it turned out that some parts of the value were not obtainable via normal ways.

I found out then when I use the vimdatastore provider, the returned FullDatastorePath contained exactly the value I neededd.

The Code

function Set-CDDriveCLIso {

    <#
    .SYNOPSIS
      Mount an ISO from a Content Library
      .DESCRIPTION
      This function will mount an ISO located on a Content Library
      on a CDDrive of a VM
    .NOTES
      Author:  Luc Dekens
      Version:
      1.0 24/12/20  Initial release
    .PARAMETER VM
      Specifies the virtual machines on whose guest operating systems
      you want to run the script.
    .PARAMETER CDDrive
      Specifies the CDDrive on which the ISO will be mounted
    .PARAMETER ContentLibrary
      Specifies the Content Library on which the ISO is located.
    .PARAMETER ContentLibraryIso
      Specifies the ISO item on the Content Library
    .EXAMPLE
      $cl = Get-ContentLibrary -Name MyCL
      $iso = Get-ContentLibraryItem -ContentLibrary $cl -Name MyISO
      Get-VM -Name 'MyVM' | Get-CDDrive -Name 'CD/DVD drive 0' |
      Set-CDDriveCLIso -ContentLibraryIso $iso -Confirm:$false
    .EXAMPLE
      $cd = Get-VM -Name MyVM | Get-CDDrive
      Set-CDDriveCLIso -CDDrive $cd -ContentLibraryIso $iso -Confirm:$false
    #>
    
    [cmdletbinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')]
    param(
        [parameter(Mandatory = $true, ValueFromPipeline = $true)]
        [VMware.VimAutomation.ViCore.Types.V1.VirtualDevice.CDDrive]$CDDrive,
        [parameter(Mandatory = $true)]
        [VMware.VimAutomation.ViCore.Types.V1.ContentLibrary.ContentLibraryItem]$ContentLibraryISO
    )
    
    $target = "VM:$($CDDrive.Parent.Name) CD:$($CDDrive.Name)"
    $action = "Mount ISO $($ContentLibraryISO.Name) from $($ContentLibraryISO.ContentLibrary.Name)"
        
    if ($PSCmdlet.ShouldProcess($target, $action)) {
        $driveName = -join ((65..90) | 
                Get-Random -Count 3 | 
                ForEach-Object -Process { [char]$_ })
        $filter = "$($ContentLibraryISO.Name)*.iso" 
        
        $ds = Get-Datastore -Name $ContentLibraryISO.ContentLibrary.Datastore
        
        New-PSDrive -Name $driveName -PSProvider VimDatastore -Root '\' -Location $ds | Out-Null
        $clPath = Get-ChildItem -Path "$($driveName):" -Filter "$($ContentLibraryIso.Id)" -Recurse |
            Select-Object -ExpandProperty FolderPath
        $isoPath = Get-ChildItem -Path "$($driveName):\$($clPath.Split(' ')[1])" -Filter $filter -Recurse | 
            Select-Object -ExpandProperty DatastoreFullPath
        Remove-PSDrive -Name $driveName -Confirm:$false | Out-Null
            
        $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
            
        $change = New-Object VMware.Vim.VirtualDeviceConfigSpec
        $change.Operation = [Vmware.vim.VirtualDeviceConfigSpecOperation]::edit
            
        $dev = $cd.ExtensionData
        $dev.Backing = New-Object VMware.Vim.VirtualCdromIsoBackingInfo
        $dev.Backing.FileName = $isoPath
            
        $change.Device += $dev
        $change.Device.Connectable.Connected = $true
            
        $spec.DeviceChange = $change
            
        $vm.ExtensionData.ReconfigVM($spec)
    
        Get-CDDrive -Id $CDDrive.Id
    }
}

Annotations

Line 34-37: The parameters that the function requires are objects returned by the Get-CDDrive and Get-ContentLibraryItem cmdlets. The function does not implement OBN support.

Line 40-41: This provides meaningful content for the message returned when the WhatIf switch is used.

Line 44-46: To avoid conflicts with PSDrives the user might already have defined, the function generates a 3-letter, random string that will be used as the drivename.

Line 52-53: To avoid a situation where the same ISO file might be uploaded to two different Content Libraries, the function first finds the path to the folder that corresponds with the requested Content Library.

Line 54-55: This search finds the actual path to the desired ISO file.

Line 60-72: A ‘regular’ call to the ReconfigVM_Task method.

Line 74: The function returns the CDDrive object, just like the regular Set-CDDrive cmdlet does. But since there is currently not yet support for ISO files from a Content Library, the filepath looks like a regular path to an ISO file. The Web Client does show that this is an ISO from a Content Library.

Sample Run

Using the function is rather straightforward.

$vmName = 'photonps'
$cdName = 'CD/DVD drive 1'
$cLibName = 'ConLib1'
$cLibItemName = 'photon-minimal-3.0-a0f216d'
    
$vm = Get-VM -Name $vmName
    
$cl = Get-ContentLibrary -Name $cLibName
$iso = Get-ContentLibraryItem -Name $cLibItemName -ContentLibrary $cl
    
Get-CDDrive -VM $vm -Name $cdName |
Set-CDDriveCLISO -ContentLibraryISO $iso -Confirm:$false

The Web Client shows that an ISO from a Content Library is loaded.

Enjoy!

The following is a write up of a part of session HBI1729BU ,that was presented at VMworld US 2019.

The code shown in this post is also available in the PowerCLI Community Repository.

Concept

The idea behind this snippet came when I had checked what was in $global:DefaultVIServers for the n-th time in a short PowerCLI session.

And it’s probably not only that specific variable that must be all too familiar to most of us. Any entry in the following list of variables and cmdlets must have been executed by most of us multiple times.

$Global:DefaultVIServer
$Global:DefaultVIServers
$PSVersionTable
Get-Location
Get-Module -Name VMware.PowerCLI

It would be an important timesaver, when the information returned by those instructions, would be permanently in view. At your fingertips, so to say.

The use of the PowerShell profile(s) and the Prompt function, immediately came to mind.

Not to overload your PS prompt, I decided to show part of the information in the console’s Window Title bar. That can be done by assigning a value to $host.ui.RawUI.WindowTitle. 

Implementation

The Code

#
# Provide environment information in the PS Console
#
# History:
# 1.0 - August 4th 2019 - LucD
#       Initial version (for session HBI1729BU VMworld US 2019)
#
# 1) PS prompt
# - current (local) time
# - execution time of the previous command
# - shortened PWD
# 2) Window title
# - User/Admin
# - PS-32/54-Edition-Version
# - PCLI version
# - git repo/branch
# - VC/ESXi:defaultServer-User [# connections]

function prompt
{
    # Current time
    $date = (Get-Date).ToString('HH:mm:ss')
    Write-Host -Object '[' -NoNewline
    Write-Host -Object $date -ForegroundColor Cyan -BackgroundColor DarkBlue -NoNewline
    Write-Host -Object ']' -NoNewline

    # Execution time previous command
    $history = Get-History -ErrorAction Ignore -Count 1
    if ($history)
    {
        $time = ([DateTime](New-TimeSpan -Start $history.StartExecutionTime -End $history.EndExecutionTime).Ticks).ToString('HH:mm:ss.ffff')
        Write-host -Object '[' -NoNewLine
        Write-Host -Object "$time" -ForegroundColor Yellow -BackgroundColor DarkBlue -NoNewline
        Write-host -Object '] ' -NoNewLine
    }

    # Shorted PWD
    $path = $pwd.Path.Split('\')
    if ($path.Count -gt 3)
    {
        $path = $path[0], '..', $path[-2], $path[-1]
    }
    Write-Host -Object "$($path -join '\')" -NoNewline

    # Prompt function needs to return something,
    # otherwise the default 'PS>' will be added
    "> "

    # Refresh the window's title
    Set-Title
}

function Set-Title
{
    # Running as Administrator or a regular user
    $userInfo = [Security.Principal.WindowsIdentity]::GetCurrent()
    if ((New-Object Security.Principal.WindowsPrincipal $userInfo).IsInRole([Security.Principal.WindowsBuiltinRole]::Administrator))
    {
        $role = 'Admin'
    }
    else
    {
        $role = 'User'
    }

    # Usertype user@hostname
    $user = "$role $($userInfo.Name)@$($env:computername)"

    # PowerShell environment/PS version
    $bits = 32
    if ([Environment]::Is64BitProcess)
    {
        $bits = 64
    }
    $ps = " - PS-$($bits): $PSEdition/$($PSVersionTable.PSVersion.ToString())"

    # PowerCLI version (derived from module VMware.PowerCLI)
    $pcliModule = Get-Module -Name VMware.PowerCLI -ListAvailable |
    Sort-Object -Property Version -Descending |
    Select-Object -First 1
    $pcli = " - PCLI: $(if($pcliModule){$pcliModule.Version.ToString()}else{'na'})"

    # If git is present and if in a git controlled folder, display repositoryname/current_branch
    $gitStr = ''
    if ((Get-Command -Name 'git' -CommandType Application -ErrorAction SilentlyContinue).Count -gt 0)
    {
        $gitTopLevel = git rev-parse --show-toplevel 2> $null
        if ($gitTopLevel.Length -ne 0)
        {
            $gitRepo = Split-Path -Path $gitTopLevel -Leaf
            $gitBranch = (git branch | Where-Object { $_ -match "\*" }).Trimstart('* ')
            $gitStr = " - git: $gitRepo/$gitBranch"
        }
    }

    # If there is an open vSphere Server connection
    # display [VC|ESXi] last_connected_server-connected_user [number of open server connections]
    if ($global:defaultviserver)
    {
        $vcObj = (Get-Variable -Scope global -Name 'DefaultVIServer').Value
        if ($vcObj.ProductLine -eq 'vpx')
        {
            $vcSrv = 'VC'
        }
        else
        {
            $vcSrv = 'ESXi'
        }
        $vc = " - $($vcSrv): $($vcObj.Name)-$($vcObj.User) [$($global:DefaultVIServers.Count)]"
    }

    # Update the Window's title
    $host.ui.RawUI.WindowTitle = "$user$ps$pcli$vc$gitStr"
}

# Set title after starting session
Set-Title

Annotations

Line 19-51: Replacement function for the PS prompt.

Line 22-25: Current time in the format ‘HH:mm:ss’. Note that ‘HH’ asks for a 24-hour value.

Line 28-35: The execution time of the previous command, in the format ‘HH:mm:ss:ffff’.

Line 28: The execution time is based on the content of PS History. With Get-History we obtain the start time of the command, and with New-TimeSpan we obtain the duration for running that command.

Line 38-43: These lines take the path to the current folder, and shorten that path to only show the start of the path and the last two directories. When the path is shortened, it is indicated by two dots.

Line 53-114: Function to populate the Windows Title bar

Line 56-64: These lines determine if the PS session is started “As Administrator” or as a regular user.

Line 67: The user and station information.

Line 70-75: The PS edition (Desktop or Core) and the PS version.

Line 78-81: If installed, the version of VMware PowerCLI. Note that this looks at the VMware.PowerCLI module to obtain that version number.

Line 84-94: If the git command is present on the station, these lines will check if the current directory is inside a git repository. It will then display the name of the git repository and the current branch.

Line 98-110: These lines check if there is an open connection to a vSphere Server. It will show if the connected vSphere Server is a vCenter or an ESXi node.

Line 109: If multiple connections to a vSphere Servers are open, this will display the number of these connections. The vSphere Server that is shown is the last connected one. This corresponds with what is kept in the $global:DefaultVIServer variable.

Line 117: Initial call to the Set-Title function. This will populate the Windows Title bar when the PS console is opened.

Installation

You can just copy and rename the code to one of the profile files. To make that process easier, I also provide the following installation script.

[cmdletbinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')]
param(
    #        [Parameter(Mandatory = $true)]
    [ValidateSet('CurrentUserCurrentHost', 'CurrentUserAllHosts',
        'AllUsersCurrentHost', 'AllUsersAllHosts')]
    [string]$Scope,
    [switch]$NoClobber,
    [switch]$Backup,
    [string]$NewProfile = '.\NewProfile.ps1'
)

if ($PSCmdlet.ShouldProcess("$($Profile.$Scope)", "Create $Scope profile"))
{
    $profilePath = $Profile."$Scope"
    Write-Verbose -Message "Target is $profilePath"
    $createProfile = $true
    if (Test-Path -Path $profilePath)
    {
        Write-Verbose -Message "Target exists"
        if ($NoClobber)
        {
            Write-Verbose -Message "Cannot overwrite target due to NoClobber"
            $createProfile = $false
        }
        elseif ($Backup)
        {
            Write-Verbose -Message "Create a backup as $profilePath.bak"
            Copy-Item -Path $profilePath -Destination "$profilePath.bak" -Confirm:$false -Force
        }
        elseif (-not $NoClobber)
        {
            Write-Verbose -Message "Target will be overwritten"
        }
        else
        {
            Write-Verbose -Message "Use -NoClobber:$false or -Backup"
        }
    }
    if ($createProfile)
    {
        if (-not $NewProfile)
        {
            $script:MyInvocation.MyCommand | select *
            $folder = Split-Path -Parent -Path $script:MyInvocation.MyCommand.Path
            $folder = Get-Location
            $NewProfile = "$folder\NewProfile.ps1"
        }
        Write-Verbose -Message "New profile expected at $NewProfile"
        if (Test-Path -Path $NewProfile)
        {
            Write-Verbose -Message "Copy $NewProfile to $profilePath"
            Copy-Item -Path $NewProfile -Destination $profilePath -Confirm:$false
        }
        else
        {
            Write-Warning -Message "Could not find the new profile file!"
            Write-Warning -Message "Use the NewProfile parameter or store a NewProfile.ps1 file in folder $folder."
        }
    }
}

Save the above code to a .ps1 file, let’s say install-profile.ps1. Also copy the profile code to a .ps1 file, let’s say newprofile.ps1.

You can now install this new profile with a simple call.

.\install-profile.ps1 -NewProfile .\NewProfile.ps1 -Scope CurrentUserCurrentHost -Backup 

The installation script has a number of parameters:

• NewProfile : the file that contains the profile you want to install
• Scope : for which Scope you want to install the new profile.
• Backup : backup the current profile before overwriting it.
• NoClobber : avoids overwriting an existing profile file.

Usage

When the new profile is installed, and when you restart your console session, you will see the new profile in action.

The following screenshot shows the features of the new profile.

And the following short video shows the installation process and the features the profile offers.

Enjoy!

Over the last couple of weeks, it has been my great pleasure to assist Chip Zoller in writing a VMware PowerCLI script, named Optimize-VMwarePKS. This script helps organise your PKS deployment at three levels: folders, tags and DRS rules, including functionality to run a clean up.

The following post by Chip describes the function in greater detail, and shows how you can use it.

This post also appears on the Sovereign Systems website as Optimize-VMwarePKS: A PowerShell Script for All Your VMware PKS Deployment Needs.

Take it away Chip.

Optimize-VMwarePKS: A PowerShell script for all your VMware PKS deployment needs

            Ever since VMware PKS (now called Enterprise PKS) came onto the market over a year ago, it’s been a big hit. With it, you get upstream Kubernetes, NSX-T, an enterprise-class container registry, automation of the entire K8s cluster creation process, and lots more all on top of the de facto private cloud platform of vSphere. It’s truly becoming the way organizations are standardizing on K8s cluster instantiation, upgrade, and management on-premises. The deployment nodes themselves are regular virtual machines. And, like all virtual machines, you and your company probably have an established way of organizing those VMs. While Enterprise PKS takes care of a lot of tasks automatically, one thing that we still need to consider is how those VMs get organized. So, as a joint effort between myself and Luc Dekens, we’re glad to announce today a new script called Optimize-VMwarePKS which is designed to bring that level of optimization to your vSphere and PKS clusters. Specifically, this script is designed to optimize your PKS deployments in three ways: vSphere folders, vSphere tags, and DRS rules–with clean-up for three of these. Read on to see this in action and learn how you can greatly simplify deployment of PKS clusters in your own environment.

FOLDERS        

As mentioned, PKS deploys K8s nodes as simple VMs into your vSphere environment. After a single deployment, you may see something like this in your vSphere Client.

All VMs are, as you can see, named with IDs that are doled out by PKS and BOSH, and tracked internally. They probably look very different from the internal naming standard you’ve developed, right? Although we shouldn’t change the names of the VMs, we can do other things. The first feature of Optimize-VMwarePKS is the ability to place all VMs from a given deployment automatically into a folder based on that name. In so doing, the above screenshot now looks like this after optimization:

As you can see, these folder names align with the PKS cluster names perfectly if we examine them with a pks clusters command. In fact, Optimize-VMwarePKS is designed to take this information directly from the PKS CLI tool, which is a requirement for the script (see later).

The nice thing about this script is

1. it’s very easy to operate (simply add the –ProcessFolders switch)
2. it’s idempotent so will keep this organization no matter how many times you run it.

TAGS

            The second feature of Optimize-VMwarePKS to highlight is the ability to assign vSphere Tags to all of the VMs that form a given PKS deployment. Tags are often used to provide a form of organization that may exist alongside or independent of vSphere folders. These tags are, again by default, created based on the name of the PKS cluster and automatically created and assigned to all VMs part of a cluster. With a tag category of your choice (or by accepting the default category of ‘PKS’), the script will detect your clusters, create a new appropriate tag in the tag category, and assign that tag to all VMs.

As you can see here, the script has created two tags for us that match the names of our clusters. If we inspect the VMs within those clusters, we can see it’s been correctly assigned as well.

This optimization is easily enabled with the -ProcessTags switch.

DRS RULES

           Since PKS has the ability to deploy HA K8s clusters–which is a tremendous strength and huge time saver–it automatically provisions multiple masters and places them behind an NSX-T load balancer. These masters then serve as the API entry point for all K8s-related commands. Masters can be spread out across multiple Availability Zones (AZ). Where AZs map directly onto distinct vSphere clusters, this provides a sufficient level of fault isolation. But in the case where either multiple Azs aren’t in use or they do not align to separate compute clusters, Enterprise PKS will place all masters on the same cluster. Optimize-VMwarePKS can therefore be used to build DRS anti-affinity rules to provide such separation.

            In my previous screenshots, you’ve noticed I have a single small cluster (1 master; 3 workers) and a large cluster (3 masters; 3 workers). If we run the script with the -ProcessDRSRules switch, it will automatically find those masters and create a DRS anti-affinity rule for us. And not to worry, it’ll only create the rule with the masters of each cluster, so multiple HA clusters get individual DRS rules created. See below. Even better, the script is written in a future-proof way in that when VMware/Pivotal support larger multi-master clusters in the future, if you scaled out masters the script will detect the increase and update the rule for you!

These abilities are, as I hope you’ll agree, extremely useful in order to not only fill some gaps but bring VMware PKS deployments into alignment with your vSphere policies today. But there’s one thing missing that we haven’t covered yet. Kubernetes clusters may come and go (another big win for Enterprise PKS is the automatic roll-back of NSX-T objects!), and when they do we want to ensure we aren’t leaving behind elements that are no longer relevant. So the last portion of Optimize-VMwarePKS is its ability to automatically maintain a tidy house.

TIDYING UP

            Each of the three optimizations shown previously (folders, tags, and DRS rules) can be switched on independently. And with each one, except DRS rules, if a cluster gets deleted we’d have some clean-up work to do. So the script has also been written with switches that you can (optionally, of course) automate the clean-up of these objects. We’re talking here about tidying up of vSphere folders and vSphere tags. By adding the -TidyFolders and/or -TidyTags switches, Optimize-VMwarePKS will search for these empty folders and tags and remove them for you. But, not to fear, it will *only* do this if you’ve provided these switches and also only consider the parent vSphere folder or tag category for clean-up. That means it won’t scour your entire estate and remove anything else.

IDEMPOTENCE

            Lastly, and before we look at some running examples, is to point out the idempotent nature of this script. Because you may have many groups of people bring up and down PKS deployments, you need something that can maintain a ready state of what you define, similar to tools like Ansible, Puppet, and Chef. Being able to schedule this script is one of the main reasons it was written, and in so doing it must be made idempotent. No matter how many times you run the script, it always checks if things exist before making a change. You can be confident that when you ask it to run with whatever parameters that it’ll maintain that state exactly. So no fear if someone drops a tag or moves a VM out of a folder. Next time it runs it’ll put it right back where it was.

RUNNING OPTIMIZE-VMWAREPKS

            Now you’re aware of all its capabilities, let’s quickly look at what’s needed to run it and then an example on how to run it.

            The script needs PowerShell, VMware PowerCLI, and the PKS CLI binary tool, all of which are cross-platform at this point. In fact, it has been tested running directly from the Pivotal Operations Manager appliance running Ubuntu with PowerShell Core but runs perfectly fine from a Windows control machine as well. The PKS CLI tool needs to be added to your PATH. You also must ensure you have credentials and network connectivity to access both vCenter Server and PKS.

            Now you have the prereqs, time to actually run this script. Grab it from the GitHub repo here. For your convenience (and so you don’t have to either read a blog or inspect the code), extensive help has been provided on how to operate the script, its parameters, but some examples.

            All the switches can be run independently, so you can mix and match whatever functionality you like and only it will run. If we want to let it process folders, tags, and DRS rules for us, we could call it like so:

$secvCPass = ConvertTo-SecureString -String 'VMware1!' -asPlainText -Force
$credvC = New-Object System.Management.Automation.PSCredential('administrator@vsphere.local',$secvCPass)
$secPKSPass = ConvertTo-SecureString -String 'VMware1!' -asPlainText -Force
$credPKS = New-Object System.Management.Automation.PSCredential('myuser',$secPKSPass)

./Optimize-VMwarePKS.ps1 -ProcessFolders -ProcessTags -ProcessDRSRules -vCenter $vc -vCenterCredential $credvC -PKSSever $pks -PKSCredential $credPKS -Verbose

We have to pass a PSCredential object to the script, and so the lines before actually running it do that for us.

If all you wanted was to clean-up (tidy) those objects, run it with only those flags set:

./Optimize-VMwarePKS.ps1 -TidyFolders -TidyTags -vCenter $vc -vCenterCredential $credvC -PKSSever $pks -PKSCredential $credPKS

Or, combine all flags into a single command:

./Optimize-VMwarePKS.ps1 -ProcessFolders -ProcessTags -ProcessDRSRules -TidyFolders -TidyTags -vCenter $vc -vCenterCredential $credvC -PKSSever $pks -PKSCredential $credPKS

As far as managing the parameters needed, one of the most convenient ways is with splatting in PowerShell. We can write a script, call it “wrapper.ps1” and put all parameters inside of it as shown below. Note that with splatting, we must set the values to “true” unlike when passing the switches to the script directly. The final line simply calls Optimize-VMwarePKS for us passing the entire splat as an argument.

$vCUser = 'myuser@mydomain.com'
$vCPass = 'VMware1!' | ConvertTo-SecureString -asPlainText -Force
$PKSUser = 'myuser'
$PKSPass = 'VMware1!' | ConvertTo-SecureString -asPlainText -Force

$params = @{
ProcessFolders = $true
ProcessTags = $true
ProcessDRSRules = $true
TidyFolders = $true
TidyTags = $true
vCenter = 'myvcenter.mydomain.com'
PKSServer = 'mypks.mydomain.com'
vCenterCredential = New-Object System.Management.Automation.PSCredential($vCUser,$vCPass)
PKSCredential = New-Object System.Management.Automation.PSCredential($PKSUser,$PKSPass)
}

/home/ubuntu/Optimize-VMwarePKS.ps1 @params

And, as mentioned earlier, because this script is entirely idempotent, it can easily be scheduled with either cron, Windows task scheduler, or any other mechanism you want. This will ensure your environment is kept optimized based on your rules and patterns with no manual clean-up required.

            I hope you find this script to be valuable in your environments. If so, we’d love to hear about it and you can drop either Luc or Chip a line on Twitter at @LucD22 or @chipzoller respectively.

Thanks Chip.

Enjoy!

The next question on your mind is probably “How can I contribute?“. Well, with the correct tools and some VMware PowerCLI knowledge, it turns out that this is not too difficult. What follows is my first attempt at contributing to the Desired State Configuration Resources for VMware.

Repositories, tools and clones

For most of you what follows might probably state the obvious, but I included all the steps for completeness.

The Repository

The DSCR for VMware are published in a GitHub repository. So you will need access to GitHub. Create yourself a GitHub account, see Signing up for a new GitHub account.

Once you are on GitHub, go to the DSCR for VMware repository. Since you don’t want to be messing with that code directly, create a fork of the repository. For instructions on that see Fork a Repo.

That completes the repository part. You now have a copy of the  DSCR for VMware repository under your own GitHub account.

Tools

I understand that you are eager to add your own code to the DSCR for VMware module, but there are a few other preparatory steps you will have to take before you can actually start coding.

Determine on which station you are going to do your coding. On this station you will have to install some tools and some PowerShell modules.

Git commands

Since you will be working with a GitHub repository, you will need the Git commands. You can download the installation file from the Git – Downloads page.

Git GUI (optional)

This is not really needed, but it can make your life a lot easier. Instead of entering all the Git commands from a prompt, you can go for a Git GUI. There are several of these available, each with its own pro and cons.

Personally I prefer to use GitKraken. 

Editor

Your choice of editor can be whatever you prefer to work with PowerShell code. My editor of choice is currently Visual Studio Code with the PowerShell Extension.

Modules

To contribute to the DSCR for VMware module, and follow the Coding guidelines, you will need to install the following PowerShell modules on your station.

• The VMware PowerCLI modules
  • See Welcome PowerCLI to the PowerShell Gallery – Install Process Updates
• The Pester module
  • If not already present on your station install (Install-Module), or update (Update-Module).

A “work” copy

We are nearly there.

To start coding, you will need to clone your copy of the repository (remember the fork earlier on) to your station.

With a Git GUI this is a simple process, provided you have set up the connection to your GitHub account beforehand.

1. Select the Clone option
2. Select your GitHub connection
3. Select the target Folder for the clone
4. Specify the name of the clone folder
5. Clone the repository

To create a branch, right click on the Master entry on the left side and then enter the name of your branch in the text box. In the example I create a branch named dev-lucd.

Once the branch is created, it becomes the active path. Meaning that when I open the local files, I’ll be working in the dev-lucd branch. And while I’m doing my coding, the master branch stays unchanged. That is one of the reasons we use repositories.

In summary, and to show how easy it is.

Coding

Intro

Now let’s get coding, but first some basic information for first time users, or as a refresher for the others.

Editor

One of the nice features of the Visual Studio Code editor, is that you can open a folder. This way all files in this folder will be shown in the Explorer.

Selecting a file in the Explorer will open that file in an Editor window.

Git Integration

A nice and useful feature of the Visual Studio Code editor is that it offers builtin Git integration. From within the editor you have access to the most common the Git commands.

Resource Organisation

Due to current limitations, all DSC resources are contained in the module’s single .psm1 file, namely VMware.vSphereDSC.psm1.

The DSC resources that are exported, are specified in the module’s .psd1 file (VMware.vSphereDSC.psd1) under the DscResourcesToExport entry.

There is a third file, named VMware.vSphereDSC.Helper.psm1, which contains a number of “helper” functions.

The DSC resources are defined as PowerShell classes. This allows the important feature of class inheritance.

A DSC resource, defined as a class, needs to have at least three methods Get, Set and Test. You can find more information in Writing a custom DSC resource with PowerShell classes.

Our First DSC Resource

Background

As a proof of concept and to document the workflow, this post will document how we can add a DSC Resource for the Issues message and the MoTD message on ESXi nodes and on a VCSA.

The theory behind these two features, motd and issue, is quite straightforward, and there are ample blog posts around the subject. Have for example a look at Wiliam Lam’s Easily manage ESXi & VCSA SSH login banner & MOTD in vSphere 6.0 post.

It all comes down to setting or clearing the two advanced settings on the target ESXi node ir VCSA.

#
# ESXi node
#

# Get
Get-AdvancedSetting -Entity $esx -Name 'Config.Etc.issue'
Get-AdvancedSetting -Entity $esx -Name 'Config.Etc.motd'

# Set
Get-AdvancedSetting -Entity $esx -Name 'Config.Etc.issue' |
Set-AdvancedSetting -Value 'Hello World!'
Get-AdvancedSetting -Entity $esx -Name 'Config.Etc.motd' |
Set-AdvancedSetting -Value 'Hello World!'

#
# VCSA
#

# Get
Get-AdvancedSetting -Entity $esx -Name 'etc.issue'
Get-AdvancedSetting -Entity $esx -Name 'etc.motd'

# Set
Get-AdvancedSetting -Entity $esx -Name 'etc.issue' |
Set-AdvancedSetting -Value 'Hello World!'
Get-AdvancedSetting -Entity $esx -Name 'etc.motd' |
Set-AdvancedSetting -Value 'Hello World!'

There is one caveat with these advanced settings, how do differentiate between clearing an entry and not passing a value in a parameter. One solution would be to make the property that defines the motd and issue, a mandatory property. But this brings other disadvantages, like obliging you to add this property to each configuration file.

I decided to go for a separate, also not mandatory, property that explicetely asks for clearing the content of the motd and issue setting. You will see how this was implemented later on in this post.

Class setup

From the Coding Guidelines we know how we have to set up our classes for the new resources.

The resource for the ESXi based resource inherits from VMHostBaseDSC and BaseDSC. 

The resource for the VCSA based resource inherits from BaseDSC.

As an example, new DSC resources for ESXi and VCSA would start like this. Note that this just a skeleton, you will most probably have to add properties and methods.

class vCenterMyResource : BaseDSC
{}

class VMHostMyResource : VMHostBaseDSC
{}

With the class inheritance, your class will have a number of inherited properties and methods. The following screenshot shows what you get from those two base classes.

Two examples

Expand an Existing Resource

The first release of the DSCR for VMware module already contains a resource named vCenterSettings. Let’s try to add the motd and issue entries to this class. As a first step, I add the properties I will need to control the configuration of the motd and issue settings.

  <#
    .DESCRIPTION

    Motd value.
    #>
  [DscProperty()]
  [string] $Motd

  <#
    .DESCRIPTION

    Clear the Motd content
    #>
  [DscProperty()]
  [bool] $MotdClear

  <#
    .DESCRIPTION

    Issue value.
    #>
  [DscProperty()]
  [string] $Issue
  <#
    .DESCRIPTION

    Clear the Issue content
    #>
  [DscProperty()]
  [bool] $IssueClear


  hidden [string] $LogLevelSettingName = "log.level"
  hidden [string] $EventMaxAgeEnabledSettingName = "event.maxAgeEnabled"
  hidden [string] $EventMaxAgeSettingName = "event.maxAge"
  hidden [string] $TaskMaxAgeEnabledSettingName = "task.maxAgeEnabled"
  hidden [string] $TaskMaxAgeSettingName = "task.maxAge"
  hidden [string] $MotdSettingName = "etc.motd"
  hidden [string] $IssueSettingName = "etc.issue"

This is only a partial extract of the vCenterSettings class properties. Also note how we added the paths to the advanced settings in two hidden properties ($MotdSettingName and $IssueSettingName).

Next we will have to update the existing methods already available in the class, in such a way that they can handle the two new advanced settings we are adding. We don not have to touch the three required methods Set, Test and Get, that every DSC resource class must have, since they are already written in a general way without any specific dependencies on the Advanced Settings.

One such method we have to adapt is the ShouldUpdatevCenterSettings method. The logic in this method is quite straightforward.

• Get all the Advanced Settings
• With the Name of the setting (see the hidden properties from earlier), the method retrieves current Value for the Advanced Setting
• With the ShouldUpdateSettingValue method the requested value is compared with the actual value. This method returns a result Boolean with the outcome of that comparison
• For the Motd and Issue settings, the MotdClear and IssueClear comes into play. As remarked earlier, this allows us to clear the content of advanced setting without the need to make it a Mandatory Property in the class
• If any advanced setting needs to be changed, the method returns $true, otherwise it returns $false

  [bool] ShouldUpdatevCenterSettings($vCenter) {
    $vCenterCurrentAdvancedSettings = Get-AdvancedSetting -Server $this.Connection -Entity $vCenter

    $currentLogLevel = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.LogLevelSettingName }
    $currentEventMaxAgeEnabledValue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.EventMaxAgeEnabledSettingName }
    $currentEventMaxAgeValue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.EventMaxAgeSettingName }
    $currentTaskMaxAgeEnabledValue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.TaskMaxAgeEnabledSettingName }
    $currentTaskMaxAgeValue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.TaskMaxAgeSettingName }
    $currentMotdValue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.MotdSettingName }
    $currentIssueValue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.IssueSettingName }

    $shouldUpdate = @()
    $shouldUpdate += $this.ShouldUpdateSettingValue($this.LoggingLevel, $currentLogLevel.Value)
    $shouldUpdate += $this.ShouldUpdateSettingValue($this.EventMaxAgeEnabled, $currentEventMaxAgeEnabled.Value)
    $shouldUpdate += $this.ShouldUpdateSettingValue($this.EventMaxAge, $currentEventMaxAge.Value)
    $shouldUpdate += $this.ShouldUpdateSettingValue($this.TaskMaxAgeEnabled, $currentTaskMaxAgeEnabled.Value)
    $shouldUpdate += $this.ShouldUpdateSettingValue($this.TaskMaxAge, $currentTaskMaxAge.Value)
    $shouldUpdate += ($this.MotdClear -and ($currentMotd.Value -ne '')) -or (-not $this.MotdClear -and ($this.Motd -ne $currentMotd.Value))
    $shouldUpdate += $this.ShouldUpdateSettingValue($this.Issue, $currentIssueValue)
    $shouldUpdate += ($this.IssueClear -and ($currentIssue.Value -ne '')) -or (-not $this.MotdClear -and ($this.Motd -ne $currentIssue.Value))

    return ($shouldUpdate -contains $true)
  }

The methods that actually change the Value of an Advanced Setting, take care of the MotdClear and IssueClear properties. In fact we use that fact that we can define separate methods depending on the number and type of parameters we pass to a method. So there is a separate method SetAdvancedSetting when we call the method with four parameters.

[void] SetAdvancedSetting($advancedSetting, $advancedSettingDesiredValue, $advancedSettingCurrentValue) {
    if ($this.ShouldUpdateSettingValue($advancedSettingDesiredValue, $advancedSettingCurrentValue)) {
      Set-AdvancedSetting -AdvancedSetting $advancedSetting -Value $advancedSettingDesiredValue -Confirm:$false
    }
  }

  [void] SetAdvancedSetting($advancedSetting, $advancedSettingDesiredValue, $advancedSettingCurrentValue, $clearValue) {
    Write-Verbose -Message "$(Get-Date) $($s = Get-PSCallStack; "Entering {0}" -f $s[0].FunctionName)"

    if ($clearValue) {
      if ($this.ShouldUpdateSettingValue('', $advancedSettingCurrentValue)) {
        Set-AdvancedSetting -AdvancedSetting $advancedSetting -Value '' -Confirm:$false
      }
    }
    else {
      if ($this.ShouldUpdateSettingValue($advancedSettingDesiredValue, $advancedSettingCurrentValue)) {
        Set-AdvancedSetting -AdvancedSetting $advancedSetting -Value $advancedSettingDesiredValue -Confirm:$false
      }
    }
  }

[void] UpdatevCenterSettings($vCenter) {
    $vCenterCurrentAdvancedSettings = Get-AdvancedSetting -Server $this.Connection -Entity $vCenter

    $currentLogLevel = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.LogLevelSettingName }
    $currentEventMaxAgeEnabled = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.EventMaxAgeEnabledSettingName }
    $currentEventMaxAge = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.EventMaxAgeSettingName }
    $currentTaskMaxAgeEnabled = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.TaskMaxAgeEnabledSettingName }
    $currentMotd = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.MotdSettingName }
    $currentIssue = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.IssueSettingName }

    $currentTaskMaxAge = $vCenterCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.TaskMaxAgeSettingName }
    $this.SetAdvancedSetting($currentLogLevel, $this.LoggingLevel, $currentLogLevel.Value)
    $this.SetAdvancedSetting($currentEventMaxAgeEnabled, $this.EventMaxAgeEnabled, $currentEventMaxAgeEnabled.Value)
    $this.SetAdvancedSetting($currentEventMaxAge, $this.EventMaxAge, $currentEventMaxAge.Value)
    $this.SetAdvancedSetting($currentTaskMaxAgeEnabled, $this.TaskMaxAgeEnabled, $currentTaskMaxAgeEnabled.Value)
    $this.SetAdvancedSetting($currentTaskMaxAge, $this.TaskMaxAge, $currentTaskMaxAge.Value)
    $this.SetAdvancedSetting($currentMotd, $this.Motd, $currentMotd.Value,$this.MotdClear)
    $this.SetAdvancedSetting($currentIssue, $this.Issue, $currentIssue.Value,$this.IssueClear)
  }

There are a number of other changes I had to make in the vCenterSettings class, but I leave it to the reader to further inspect those once my changes are merged in the repository.

A New Resource

There is currently no resource yet for Advanced Settings for ESXi nodes. In this new class I will try to create such a resource, and for starters only provide the motd and issue settings. It should be rather trivial to add other advanced settings later on.

[DscResource()]
class VMHostSettings : VMHostBaseDSC {
  <#
    .DESCRIPTION

    Motd value.
    #>
  [DscProperty()]
  [string] $Motd

  <#
    .DESCRIPTION

    Clear the Motd content
    #>
  [DscProperty()]
  [bool] $MotdClear

  <#
    .DESCRIPTION

    Issue value.
    #>
  [DscProperty()]
  [string] $Issue

  <#
    .DESCRIPTION

    Clear the Issue content
    #>
  [DscProperty()]
  [bool] $IssueClear


  hidden [string] $IssueSettingName = "Config.Etc.issue"
  hidden [string] $MotdSettingName = "Config.Etc.motd"

Note how use two hidden properties ($IssueSettingName and $MotdSettingName) in the class to specify the path to both advanced settings.

The basic layout of the methods we need in the class are more or less copies of the methods we also found in the vCenterSettings class. The major difference is that I left out the ShouldUpdateSettingValue method, since it was not required for the motd and issue settings. The decision if a change is required is already made in the ShouldUpdateVMHostSettings method.

[bool] ShouldUpdateVMHostSettings($VMHost) {
    Write-Verbose -Message "$(Get-Date) $($s = Get-PSCallStack; "Entering {0}" -f $s[0].FunctionName)"

    $VMHostCurrentAdvancedSettings = Get-AdvancedSetting -Server $this.Connection -Entity $VMHost

    $currentMotd = $VMHostCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.MotdSettingName }
    $currentIssue = $VMHostCurrentAdvancedSettings | Where-Object { $_.Name -eq $this.IssueSettingName }

    $shouldUpdate = @()
    $shouldUpdate += ($this.MotdClear -and ($currentMotd.Value -ne '')) -or (-not $this.MotdClear -and ($this.Motd -ne $currentMotd.Value))
    $shouldUpdate += ($this.IssueClear -and ($currentIssue.Value -ne '')) -or (-not $this.IssueClear -and ($this.Issue -ne $currentIssue.Value))

    return ($shouldUpdate -contains $true)
  }

Debugging

So you have coded your new resource, or update an existing, and you want to try it out.

As a first step you would run your Pester Unit tests, and fix all the issues you encounter in there. There is a section later on on these Pester tests.

The next step would of course be that you are going to try and use your resource on a test environment. But, just like always happens to me, you might notice some errors and/or issues when you run your tests.

As I imagine that using and writing DSC resources might be new territory for some of you, I added this Debugging section. It is definitely not final, but just lists some of the handy tricks I discovered/learned while developing DSC resources.

How do you test these?

First, make sure that you have read Getting Started with Desired State Configuration Resources for VMware. The only comment I can make on that post is that for “Kyle and the beard” everything always works

For us mere mortals some handy tips.

• Configure the LCM agent on the station you will test on to ignore configuration signing and to work in debug mode (that will force the LCM to reload the DSC module each time, which is handy when you are making changes to the module)

[DSCLocalConfigurationManager()]
configuration LCMConfig
{
    Node localhost
    {
        Settings
        {
            RefreshMode = 'Push'
            ConfigurationMode = 'ApplyAndMonitor'
            SignatureValidations  = @{}
            DebugMode = 'ForceModuleImport'
        }
    }
}

LCMconfig
Set-DscLocalConfigurationManager -Path '.\LCMconfig' -ComputerName localhost

• Use the Verbose switch when applying a configuration with Start-DscConfiguration. It provides you with a lot of information on what your DSC module is doing.

$motdConfigInput = @{
    name = 'esx.local.lab'
    server = 'vcsa.local.lab'
    user = 'administrator@vsphere.local'
    password = 'VMware1!'
    motd = 'Hello Stranger!'
    issue = 'Enter on your own risk!'
}

. .\VMHostSettings_Config.ps1 @motdConfigInput

$sConfig = @{
    ComputerName = 'localhost'
    Path = '.\VMHostSettings_Config'
    Verbose = $true
    Wait = $true
    Force = $true
}
Start-DscConfiguration @sConfig

• Write your own verbose messages in your DSC resource classes. With a simple Write-Verbose in your code, you can provide debugging information. I for example, add a message at the start of each method that shows the code has entered that method. You will find some examples in the verbose output above

Write-Verbose -Message "$(Get-Date) $($s = Get-PSCallStack; "Entering {0}" -f $s[0].FunctionName)"

Configurations

The ultimate goal of your DSC resource is of course to use it to configure and monitor parts of your vSphere environment.

As was shown, perhaps implicitly, in Getting Started with Desired State Configuration Resources for VMware the commonly accepted way to apply DSC configurations, is to separate configuration from environment data. See also Separating configuration and environment data.

In practice this comes down to having code, that is static, unless you add properties to your DSC resource class, and code that contains the actual configuration data. These two types of data can be combined in one .ps1 file, or they can be kept in two separate .ps1 files.

For our newly created DSC resource, we would have a configuration file, something like this. Note how this file works with parameters, and that nothing would need to be changed to this file, while we change the configuration itself, the so-called environmental data.

param(
        [Parameter(Mandatory = $true)]
        [string]
        $Name,

        [Parameter(Mandatory = $true)]
        [string]
        $Server,

        [Parameter(Mandatory = $true)]
        [string]
        $User,

        [Parameter(Mandatory = $true)]
        [string]
        $Password,

        [string]
        $Motd,

        [string]
        $Issue,

        [Boolean]$MotdClear = $false,

        [Boolean]$IssueClear = $false
)

$script:configurationData = @{
    AllNodes = @(
        @{
            NodeName = 'localhost'
            PSDscAllowPlainTextPassword = $true
        }
    )
}

Configuration VMHostSettings_Config
{
    Import-DscResource -ModuleName VMware.vSphereDSC

    Node localhost
    {
        $Password = $Password | ConvertTo-SecureString -AsPlainText -Force
        $Credential = New-Object System.Management.Automation.PSCredential($User, $Password)

        VMHostSettings vmHostSettings
        {
            Name = $Name
            Server = $Server
            Credential = $Credential
            Motd = $motd
            MotdClear = $MotdClear
            Issue = $Issue
            IssueClear = $IssueClear
        }
    }
}

VMHostSettings_Config -ConfigurationData $script:configurationData

The environmental data is kept in a separate file. Something like this for example.

$motdConfigInput = @{
    name = 'esx1.local.lab'
    server = 'vcsa.local.lab'
    user = 'administrator@vsphere.local'
    password = 'VMware1!'
    motd = 'Hello Stranger!'
    issue = 'Welcome to my world!'
}

. .\VMHostSettings_Config.ps1 @motdConfigInput

$sConfig = @{
    ComputerName = 'localhost'
    Path = '.\VMHostSettings_Config'
    Verbose = $true
    Wait = $true
    Force = $true
}
Start-DscConfiguration @sConfig

As you notice, we call the other file (VMHostSettings_Config.ps1). This script will generate the MOF file, and will apply the configuration with Start-DscConfiguration. Notice how I added the Verbose switch to have a better view on what is happening. Once your DSC resource is sufficiently tested and considered stable, you might remove the Verbose switch.

Since the proof is in the pudding, this is what a SSH session looks like after the configuration was applied.

For the vCenterSettings resource we can apply our configuration in the same way. First our configuration part.

Note that I left out the parameters for the settings that were already available in this vCenterSettings resource.

param(
        [Parameter(Mandatory = $true)]
        [string]
        $Server,

        [Parameter(Mandatory = $true)]
        [string]
        $User,

        [Parameter(Mandatory = $true)]
        [string]
        $Password,

        [string]
        $Motd,

        [string]
        $Issue,

        [Boolean]$MotdClear = $false,

        [Boolean]$IssueClear = $false
)

$script:configurationData = @{
    AllNodes = @(
        @{
            NodeName = 'localhost'
            PSDscAllowPlainTextPassword = $true
        }
    )
}

Configuration vCenterSettings_Config
{
    Import-DscResource -ModuleName VMware.vSphereDSC

    Node localhost
    {
        $Password = $Password | ConvertTo-SecureString -AsPlainText -Force
        $Credential = New-Object System.Management.Automation.PSCredential($User, $Password)

        vCenterSettings vCenterSettings
        {
            Server = $Server
            Credential = $Credential
            Motd = $motd
            MotdClear = $MotdClear
            Issue = $Issue
            IssueClear = $IssueClear
        }
    }
}

vCenterSettings_Config -ConfigurationData $script:configurationData

And then the environmental part.

$motdConfigInput = @{
    server = 'vcsa.local.lab'
    user = 'administrator@vsphere.local'
    password = 'VMware1!'
    motd = 'Hello VCSA Stranger!'
    issue = 'Welcome to my VCSA world!'
}

. .\vCenterSettings_Config.ps1 @motdConfigInput

$sConfig = @{
    ComputerName = 'localhost'
    Path = '.\vCenterSettings_Config'
    Verbose = $true
    Wait = $true
    Force = $true
}
Start-DscConfiguration @sConfig

And the result after applying the configuration.

Pester Tests

Unit Testing

As specified in the Coding Guidelines, you have to provide Pester Unit test scripts for the resources you add to the module. Unit tests test how the code flows through your methods.

From my own experience, all this Pester code looks daunting and complex at first, but at some point you ‘make the click‘, and from then on it all looks obvious and not too hard at all.

Let’s take a small extract from the Pester Unit tests I wrote for the VMHostSettings class. There is some structure and logic in such tests.

  Describe 'VMHostSettings\Set' {
    AfterEach {
      $script:resourceProperties.Motd = [string]::Empty
      $script:resourceProperties.Issue = [string]::Empty
    }

    Context 'Invoking with empty settings' {
      BeforeAll {
        # Arrange
        $viServer = [VMware.Vim.VIServer] @{ Name = '10.23.82.112'; User = 'user' }
        $vmhost = [VMware.Vim.VMHost] @{ Id = 'VMHostId' }

        $viServerMock = {
          return [VMware.Vim.VIServer] @{ Name = '10.23.82.112'; User = 'user' }
        }
        $vmHostMock = {
          return [VMware.Vim.VMHost] @{ Id = 'VMHostId' }
        }

        Mock -CommandName Connect-VIServer -MockWith $viServerMock -ModuleName $script:moduleName
        Mock -CommandName Get-VMHost -MockWith $vmHostMock -ModuleName $script:moduleName
        Mock -CommandName Get-AdvancedSetting -MockWith { return $null } -ModuleName $script:moduleName
      }

      # Arrange
      $resource = New-Object -TypeName $script:resourceName -Property $script:resourceProperties

      It 'Should call the Connect-VIServer mock with the passed server and credentials once' {
        # Act
        $resource.Set()

        # Assert
        Assert-MockCalled -CommandName Connect-VIServer 
          -ParameterFilter { $Server -eq $script:resourceProperties.Server -and $Credential -eq $script:resourceProperties.Credential } 
          -ModuleName $script:moduleName -Exactly 1 -Scope It
      }
   }
}

• Your Unit test has to cover the Set, Test and Get functionality of your resource. Each of these is covered in one Describe block.
• As not to suffer from relics from test 1 while doing test 2, you can reset your environment. This is defined in a AfterEach block
• We group our tests in Context blocks. Such a Context block generally consists of a number of tests that start from the same situation. That could for example be, all tests that run with no parameters passed. The text that you specify on such a Context block, will appear on the Pester output.
• Since you don’t want to run these Unit tests against a live environment, you are going to mock the cmdlets used in your methods.
• In a mock definition you specify which cmdlet you are mocking and what this mock shall return to your tests. Most of the time this is minimal information, and just enough that the methods you are testing can proceed.
• These mock definitions are provided in a BeforeAll block, meaning that these mock definitions are done before any actual tests.
• Since we are testing a class, we have to create a minimal representation of such a class for the tests. This is done with the New-Object cmdlet, and the resulting object is stored in the variable $resource.
• Now we can define the actual tests. These are defined in It blocks. Add a meaningful description of the test, this is the text you will see appearing, hopefully in green, when the Unit test are run.
• Since we are in the Set testing part, remember the Describe from earlier on, we have to call the Set method on the class with $resource.Set()
• A series of tests will verify that all cmdlets that you expect to be called are called. This is defined with the Assert-MockCalled. There are multiple options on that command, including how many times you expect the (mocked) cmdlet to be called.

The DSCR for VMware provides a script to run all the Unit tests in one run, but sometimes you might have the need to run a specific Unit test. That can be done by just running that single test.

Go to Tests folder, and start the test.

Invoke-Pester -Path .\Unit\VMHostSettings.Unit.Tests.ps1

If all goes well, you should see something like this.

Integration Testing

Integration tests verify that your code actually does what it is supposed to do, by running it against an actual environment. There is no mocking involved, when your code is supposed to create a VM, the Integration test will verify that the VM is actually created.

All the Pester testing will eventually be incorporated in CI/CD pipeline. In that pipeline you will specify how the test environment for your Integration test can be reached. That is why you will notice that driving scripts for this testing requires (mandatory) some parameters like:

• Server: which vSphere Server
• Name: only for tests that run against a VMHost resource
• User/Password: the credentials to connect to Server

For an Integration test you will need at least two files:

• Pester script: this contains the tests
• Configuration file(s): this contains the DSC configurations you are testing with

Pester script

This Pester script contains the actual Integration tests that will be executed. The filename is <DSC-resource-name>.Integration.Tests.ps1.

In that file, under the Describe block, you have a number of Context blocks. Each Context block describes tests for one or more of the properties available in the DSC resource.

In the Context blocks, you have a number of It blocks. In these It blocks you perform that actual tests with the DSC resource.

A minimal set of It blocks for a Context are:

• Does the Configuration create a MOF and is it applied without any errors?
• Does applying the Configuration have the expected result?
• Does a Test with the Configuration show compliance (is the result $true)?

A sample Describe block looks like this

Describe "$($script:dscResourceName)_Integration" {
  Context "When using configuration $($script:config)" {
    BeforeAll {
      BeforeAllTests
    }

    AfterAll {
      AfterAllTests
    }

    BeforeEach {
      # Arrange
      $startDscConfigurationParameters = @{
        Path         = $script:mofFilePath
        ComputerName = 'localhost'
        Wait         = $true
        Force        = $true
      }

      # Act
      $script:dscConfig = Start-DscConfiguration @startDscConfigurationParameters
    }

    It 'Should compile and apply the MOF without throwing' {
      # Assert
      { $script:dscConfig } | Should -Not -Throw
    }

    It 'Should be able to call Get-DscConfiguration without throwing and all the parameters should match' {
      # Arrange && Act
      $script:dscConfig = Get-DscConfiguration 
        | Where-Object {$_.configurationName -eq $script:config }

      $configuration = $script:dscConfig 
        | Select-Object -Last 1

      # Assert
      { $script:dscConfig } | Should -Not -Throw

      $configuration.Name | Should -Be $script:resourceProperties.Name
      $configuration.Server | Should -Be $script:resourceProperties.Server
      $configuration.Motd | Should -Be $script:motd
      $configuration.Issue | Should -Be $script:issue
    }

    It 'Should return $true when Test-DscConfiguration is run' {
      # Arrange && Act && Assert
      Test-DscConfiguration | Should -Be $true
    }
  }
}

Configuration file

This Configuration file contains one or more DSC Configuration definitions that you are testing with the Pester script.

The filename follows this layout, <DSC-resource-name>_Config.ps1.

In the file there are one or more Configurations that allow you you test all aspects of your DSC resource. In the following example we test the motd and issue properties of the VMHostSettings resource.

$Password = $Password | ConvertTo-SecureString -AsPlainText -Force
$script:vmHostCredential = New-Object System.Management.Automation.PSCredential($User, $Password)

$script:motd = 'VMHostSettings motd test'
$script:issue = 'VMHostSettings issue test'

$script:configurationData = @{
  AllNodes = @(
    @{
      NodeName                    = 'localhost'
      PSDscAllowPlainTextPassword = $true
    }
  )
}

$moduleFolderPath = (Get-Module VMware.vSphereDSC -ListAvailable).ModuleBase
$integrationTestsFolderPath = Join-Path (Join-Path $moduleFolderPath 'Tests') 'Integration'

Configuration VMHostSettings_Config
{
  Import-DscResource -ModuleName VMware.vSphereDSC

  Node localhost
  {
    VMHostSettings vmHostSettings {
      Name       = $Name
      Server     = $Server
      Credential = $script:vmHostCredential
      Motd       = $script:motd
      Issue      = $script:issue
    }
  }
}

VMHostSettings_Config -OutputPath "$integrationTestsFolderPath\VMHostSettings_Config" -ConfigurationData $script:configurationData

Submit your Code

Once you have your new DSC resources ready, tested and Pester tested, you are ready to submit to the repository.

First, stage all the files that you changed on your station, to your repository. Make sure to clearly document what the changes are all about.

One requirement is that all your commits shall be signed. First you will need to set up global settings for your account and email.

git config --global user.name "Luc Dekens"
git config --global user.email "dekens.luc@gmail.com"

And verify the settings with

git config --list --show-origin

For each commit you make to your repository, you can now “sign” it with

git commit --amend --signoff

This will open an editor window with the commit content, including the signoff.

Once your repository has the changes and you made sure all your commits are signed, you are ready to create your Pull Request (PR) against the vmware/dscr-for-vmware repository. Make sure to follow the instructions in the CONTRIBUTING document.

Some important points:

• the base is the VMware repository
• open the PR against the dev branch
• make sure to select the branch with your latest changes on your side (here dev-lucd)

Once your PR is submitted, it will be reviewed by the PowerCLI Team. When they have questions, or want you make changes, you can follow that up in the comments of your PR.

When the review is completed, the reviewer will approve the merge of your code. And you will see your code being merged in the dev branch.

Epilogue

I intend this to be a living post, in other words any new information or any new experiences I gather, I will add them here.
And of course, if you have questions or remarks, feel free use the Comments or my Contact Form.

Enjoy!

There are numerous articles and blog posts that describe how set up a Photon 2.0 VM, but most of these use the ovftool or the vSphere Web Client to install it. Followed by quite of a bit of editing config files, to have your Photon 2.0 VM running the way you prefer it.
With the latest version of my Invoke-VMScriptPlus function, you can now automate this entire process, the SDDC way! 

Update August 21st 2018

• Added “reboot” action
• Added optional Environment variables for the customisation scripts
• Added Folder field for template destination

Intro

As I briefly mentioned in the introduction paragraph, setting up a VM with the Photon OS 2.0, is a two-step process.

• Deploy the appliance
• Configure the OS

The first step is rather straight-forward with PowerCLI’s Import-VApp cmdlet.

The second step, configuring the Photon OS, is presented as a manual step in most of the articles I could find. Till now!

With my Invoke-VMScripPlus function, this step can now also be automated. And that automation step is driven by a JSON file. This JSON file is the part that you place under version control.

The following scenario, which I’m currently using, creates a VM with a configured Photon 2.0 guest OS. Then converts that VM into a Template, which can then be used to roll out your VMs.

Create a Photon VM Template

This step is fully automated and driven by a JSON configuration file. In a matter of minutes you can create a new Photon 2.0 template, without even logging on to the machine!

The JSON Configuration file

The objective to have all configuration parameters in a flat text file is of course to be able to easily place it under version control and to easily compare between versions.

The JSON file contains a number of Level-1 entries, each specifying part of the final configuration of the Photon Template.

• vSphere: describes the Location for the template
• Template: characteristics of the Template
• Network: the network configuration inside the guest OS
• Account: the current and new password for the root account
• Proxy: the proxy settings, if present
• Docker: the docker configuration
• Code: a collection of customisation scripts

vSphere

• VMHost: a cluster or an ESXi node where the VM will be installed. If you specify a cluster, the script takes a random ESXi node in that cluster.
• Storage: a datastorecluster or datastore where the VM will be installed.
• OvaPath: where the script can find the Photon 2.0 ova file. Note the escaped back-slashes in the path.

Template

• TemplateName, TemplateNote: self explanatory
• Folder: a VM type folder where the template shall be stored
• NumCpu, MemoryGB, HarddiskGB: self explanatory
• GuestID: has to be other3xLinux64Guest for Photon 2.0
• CreateTemplate: when set to false, the script will not power off the VM, nor will it convert the VM into a Template. This can be used will fine-tuning the customisation scripts (see later).
• CustomisationCode: the names of the customisation scripts that will be executed once the VM is installed. See later in the Code section.

Network

Since most of the customization of the Photon guest OS was done when we created the Template, this step requires very little extra customization steps.

I assume these settings all speak for themselves. Nothing out of the ordinary, just you regular network settings. Perhaps one point to note, when you have multiple values for a field, separate them with a space.

Account

Again, quite straight-forward. The account, and the old and new password.

Proxy

When your environment into which you are installing the VM is located behind a proxy, you can specify the proxy details in this section. This is required to get the Photon repositories working (tdnf), and also the docker repositories,

Docker

Only one setting, the default port to access docker.

Code

Under the Code section, you can define a number of customisation scripts, that will be executed inside the Photon 2.0 guest OS. Each entry has a two fixed subfields.

• Name: the name for this particular customisation script. This is the name that is used in the Template section, in CustomisationCode field.
• Script: the actual customisation script. Due to restrictions in the JSON syntax, the script must be provided as an array of strings. Each line of the script is an element in this array
• Environment: an array with environment variables that will be passed to the environment where the customisation script is executed.

In the example above, the customisation script is a one-liner. Hence one string in the Code array. In this example, the Code contains the line to pull the docker container that contains VMware PowerCLI.

There are of course multi-line customisation scripts. In that case the array contains multiple elements. The following example is an excerpt of the docker customisation script. As you notice, the code also contains comment lines and empty lines.

Since we do not want to hard-code values in our customisation scripts, we use a PowerShell style notation to refer to variables defined in other sections in the JSON file.

In line 126 of excerpt above, we use the expression $($paramData.Docker.Port) to refer to the port number we defined in the Docker section. 

To avoid that you have to type in your customisation scripts in this specific JSON format, the following short script can help you convert your regular bash script to JSON notation.

function Convert-Script2Json
{
  [cmdletbinding()]
  param(
    [Parameter(Mandatory = $true)]
    [string]$ScriptName,
    [Parameter(Mandatory = $true)]
    [string[]]$Script
  )

  $obj = New-Object PSObject -Property @{
      Name = $ScriptName
      Script = $script.Split("n") | %{$_.TrimEnd("r")}
  }
  $obj | ConvertTo-Json
}

A sample run shows you how you could use the above code.

@'
# Update packages
# Separate to avoid script ending with VMware Tools update

tdnf update -y
'@

Convert-Script2Json -ScriptName packages -Script $code

Will result in the following output, which you can copy/paste into your JSON file.

The New-PhotonTemplate function

Before you can run the New-PhotonTemplate function, you will need two functions that are used in the function. These are

• Invoke-VMScriptPlus from my Invoke-VMScriptPlus v2 post.
• Set-VMKeystrokes, from William Lam’s Automating VM keystrokes using the vSphere API & PowerCLI 

You have multiple options to make these two functions available to the New-PhotonTemplate function.

• Add them to the .ps1 file where you copied the New-PhotonTemplate function
• Store the two functions in separate .ps1 files, and dot-source these .ps1 files before calling the New-PhotonTemplate function
• Create a module, and add the two functions to that module

The Code

function New-PhotonTemplate {
<#
.SYNOPSIS
 Create a Template with the Photon 2.0 guest OS
.DESCRIPTION
 This function will create a new Template with the Photon 2.0
 guest OS. The parameters and the configuration scripts are passed via
 a JSON file.
.NOTES
 Author:  Luc Dekens
 Version:
 1.0 11/08/18  Initial release
 1.1 16/08/18  Create function
               Added support for Reboot action
               Added support for environment variable
 1.2 17/08/18  Added Folder (in JSON file) support
              
.PARAMETER JSONPath
 The location of the JSON file with the configuration parameters
.PARAMETER LogFile
 Optional file for capturing logging information.
 This includes the output of the customisation scripts
.EXAMPLE
 New-PhotonTemplate -JSONPath .\photon.json
.EXAMPLE
 New-PhotonTemplate -JSONPath .\photon.json -LogFile .\photon.log
#>

  [cmdletbinding()]
  param(
    [Parameter(Mandatory = $true)]
    [string]$JSONPath,
    [string]$LogFile
  )

  function Write-ToLog {
    [CmdletBinding()]
    param(
      [string]$Category,
      [string]$Text
    )

    $timeStamp = (Get-Date).ToString('yyyyMMddThhmmss')
    $script = Split-Path -Path $myInvocation.ScriptName -Leaf
    $user = $env:UserName

    $line = "$timeStamp - $script - $user - [$Category] $Text"
    if ($LogFile) {
      $line | Out-File -FilePath $LogFile -Append
    }
    else {
      $line
    }
  }

  function Invoke-VMReboot {
    [CmdletBinding()]
    param(
      [string]$VMName
    )

    $vmGuest = Get-VMGuest -VM $VMName

    Stop-VMGuest -VM $vm -Confirm:$false | Out-Null
    while ($vmGuest.VM.PowerState -ne 'PoweredOff') {
      Start-Sleep 1
      $vmGuest = Get-VMGuest -VM $vmGuest.VM
    }

    $vmGuest = Start-VM -VM $vmGuest.VM -Confirm:$false | Get-VMGuest
    while ($vmGuest.VM.PowerState -ne 'PoweredOn' -or -not $vmGuest.ExtensionData.GuestOperationsReady) {
      Start-Sleep 1
      $vmGuest = Get-VMGuest -VM $vmGuest.VM
    }
  }

  # Get Parameters

  $paramDataJSON = Get-Content -Path $JSONPath | Out-String
  $paramData = ConvertFrom-Json -InputObject $paramDataJSON

  # Determine location for template

  $obj = Get-Inventory -Name $paramData.vSphere.VMHost
  if ($obj -is [VMware.VimAutomation.ViCore.Types.V1.Inventory.Cluster]) {
    $obj = Get-VMHost -Location $obj | Get-Random
  }
  $esx = $obj

  $dsc = Get-DatastoreCluster -Name $paramData.vSphere.Storage -ErrorAction SilentlyContinue
  if ($dsc) {
    $ds = Get-Datastore -RelatedObject $dsc | Get-Random
  }
  else {
    $ds = Get-Datastore -Name $paramData.vSphere.Storage
  }

  # Clean up eventual artifacts

  if ($vm = Get-VM -Name $paramData.Template.TemplateName -ErrorAction SilentlyContinue) {
    if ($vm.PowerState -eq [VMware.VimAutomation.ViCore.Types.V1.Inventory.PowerState]::PoweredOn) {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -Confirm:$false -DeletePermanently
    Write-ToLog -Category Info -Text "Removed VM $($vm.Name)"
  }
  if ($template = Get-Template -Name $paramData.Template.TemplateName -ErrorAction SilentlyContinue) {
    Remove-Template -Template $template -DeletePermanently -Confirm:$false
    Write-ToLog -Category Info -Text "Removed Template $($template.Name)"
  }

  # Import Appliance

  $ovfParm = Get-OvfConfiguration -Ovf $paramData.vSphere.OvaPath
  $ovfParm.NetworkMapping.None.Value = $paramData.Network.PortGroup
  $sVApp = @{
    Name              = $paramData.Template.TemplateName
    Source            = $paramData.vSphere.OvaPath
    OvfConfiguration  = $ovfParm
    VMHost            = $esx
    Datastore         = $ds
    DiskStorageFormat = [VMware.VimAutomation.ViCore.Types.V1.VirtualDevice.VirtualDiskStorageFormat]::Thin
    Confirm           = $false
  }
  if($paramData.Template.Folder){
    $sVApp.Add('InventoryLocation',(Get-FolderByPath -Path $paramData.Template.Folder))
  }
  Write-ToLog -Category Info -Text "Importing vApp..."
  $vm = Import-VApp @sVApp
  Write-ToLog -Category Info -Text "Imported vApp $($sVApp.Name)"

  # Configure VM

  $vm = Get-VM -Name $paramData.Template.TemplateName

  $sSet = @{
    VM       = $vm
    NumCpu   = $paramData.Template.NumCpu
    MemoryGB = $paramData.Template.MemoryGB
    Notes    = $paramData.Template.TemplateNote.Replace('#timestamp#', (Get-Date -Format 'dd/MM/yyyy HH:mm'))
    Confirm  = $false
  }
  Set-VM @sSet > $null
  Get-HardDisk -VM $vm | Set-HardDisk -CapacityGB $paramData.Template.HarddiskGB -Confirm:$false  > $null
  Get-FloppyDrive -VM $vm | Remove-FloppyDrive -Confirm:$false > $null
  Write-ToLog -Category Info -Text "Configured VM $($vm.Name)"

  Start-VM -VM $vm > $null
  Write-ToLog -Category Info -Text "Power on VM $($vm.Name)"

  while ($vm.ExtensionData.Guest.ToolsRunningStatus -ne [VMware.Vim.VirtualMachineToolsRunningStatus]::guestToolsRunning -or
    $vm.ExtensionData.Runtime.PowerState -ne [VMware.Vim.VirtualMachinePowerState]::poweredOn) {
    Start-Sleep 2
    $vm.ExtensionData.UpdateViewData()
  }
  Write-ToLog -Category Info -Text "VM $($vm.Name) is powered on"

  # Change Photon OS default root password
  # Thanks to William Lam for his Set-VMKeystrokes function
  # See https://www.virtuallyghetto.com/2017/09/automating-vm-keystrokes-using-the-vsphere-api-powercli.html

  Write-ToLog -Category Info -Text "Change password for user $($paramData.Account.User)"

  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.User -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.OldPassword -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.OldPassword -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.NewPassword -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.NewPassword -ReturnCarriage $true | Out-Null

  Set-VMKeystrokes -VMName $VM -StringInput "exit" -ReturnCarriage $true | Out-Null

  $vm = Get-VM -Name $paramData.Template.TemplateName

  # Run all customisation scripts

  foreach ($code in $paramData.Template.CustomisationCode) {
    if ($code -eq 'reboot') {
      Write-ToLog -Category Info -Text "Rebooting VM $($vm.Name)..."
      Invoke-VMReboot -VMName $vm.Name
      Write-ToLog -Category Info -Text "Rebooted VM $($vm.Name)"
    }
    else {
      $codeObj = $paramData.Code.Where{$_.Name -eq $code}
      $sInvoke = @{
        VM               = $vm
        ScriptType       = 'Bash'
        ScriptText       = $ExecutionContext.InvokeCommand.ExpandString(($codeObj.Script -join "`r`n"))
        GuestUser        = $paramData.Account.User
        GuestPassword    = ConvertTo-SecureString -String $paramData.Account.NewPassword -AsPlainText -Force
        GuestOSType      = 'Linux'
      }
      if($codeObj.Environment){
        $sInvoke.Add('ScriptEnvironment',$codeObj.Environment.foreach{$ExecutionContext.InvokeCommand.ExpandString($_)})
      }
      Write-ToLog -Category Info -Text "Running $code"
      $result = Invoke-VMScriptPlus @sInvoke
      if ($LogFile) {
        $result | Out-File -FilePath $LogFile -Append
      }
      else {
        $result
      }
    }
  }

  # Create Template

  if ($paramData.Template.CreateTemplate) {
    $vm = Get-VM -Name $paramData.Template.TemplateName

    if ($vm.PowerState -eq [VMware.VimAutomation.ViCore.Types.V1.Inventory.PowerState]::PoweredOn) {
      Write-ToLog -Category Info -Text "Stopping VM $($vm.Name) ..."
      Shutdown-VMGuest -VM $vm -Confirm:$false > $null
      while ($vm.PowerState -ne [VMware.VimAutomation.ViCore.Types.V1.Inventory.PowerState]::PoweredOff) {
        Start-Sleep 3
        $vm = Get-VM -Name $paramData.Template.TemplateName
      }
      Write-ToLog -Category Info -Text "VM $($vm.Name) stopped"
    }

    Write-ToLog -Category Info -Text "Convert VM $($vm.Name) to Template"
    Set-vm -VM $vm -ToTemplate -Confirm:$false > $null
  }
}

Annotations

Line 36-54: An inline function to have all messages in the same format.

Line 56-75: A reboot the guest OS function. Called when the “reboot” instruction is encountered. See later.

Line 79-80: These lines read the JSON file and convert the data to a PowerShell object

Line 84-88: The vSphere.VMHost field can contain a cluster or an ESXi node. These lines handle those options. In case of a cluster, the function selects a random ESXi node from the cluster.

Line 90-96: The vSphere.Storage field can contain a DatastoreCluster or a Datastore. These lines handle those options. In case of a datastorecluster, the function selects a random datastore from the datastorecluster.

Line 100-110: These lines check if there is already a Template or a VirtualMachine with a name as specified in Template.TemplateName. If there is, the Template or VirtualMachine are removed.

Line 114-130: A straight-forward method to import the Photon OVA as defined in vSphere.OvaPath.

Line 134-146:  Further HW configuration of the VM

Line 148-156: The new VM is powered on, and the function waits till the VMware Tools are responding

Line 162-170: The function changes the root password with the new password defined in Account.NewPassword. For this action the code uses the Set-VMKeystrokes function. William describes this ingenious method in his Automating VM keystrokes using the vSphere API & PowerCLI post, well worth a read!

Line 176-204: The code now runs all the customisation script that are defined in Template.CustomisationCode. The actual customisation scripts are defined in the Code section of the JSON file.

Line 177-182: With reserved name “reboot” you can request a reboot of the guest OS. This is executed in the sequence order with the customisation scripts as defined in the Template.CustomisationCode field.

Line 187,193: With the ExpandString method the script expands all $paramData occurrances in the customisation scripts with the actual values. This allows us to keep the customisation script general and not coded for a specific instance.

Line 192-194: If the customisation script contains environment variables in the Code.Environment field, these are environment variables are passed along with the customisatoin script to the Invoke-VMScriptPlus function.

Line 208-223: Based on the Template.CreateTemplate value (true or false) the VM is converted into a Template. One reason not to convert the VM to a Template, could be that your debugging one or more of the customisation scripts.

Sample Run

Now we have everything in place to create a Photon 2.0 template, based on the  configuration settings and customisation scripts in our JSON file. Since it can be useful to review the output, especially of the customisation scripts, we use the LogFile parameter to specify a file where the function can write the output.

New-PhotonTemplate -JSONPath .\LabParams.json -LogFile C:\Temp\photon-template.log

When we call the New-PhotonTemplate with the LogFile parameter, there will be no regular output on the PowerShell console.

The logfile contains useful information, especially if we add statements to our customisation scripts to retrieve information on the components. This excerpt shows some information that was obtained in the docker customisation script.

When all goes well, the New-PhotonTemplate will have created a Template with the Photon 2.0 guest OS.

Known Issues

open-vm-tools

When you use one of the customisation scripts to upgrade the Photon components (command tdnf upgrade), and the open-vm-tools are in there, the returned information of the script might not be complete. You will miss the ScriptOutput and the ExitCode.

As a workaround, upgrade the open-vm-tools separately before upgrading any of the other components.

Files

Attached to this post are some files that will help you get started. There is an empty, skeleton JSON file and two Photon customisation scripts, one for the network and one for docker.

Next

In Part 2 of this series I will be showing how the Template we just created, can be used to roll out any number of VMs.

Enjoy!

]]> https://www.lucd.info/2018/08/14/deploy-photon-2-0-part-1/feed/ 5 https://www.lucd.info/2018/08/05/invoke-vmscriptplus-v2/ https://www.lucd.info/2018/08/05/invoke-vmscriptplus-v2/#comments Sun, 05 Aug 2018 11:16:26 +0000 http://www.lucd.info/?p=5804 The ability to execute scripts inside the guest OS of your VMs, is definitely one of the more useful cmdlets available in VMware PowerCLI. A year ago I published the first version of my Invoke-VMScriptPlus function to solve some of the issues the Invoke-VMScript cmdlet has in my opinion.
That function allowed you to run multi-line scripts in a Linux guest OS on your VMs. It also allowed you to use she-bang lines, to indicate which interpreter your script had to run in (bash, perl, python, nodejs, php…). Another handy feature was that you could use Linux here-documents in your scripts.

With the introduction of PowerShell Core (aka PowerShell v6), the lack of support for any Guest OS of the Windows family became obvious. Prompted by a recent thread in the VMTN PowerCLI Community, I decided it was time to publish a new version of my Invoke-VMScriptPlus function.

Update August 21st 2018

• Added ScriptEnvironment

Intro

I defined some target features  for the new version of the Invoke-VMScriptPlus function:

• Support the Windows family of Guest OS
• Support using PowerShell Core scripts (Windows and Linux)
• Fix the ScriptText length limitation

The Code

class MyOBN:System.Management.Automation.ArgumentTransformationAttribute {
  [ValidateSet(
    'Cluster', 'Datacenter', 'Datastore', 'DatastoreCluster', 'Folder',
    'VirtualMachine', 'VirtualSwitch', 'VMHost', 'VIServer'
  )]
  [String]$Type
  MyOBN([string]$Type) {
    $this.Type = $Type
  }
  [object] Transform([System.Management.Automation.EngineIntrinsics]$engineIntrinsics, [object]$inputData) {
    if ($inputData -is [string]) {
      if (-NOT [string]::IsNullOrWhiteSpace( $inputData )) {
        $cmdParam = "-$(if($this.Type -eq 'VIServer'){'Server'}else{'Name'}) $($inputData)"
        $sCmd = @{
          Command = "Get-$($this.Type.Replace('VirtualMachine','VM')) $($cmdParam)"
        }
        return (Invoke-Expression @sCmd)
      }
    }
    elseif ($inputData.GetType().Name -match "$($this.Type)Impl") {
      return $inputData
    }
    elseif ($inputData.GetType().Name -eq 'Object[]') {
      return ($inputData | ForEach-Object {
          if ($_ -is [String]) {
            return (Invoke-Expression -Command "Get-$($this.Type.Replace('VirtualMachine','VM')) -Name $_")
          }
          elseif ($_.GetType().Name -match "$($this.Type)Impl") {
            $_
          }
        })
    }
    throw [System.IO.FileNotFoundException]::New()
  }
}
function Invoke-VMScriptPlus {
  <#
.SYNOPSIS
  Runs a script in a Linux guest OS.
  The script can use the SheBang to indicate which interpreter to use.
  .DESCRIPTION
  This function will launch a script in a Linux guest OS.
  The script supports the SheBang line for a limited set of interpreters.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 14/09/17  Initial release
  1.1 14/10/17  Support bash here-document
  2.0 01/08/18  Support Windows guest OS, bat & powershell
  2.1 03/08/18  PowerShell she-bang for Linux
  2.2 17/08/18  Added ScriptEnvironment
.PARAMETER VM
  Specifies the virtual machines on whose guest operating systems
  you want to run the script.
.PARAMETER GuestUser
  Specifies the user name you want to use for authenticating with the
  virtual machine guest OS.
.PARAMETER GuestPassword
  Specifies the password you want to use for authenticating with the
  virtual machine guest OS.
.PARAMETER GuestCredential
  Specifies a PSCredential object containing the credentials you want
  to use for authenticating with the virtual machine guest OS.
.PARAMETER ScriptText
  Provides the text of the script you want to run. You can also pass
  to this parameter a string variable containing the path to the script.
  Note that the function will add a SheBang line, based on the ScriptType,
  if none is provided in the script text.
.PARAMETER ScriptType
  The supported Linux interpreters.
  Currently these are bash,perl,python3,nodejs,php,lua
.PARAMETER ScriptEnvironment
  A string array with environment variables.
  These environment variables are available to the script from ScriptText
.PARAMETER GuestOSType
  Indicates which type of guest OS the VM is using.
  The parameter accepts Windows or Linux. This parameter is a fallback for
  when the function cannot determine which OS Family the Guest OS
  belongs to
.PARAMETER PSv6Version
  Indicates which PowerShell Core version to use.
  The default is 6.0.2
.PARAMETER CRLF
  Switch to indicate of the NL that is returned by Linux, shall be
  converted to a CRLF
.PARAMETER Server
  Specifies the vCenter Server systems on which you want to run the
  cmdlet. If no value is passed to this parameter, the command runs
  on the default servers. For more information about default servers,
  see the description of Connect-VIServer.
.EXAMPLE
  $pScript = @'
  #!/usr/bin/env perl
  use strict;
  use warnings;
  print "Hello world\n";
  '@
  $sCode = @{
  VM = $VM
  GuestCredential = $cred
  ScriptType = 'perl'
  ScriptText = $pScript
  }
  Invoke-VMScriptPlus @sCode
.EXAMPLE
  $pScript = @'
  print("Happy 10th Birthday PowerCLI!")
  '@
  $sCode = @{
  VM = $VM
  GuestCredential = $cred
  ScriptType = 'python3'
  ScriptText = $pScript
  }
  Invoke-VMScriptPlus @sCode
  #>
  [cmdletbinding()]
  param(
    [parameter(Mandatory = $true, ValueFromPipeline = $true)]
    [MyOBN('VirtualMachine')]
    [VMware.VimAutomation.ViCore.Types.V1.Inventory.VirtualMachine[]]$VM,
    [Parameter(Mandatory = $true, ParameterSetName = 'PlainText')]
    [String]$GuestUser,
    [Parameter(Mandatory = $true, ParameterSetName = 'PlainText')]
    [SecureString]$GuestPassword,
    [Parameter(Mandatory = $true, ParameterSetName = 'PSCredential')]
    [PSCredential[]]$GuestCredential,
    [Parameter(Mandatory = $true)]
    [String]$ScriptText,
    [Parameter(Mandatory = $true)]
    [ValidateSet('bash', 'perl', 'python3', 'nodejs', 'php', 'lua', 'powershell',
      'powershellv6', 'bat')]
    [String]$ScriptType,
    [String[]]$ScriptEnvironment,
    [ValidateSet('Windows', 'Linux')]
    [String]$GuestOSType,
    [String]$PSv6Version = '6.0.2',
    [Switch]$CRLF,
    [MyOBN('VIServer')]
    [VMware.VimAutomation.ViCore.Types.V1.VIServer]$Server = $global:DefaultVIServer
  )
  Begin {
    $si = Get-View ServiceInstance
    $guestMgr = Get-View -Id $si.Content.GuestOperationsManager
    $gFileMgr = Get-View -Id $guestMgr.FileManager
    $gProcMgr = Get-View -Id $guestMgr.ProcessManager
    $shebangTab = @{
      'bash'         = '#!/usr/bin/env bash'
      'perl'         = '#!/usr/bin/env perl'
      'python3'      = '#!/usr/bin/env python3'
      'nodejs'       = '#!/usr/bin/env nodejs'
      'php'          = '#!/usr/bin/env php'
      'lua'          = '#!/usr/bin/env lua'
      'powershellv6' = '#!/usr/bin/env pwsh'
    }
  }
  Process {
    foreach ($vmInstance in $VM) {
      # Preamble
      if ($vmInstance.PowerState -ne 'PoweredOn') {
        Write-Error "VM $($vmInstance.Name) is not powered on"
        continue
      }
      if ($vmInstance.ExtensionData.Guest.ToolsRunningStatus -ne 'guestToolsRunning') {
        Write-Error "VMware Tools are not running on VM $($vmInstance.Name)"
        continue
      }
      $moref = $vmInstance.ExtensionData.MoRef
      # Are we targetting a Windows or a Linux box?
      if (-not $GuestOSType) {
        switch -Regex ($vmInstance.Guest.OSFullName) {
          'Windows' {
            $GuestOSType = 'Windows'
            if ('bat', 'powershell', 'powershellv6' -notcontains $ScriptType) {
              Write-Error "For a Windows guest OS the ScriptType can be Bat, PowerShell or PowerShellv6"
              continue
            }
          }
          'Linux' {
            $GuestOSType = 'Linux'
            if ('bat', 'powershell' -contains $ScriptType) {
              Write-Error "For a Linux guest OS the ScriptType cannot be Bat or PowerShell"
              continue
            }
          }
          Default {
            Write-Error "Unable to determine the guest OS type on VM $($vmInstance.Name)"
            continue
          }
        }
      }
      if ($GuestOSType -eq 'Linux') {
        Write-Verbose "Seems to be a Linux guest OS"
        # Test if code contains a SheBang, otherwise add it
        $targetCode = $shebangTab[$ScriptType]
        if ($ScriptText -notmatch "^$($targetCode)") {
          Write-Verbose "Add SheBang $targetCode"
          $ScriptText = "$($targetCode)nr$($ScriptText)"
        }
      }
      # Create Authentication Object (User + Password)
      if ($PSCmdlet.ParameterSetName -eq 'PSCredential') {
        $GuestUser = $GuestCredential.GetNetworkCredential().username
        $plainGuestPassword = $GuestCredential.GetNetworkCredential().password
      }
      if ($PSCmdlet.ParameterSetName -eq 'PlainText') {
        $bStr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($GuestPassword)
        $plainGuestPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($bStr)
      }
      $auth = New-Object VMware.Vim.NamePasswordAuthentication
      $auth.InteractiveSession = $false
      $auth.Username = $GuestUser
      $auth.Password = $plainGuestPassword
      # Copy script to temp file in guest
      # Create temp file for script
      $suffix = ''
      if ($ScriptType -eq 'bat') {
        $suffix = ".cmd"
      }
      if ('powershell', 'powershellv6' -contains $ScriptType) {
        $suffix = ".ps1"
      }
      Try {
        $tempFile = $gFileMgr.CreateTemporaryFileInGuest($moref, $auth, "$($env:USERNAME)_$($PID)", $suffix, $null)
          Write-Verbose "Created temp script file in guest OS $($tempFile.Name)"
      }
      Catch {
        Throw "$error[0].Exception.Message"
      }
      # Create temp file for output
      Try {
        $tempOutput = $gFileMgr.CreateTemporaryFileInGuest($moref, $auth, "$($env:USERNAME)_$($PID)_output", $null, $null)
          Write-Verbose "Created temp output file in guest OS $($tempOutput.Name)"
      }
      Catch {
        Throw "$error[0].Exception.Message"
      }
      # Copy script to temp file
      if ($GuestOSType -eq 'Linux') {
        $ScriptText = $ScriptText.Split("r") -join ''
      }
      $attr = New-Object VMware.Vim.GuestFileAttributes
      $clobber = $true
      $filePath = $gFileMgr.InitiateFileTransferToGuest($moref, $auth, $tempFile, $attr, $ScriptText.Length, $clobber)
      $ip = $filePath.split('/')[2].Split(':')[0]
      $hostName = Resolve-DnsName -Name $ip | Select-Object -ExpandProperty NameHost
      $filePath = $filePath.replace($ip, $hostName)
      $copyResult = Invoke-WebRequest -Uri $filePath -Method Put -Body $ScriptText
      if ($copyResult.StatusCode -ne 200) {
        Throw "ScripText copy failed!rStatus $($copyResult.StatusCode)r$(($copyResult.Content | ForEach-Object{[char]$_}) -join '')"
      }
      Write-Verbose "Copied scipttext to temp script file"

      # Get current environment variables

      $SystemEnvironment = $gProcMgr.ReadEnvironmentVariableInGuest($moref, $auth, $null)

      # Run script

      switch ($GuestOSType) {
        'Linux' {
          # Make temp file executable
          $spec = New-Object VMware.Vim.GuestProgramSpec
          $spec.Arguments = "751 $tempFile"
          $spec.ProgramPath = '/bin/chmod'
          Try {
            $procId = $gProcMgr.StartProgramInGuest($moref, $auth, $spec)
            Write-Verbose "Run script file"
          }
          Catch {
            Throw "$error[0].Exception.Message"
          }
          # Run temp file
          $spec = New-Object VMware.Vim.GuestProgramSpec
          if($ScriptEnvironment){
            $spec.EnvVariables = $SystemEnvironment + $ScriptEnvironment
          }
          $spec.Arguments = " > $($tempOutput)"
          $spec.ProgramPath = "$($tempFile)"
          Try {
            $procId = $gProcMgr.StartProgramInGuest($moref, $auth, $spec)
            Write-Verbose "Run script with '$($tempFile) > $($tempOutput)'"
          }
          Catch {
            Throw "$error[0].Exception.Message"
          }
        }
        'Windows' {
          # Run temp file
          $spec = New-Object VMware.Vim.GuestProgramSpec
          if($ScriptEnvironment){
            $spec.EnvVariables = $SystemEnvironment + $ScriptEnvironment
          }
          switch ($ScriptType) {
            'PowerShell' {
              $spec.Arguments = " /C powershell -NonInteractive -File $($tempFile) > $($tempOutput)"
              $spec.ProgramPath = "cmd.exe"
            }
            'PowerShellv6' {
              $spec.Arguments = " /C ""C:\Program Files\PowerShell\$($PSv6Version)\pwsh.exe"" -NonInteractive -File $($tempFile) > $($tempOutput)"
              $spec.ProgramPath = "cmd.exe"
            }
            'Bat' {
              $spec.Arguments = " /s /c cmd > $($tempOutput) 2>&1 /s /c $($tempFile)"
              $spec.ProgramPath = "cmd.exe"
            }
          }
          Try {
            $procId = $gProcMgr.StartProgramInGuest($moref, $auth, $spec)
            Write-Verbose "Run script with '$($spec.ProgramPath) $($spec.Arguments)'"
          }
          Catch {
            Throw "$error[0].Exception.Message"
          }
        }
      }
      # Wait for script to finish
      Try {
        $pInfo = $gProcMgr.ListProcessesInGuest($moref, $auth, @($procId))
        Write-Verbose "Wait for process to end"
        while ($pInfo -and $null -eq $pInfo.EndTime) {
          Start-Sleep 1
          $pInfo = $gProcMgr.ListProcessesInGuest($moref, $auth, @($procId))
        }
      }
      Catch {
        Throw "$error[0].Exception.Message"
      }
      # Retrieve output from script
      $fileInfo = $gFileMgr.InitiateFileTransferFromGuest($moref, $auth, $tempOutput)
      $fileContent = Invoke-WebRequest -Uri $fileInfo.Url -Method Get
      if ($fileContent.StatusCode -ne 200) {
        Throw "Retrieve of script output failed!rStatus $($fileContent.Status)r$(($fileContent.Content | ForEach-Object{[char]$_}) -join '')"
      }
      Write-Verbose "Get output from $($fileInfo.Url)"
      # Clean up
      # Remove output file
      $gFileMgr.DeleteFileInGuest($moref, $auth, $tempOutput)
      Write-Verbose "Removed file $($tempOutput.Name)"
      # Remove temp script file
      $gFileMgr.DeleteFileInGuest($moref, $auth, $tempFile)
      Write-Verbose "Removed file $($tempFile.Name)"
      New-Object PSObject -Property @{
        VM           = $vmInstance
        ScriptOutput = & {
          $out = ($fileContent.Content | ForEach-Object {[char]$_}) -join ''
          if ($CRLF) {
            $out.Replace("n", "n`r")
          }
          else {
            $out
          }
        }
        Pid          = $procId
        PidOwner     = $pInfo.Owner
        Start        = $pInfo.StartTime
        Finish       = $pInfo.EndTime
        ExitCode     = $pInfo.ExitCode
        ScriptType   = $ScriptType
        ScriptSize   = $ScriptText.Length
        ScriptText   = $ScriptText
        GuestOS      = $GuestOSType
      }
    }
  }
}

Annotations

Line 1-35: The latest version of my OBN (Object By Name) class. It allows one to pass, or the actual .Net object, or the name of the object, as an argument to a parameter. See also Home Made OBN

Line 136: When the Invoke-VMScriptPlus  function cannot determine the Guest OS family, this parameter allows you to force a specific Guest OS family. Accepted values are Windows and Linux.

Line 133: The installation path of PowerShell Core on a Windows OS contains the version number. I could not find an easy way to determine the version number. This parameter allows to specify a specific PowerShell Core version. The default version is 6.0.2, and that is because that is the highest version VMware PowerCLI currently supports. Note that for the time being running VMware PowerCLI does not officially support running in PowerShell Core on a Windows Guest OS. Always consult the latest Compatibility Matrix to find out what is officially supported by VMware PowerCLI.

Line 134: This parameter allows you to pass a number of environment variables as an array of strings

Line 138: Most Linux OS return output with only a LF. This switch can be used to convert the LF to a CRLF, when the resulting output of the script is returned.

Line 147-155: A hard-coded table with the supported interpreters, and their corresponding SheBang line. This version of the Invoke-VMScriptPlus  function adds PowerShell Core to the table.

Line 160-167: If the VM is not powered on, or if the VMware Tools are not running, or if the Guest OS family can not be determined, the function will return with an error message.

Line 170-191: The Invoke-VMScriptPlus  function tests, based on the Guest OS family, if the requested ScriptType is valid.

Line 192-199: For a VM with a Guest OS in the Linux family, tests if there is a SheBang line in the ScriptText. If not, add a line based on the ScriptType value.

Line 202-213: Create the Authentication object for the GuestOperations methods. Note that in this release of the function, the GuestPassword requires a SecureString instead of a String.

Line 216-222: Depending on the ScriptType, the temporary file, see the following annotations, will have a filetype.

Line 223-237: The Invoke-VMScriptPlus  function uses two temporary files to store the script and the script’s output.

Line 239-241: If the target Guest OS family is Linux, the line endings in the ScriptText are converted from CRLF to LF

Line 242-252: The ScriptText is copied to the temporary file. This is done over HTTPS with the Invoke-WebRequest cmdlet. Note that the temporary filename returned earlier contains the IP address of the ESXi node on which the VM is running. This potentially causes problems with the Invoke-WebRequest and invalid certificates. In the URI for the file, the function replaces the IP address with FQDN.

Line 256: Fetch the current system environment variables

Line 263-272: On a Guest OS in the Linux family, the file containing the ScriptText needs to be made “executable”

Line 276: The function merges the system environment variables with the provided environment variables. The resulting array is passed to the StartProgramInGuest call.

Line 278-286: On a Guest OS in the Windows family, the path to the PowerShell Core EXE, contains the version number. This is where the $PSv6Version parameter comes into play. The default value is currently set to 6.0.2.

Line 280-286,308-314: Script execution is started

Line 318-328: Wait for the script execution to end

Line 330-335: The output is fetched, again with an Invoke-WebRequest.

Line 338-341: Clean up the temporary files

Line 343-364: Return an object containing the script output and further info about the script execution

Sample Use

The following snippet lists the content of the os-release file, a common method in Linux distros to obtain the name and version of the OS. We use the following simple bash script.

Note that in this new version of the Invoke-VMScriptPlus function you have to use a SecureString (instead of a String in the previous version) for the GuestPassword parameter. This is a breaking change, but was done to comply with the ScriptAnalyzer rules!

$user = 'luc'
$pswd = 'SuperSecret1!'
$sPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force

$code =@'
if [ -f /etc/os2-release ]
then
        cat /etc/os-release
else
        echo 'File os-release not found'
fi
'@

$sInvP = @{
   VM = 'opensuse1'
   ScriptType = 'Bash'
   ScriptText = $code
   GuestOSType = 'Linux'
   GuestUser = $user
   GuestPassword = $sPswd
}
Invoke-VMScriptPlus @sInvP

For an openSuse guest OS the result comes back as

For an Ubuntu guest OS

And for a CentOS guest OS as

One of the target goals of the Invoke-VMScriptPlus v2 was to be able to use the function to runs scripts in a Windows guest OS. The following sample uses the wmic command.

$user = 'local\administrator'
$pswd = 'SuperSecret1!'
$sPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force

$code = @'
wmic os get version
'@

$sInvP = @{
   VM = 'Server1'
   ScriptType = 'bat'
   ScriptText = $code
   GuestUser = $user
   GuestPassword = $sPswd
}
Invoke-VMScriptPlus @sInvP

The result comes back as

And of course PowerShell scripts

$user = 'local\administrator'
$pswd = 'SuperSecret1!'
$sPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential -ArgumentList $user,$sPswd

$code = @'
(Get-CimInstance Win32_OperatingSystem).version
'@


$sInvP = @{
   VM = 'vEng'
   ScriptType = 'PowerShell'
   ScriptText = $code
   GuestCredential = $cred
}
Invoke-VMScriptPlus @sInvP

Which returns

Note how this last invocation uses the GuestCredential parameter instead of GuestUser and GuestPassword. The Invoke-VMScriptPlus function supports both methods of passing the guest OS credentials.

This last sample is run under the “normal” PowerShell installed in the Windows guest OS, as we can easily see with

$user = 'local\administrator'
$pswd = 'SuperSecret1!'
$sPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential -ArgumentList $user,$sPswd

$code = @'
$PSVersionTable
'@

$sInvP = @{
   VM = 'vEng'
   ScriptType = 'PowerShell'
   ScriptText = $code
   GuestCredential = $cred
}
Invoke-VMScriptPlus @sInvP

The output of that script

Another new feature of the Invoke-VMScriptPlus function is the ability to call PowerShell Core.

On Windows

$user = 'local\administrator'
$pswd = 'SuperSecret1!'
$sPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential -ArgumentList $user,$sPswd

$code = @'
$PSVersionTable
'@

$sInvP = @{
   VM = 'vEng'
   ScriptType = 'PowerShellv6'
   ScriptText = $code
   GuestCredential = $cred
}
Invoke-VMScriptPlus @sInvP

which gives.
As you might have noted, we have both PowerShell version installed side-by-side on that station.

And also on Linux

$user = 'luc'
$pswd = 'SuperSecret1!'
$sPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force

$code =@'
$PSVersionTable
'@

$sInvP = @{
   VM = 'opensuse1'
   ScriptType = 'PowerShellv6'
   ScriptText = $code
   GuestOSType = 'Linux'
   GuestUser = $user
   GuestPassword = $sPswd
}
Invoke-VMScriptPlus @sInvP

Which results in

The final new feature I introduced in Invoke-VMScriptPlus v2 is the length limitation that exists in the regular Invoke-VMScript cmdlet. That limitation blocks you when you want to transfer scripts that surpass a specific length. Depending on the circumstances that limitation pops up in scripts that have a length somewhere in the neighbourhood of 2600 characters.

A new feature that was introduced is the ScriptEnvironment parameter. This allows the caller to pass environment variables. A simple example:

$vm = Get-VM ubuntu1

$sInvoke = @{
    VM            = $vm
    ScriptType    = 'Bash'
    ScriptText    = 'printenv'
    GuestUser     = 'luc'
    GuestPassword = ConvertTo-SecureString -String 'SuperSecret1!' -AsPlainText -Force
    GuestOSType   = 'Linux'
    ScriptEnvironment = 'Msg1=Hello','Msg2=World'
}

Invoke-VMScriptPlus @sInvoke

And we see our environment variables added to the execution environment.

If you encounter “issues” or have suggestions for additions and improvements, please let me know.

Enjoy!

Is this something you should know about? Yes, you should!

In a two-part article on TechGenix, I wrote down my thoughts and observations on REST API. The article goes into what this move towards the REST API might mean for you as a scripter/administrator.

Since an article on coding, without a coding example doesn’t make much sense, I added a module, named rCisTag on the PowerCLI Examples repository.

Articles

Enjoy reading Part 1, Understanding the VMware REST API interface!

and Part 2, SOAP vs REST for performing tasks in VMware environments.

When you are interacting with a Windows based Guest OS you can run old-fashioned BAT files or use PowerShell scripts. When the Guest OS is Linux based, you currently only can run Bash scripts.

Most Linux flavours have a feature that is called SheBang, and which allows you to specify in the first line of your bash script, which interpreter shall be used to run the following lines of the script. Unfortunately, the current Invoke-VMScript cmdlet doesn’t allow one to use that feature.

Time to tackle that issue, and expand the possibilities for all VMs that have a Linux-based Guest OS. So I decided to write my Invoke-VMScriptPlus function.

Update October 14th 2017

• Added here-document bash sample

When the PowerCLI Feature Request website was announced during VMworld session #SER2529BU, it was no surprise to me, to rather quickly see a request appearing to support other languages, besides bash.

First I did some tests, to check if I could get the Invoke-VMScript cmdlet to work with a SheBang line.

$vmName = 'ubuntu1'
$vm = Get-VM -Name $vmName

$codeBashPlain = @"
echo "Hello World!"
"@

$codeBashSheBang = @"
#!/usr/bin/env bash
echo "Hello World!"
"@

$sScript = @{
    VM = $vm
    GuestUser = 'lucd'
    GuestPassword = 'Just@Password1!'
    ScriptType = 'Bash'
    ScriptText = $codeBashPlain
}

Invoke-VMScript @sScript

$sScript['ScriptText'] = $codeBashSheBang
Invoke-VMScript @sScript

Unfortunately it doesn’t.

When a cmdlet doesn’t do what you are trying to achieve, there is always the vSphere API that can help. The GuestOperationsManager is the place to look. From there we can access methods to start and monitor a process in the guest  and to handle files inside the guest OS.
The StartProgramInGuest is the central method of the Invoke-VMScriptPlus function. The function uses the arguments property on the GuestProgramSpec object to redirect the stdio of the process.

The following flow-chart shows a high-level view of the logic that is used in the Invoke-VMScriptPlus function, and shows which method is used at which point.

The Code

#requires -Version 5.0
#requires -Modules VMware.VimAutomation.Core

class MyOBN:System.Management.Automation.ArgumentTransformationAttribute
{
    [ValidateSet(
        'Cluster','Datacenter','Datastore','DatastoreCluster','Folder',
        'VirtualMachine','VirtualSwitch','VMHost','VIServer'
    )]
    [String]$Type

    MyOBN([string]$Type)
    {
        $this.Type = $Type
    }
    [object] Transform([System.Management.Automation.EngineIntrinsics]$engineIntrinsics,[object]$inputData)
    {
        if ($inputData -is [string])
        {
            if (-NOT [string]::IsNullOrWhiteSpace( $inputData ))
            {
                $cmdParam = "-$(if($this.Type -eq 'VIServer'){'Server'}else{'Name'}) $($inputData)"
                $sCmd = @{
                    Command = "Get-$($this.Type.Replace('VirtualMachine','VM')) $($cmdParam)"
                }
                return (Invoke-Expression @sCmd)
            }
        }
        elseif($inputData.GetType().Name -match "$($this.Type)Impl")
        {
            return $inputData
        }
        elseif($inputData.GetType().Name -eq 'Object[]')
        {
            return ($inputData | %{
                if($_ -is [String])
                {
                    return (Invoke-Expression -Command "Get-$($this.Type.Replace('VirtualMachine','VM')) -Name `$_")
                }
                elseif($_.GetType().Name -match "$($this.Type)Impl")
                {
                    $_
                }
            })
        }
        throw [System.IO.FileNotFoundException]::New()
    }
}

function Invoke-VMScriptPlus
{
<#
.SYNOPSIS
  Runs a script in a Linux guest OS.
  The script can use the SheBang to indicate which interpreter to use.
.DESCRIPTION
  This function will launch a script in a Linux guest OS.
  The script supports the SheBang line for a limited set of interpreters.
.NOTES
  Author:  Luc Dekens
.PARAMETER VM
  Specifies the virtual machines on whose guest operating systems
  you want to run the script.
.PARAMETER GuestUser
  Specifies the user name you want to use for authenticating with the
  virtual machine guest OS.
.PARAMETER GuestPassword
  Specifies the password you want to use for authenticating with the
  virtual machine guest OS.
.PARAMETER GuestCredential
  Specifies a PSCredential object containing the credentials you want
  to use for authenticating with the virtual machine guest OS.
.PARAMETER ScriptText
  Provides the text of the script you want to run. You can also pass
  to this parameter a string variable containing the path to the script.
  Note that the function will add a SheBang line, based on the ScriptType,
  if none is provided in the script text.
.PARAMETER ScriptType
  The supported Linux interpreters.
  Currently these are bash,perl,python3,nodejs,php,lua
.PARAMETER CRLF
  Switch to indicate of the NL that is returned by Linux, shall be
  converted to a CRLF
.PARAMETER Server
  Specifies the vCenter Server systems on which you want to run the
  cmdlet. If no value is passed to this parameter, the command runs
  on the default servers. For more information about default servers,
  see the description of Connect-VIServer.  
.EXAMPLE
  $pScript = @'
  #!/usr/bin/env perl
  use strict;
  use warnings;
 
  print "Hello world\n";
  '@
    $sCode = @{
      VM = $VM
      GuestCredential = $cred
      ScriptType = 'perl'
      ScriptText = $pScript
  }
  Invoke-VMScriptPlus @sCode
.EXAMPLE
  $pScript = @'
  print("Happy 10th Birthday PowerCLI!") 
  '@
    $sCode = @{
      VM = $VM
      GuestCredential = $cred
      ScriptType = 'python3'
      ScriptText = $pScript
  }
  Invoke-VMScriptPlus @sCode
#>    
    [cmdletbinding()]    
    param(
        [parameter(Mandatory=$true,ValueFromPipeline=$true)]
        [MyOBN('VirtualMachine')]
        [VMware.VimAutomation.ViCore.Types.V1.Inventory.VirtualMachine[]]$VM,
        [Parameter(Mandatory=$true,ParameterSetName='PlainText')]
        [String]$GuestUser,
        [Parameter(Mandatory=$true,ParameterSetName='PlainText')]
        [String]$GuestPassword,
        [Parameter(Mandatory=$true,ParameterSetName='PSCredential')]
        [PSCredential[]]$GuestCredential,
        [Parameter(Mandatory=$true)]
        [String]$ScriptText,
        [Parameter(Mandatory=$true)]
        [ValidateSet('bash','perl','python3','nodejs','php','lua')]
        [String]$ScriptType,
        [Switch]$CRLF,
        [MyOBN('VIServer')]
        [VMware.VimAutomation.ViCore.Types.V1.VIServer]$Server = $global:DefaultVIServer

    )

    Begin
    {
        $si = Get-View ServiceInstance
        $guestMgr = Get-View -Id $si.Content.GuestOperationsManager
        $gFileMgr = Get-View -Id $guestMgr.FileManager
        $gProcMgr = Get-View -Id $guestMgr.ProcessManager

        $shebangTab = @{
            'bash' = '#!/usr/bin/env bash'
            'perl' = '#!/usr/bin/env perl'
            'python3' = '#!/usr/bin/env python3'
            'nodejs' = '#!/usr/bin/env nodejs'
            'php' = '#!/usr/bin/env php'
            'lua' = '#!/usr/bin/env lua'
        }
    }

    Process
    {
        foreach($vmInstance in $VM){
            # Preamble
            if($vmInstance.PowerState -ne 'PoweredOn')
            {
                Write-Error "VM $($vmInstance.Name) is not powered on"
                continue
            }
            if($vmInstance.ExtensionData.Guest.ToolsRunningStatus -ne 'guestToolsRunning')
            {
                Write-Error "VMware Tools are not running on VM $($vmInstance.Name)"
                continue
            }

            $moref = $vmInstance.ExtensionData.MoRef

            # Test if code contains a SheBang, otherwise add it
            $targetCode = $shebangTab[$ScriptType]
            if($ScriptText -notmatch "^$($targetCode)"){
                $ScriptText = "$($targetCode)`n`r$($ScriptText)"
            }
    
            # Create Authentication Object (User + Password)
            
            if($PSCmdlet.ParameterSetName -eq 'PSCredential')
            {
                $GuestUser = $GuestCredential.GetNetworkCredential().username
                $GuestPassword = $GuestCredential.GetNetworkCredential().password
            }
    
            $auth = New-Object VMware.Vim.NamePasswordAuthentication
            $auth.InteractiveSession = $false
            $auth.Username = $GuestUser
            $auth.Password = $GuestPassword
            
            # Copy script to temp file in guest
            
            # Create temp file for script
            Try{
                $tempFile = $gFileMgr.CreateTemporaryFileInGuest($moref,$auth,"$($env:USERNAME)_","_$($PID)",'/tmp')
            }
            Catch{
                Throw "$error[0].Exception.Message"
            }
            
            # Create temp file for output
            Try{
                $tempOutput = $gFileMgr.CreateTemporaryFileInGuest($moref,$auth,"$($env:USERNAME)_","_$($PID)_output",'/tmp')
            }
            Catch{
                Throw "$error[0].Exception.Message"
            }
           
            # Copy script to temp file
            $lCode = $ScriptText.Split("`r") -join ''
            $attr = New-Object VMware.Vim.GuestFileAttributes
            $clobber = $true
            $filePath = $gFileMgr.InitiateFileTransferToGuest($moref,$auth,$tempFile,$attr,$lCode.Length,$clobber)
            $copyResult = Invoke-WebRequest -Uri $filePath -Method Put -Body $lCode
            
            if($copyResult.StatusCode -ne 200)
            {
                Throw "ScripText copy failed!`rStatus $($copyResult.StatusCode)`r$(($copyResult.Content | %{[char]$_}) -join '')"
            }
                
            # Make temp file executable
            $spec = New-Object VMware.Vim.GuestProgramSpec
            $spec.Arguments = "751 $($tempFile.Split('/')[-1])"
            $spec.ProgramPath = '/bin/chmod'
            $spec.WorkingDirectory = '/tmp'
            Try{
                $procId = $gProcMgr.StartProgramInGuest($moref,$auth,$spec)
            }
            Catch{
                Throw "$error[0].Exception.Message"
            }
            
            # Run temp file
            
            $spec = New-Object VMware.Vim.GuestProgramSpec
            $spec.Arguments = " > $($tempOutput)"
            $spec.ProgramPath = "$($tempFile)"
            $spec.WorkingDirectory = '/tmp'
            Try{
                $procId = $gProcMgr.StartProgramInGuest($moref,$auth,$spec)
            }
            Catch{
                Throw "$error[0].Exception.Message"
            }
            
            # Wait for script to finish
            Try{
                $pInfo = $gProcMgr.ListProcessesInGuest($moref,$auth,@($procId))
                while($pInfo.EndTime -eq $null){
                    sleep 1
                    $pInfo = $gProcMgr.ListProcessesInGuest($moref,$auth,@($procId))
                }
            }
            Catch{
                Throw "$error[0].Exception.Message"
            }

            # Retrieve output from script
            
            $fileInfo = $gFileMgr.InitiateFileTransferFromGuest($moref,$auth,$tempOutput)
            $fileContent = Invoke-WebRequest -Uri $fileInfo.Url -Method Get
            if($fileContent.StatusCode -ne 200)
            {
                Throw "Retrieve of script output failed!`rStatus $($fileContent.Status)`r$(($fileContent.Content | %{[char]$_}) -join '')"
            }
            
            # Clean up

            # Remove output file
            $gFileMgr.DeleteFileInGuest($moref,$auth,$tempOutput)
            
            # Remove temp script file
            $gFileMgr.DeleteFileInGuest($moref,$auth,$tempFile)
    
            New-Object PSObject -Property @{
                VM = $vmInstance
                ScriptOutput = &{
                    $out = ($fileContent.Content | %{[char]$_}) -join ''
                    if($CRLF)
                    {
                        $out.Replace("`n","`n`r")
                    }
                    else
                    {
                        $out
                    }
                }
                Pid = $procId
                PidOwner = $pInfo.Owner
                Start = $pInfo.StartTime
                Finish = $pInfo.EndTime
                ExitCode = $pInfo.ExitCode
                ScriptType = $ScriptType
                ScriptText = $ScriptText
            }
        }
    }
}

Annotations

Line 1: The function requires PowerShell v5 or higher

Line 2: The function requires the PowerCLI Core module

Line 4-48: The latest version of my OBN (Object By Name) class. It allows one to pass or the actual .Net object, or the name of the object, as an argument to a parameter. See also Home Made OBN

Line 132: Most Linux OS return output with only a LF. This switch can be used to convert the LF to a CRLF, when the resulting output of the script is returned.

Line 145-152: A hard-coded table with the supported interpreters, and their corresponding SheBang line.

Line 159-168: If the VM is not powered on, or if the VMware Tools are not running, the function will return with a result.

Line 173-176: Tests if there is a SheBang line in the ScriptText. If not, it will add a line based on the ScriptType value.

Line 193-207: The function uses two temporary files to store the script and the script’s output.

Line 210-219: The ScriptText is copied to the temporary file. This is done over HTTPS with the Invoke-WebRequest cmdlet.

Line 222-231: The file containing the ScriptText needs to be made “executable”

Line 235-256: Script execution is started, and the function waits till the process completes.

Line 236: The function uses the Arguments property to redirect stdio to the second temporary file

Line 260-265: The output is fetched, again with an Invoke-WebRequest.

Line 270-273: Clean up the temporary files

Line 275-295: Return an object containing the script output and further info about the script execution

Sample Use

The use of the function Invoke-VMScriptPlus is quite similar to the use of the original Invoke-VMScript.

Some examples.

Bash

$codeBash = @"
#!/usr/bin/env bash
echo "Hello World!"
"@

$sCode = @{
  VM = $vm
  GuestCredential = $cred
  ScriptTYpe = 'bash'
  ScriptText = $codeBash
}

Invoke-VMScriptPlus @sCode

And the result

Here document

One feature that is often used in bash, is the so-called here document.

Unfortunately the Invoke-VMScript cmdlet doesn’t seem to support that feature for bash scripts. Code like this …

$code = @"
rm /tmp/test.txt
cat > /tmp/test.txt << EOF
Line 1
Line 2
EOF
ls -l /tmp/test.txt
echo File content
echo ------------
cat /tmp/test.txt
echo ------------
"@

$sINvoke = @{
    VM = Get-VM -Name $vmName
    ScriptType = 'bash'
    ScriptText = $code
    GuestUser = $user
    GuestPassword = $pswd
}
Invoke-VMScript @sINvoke | select -ExpandProperty ScriptOutput

… produces an error like this.

But due to the way the Invoke-VMScriptPlus function transfers the script text to the guest, this feature is working without a glitch.

The same code as above, except that Invoke-VMScript is replaced by Invoke-VMScriptPlus.

$code = @"
rm /tmp/test.txt
cat > /tmp/test.txt << EOF
Line 1
Line 2
EOF
ls -l /tmp/test.txt
echo File content
echo ------------
cat /tmp/test.txt
echo ------------
"@

$sINvoke = @{
    VM = Get-VM -Name $vmName
    ScriptType = 'bash'
    ScriptText = $code
    GuestUser = $user
    GuestPassword = $pswd
}
Invoke-VMScriptPlus @sInvoke | select -ExpandProperty ScriptOutput

… now produces the expected result.

Python3

In this example we are not adding the SheBang line in the ScriptText, but through the ScriptType value, the function will add this line.

$codePython = @"
print("Happy 10th Birthday PowerCLI!")
"@

$sCode = @{
 VM = $vm
 GuestCredential = $cred
 ScriptTYpe = 'python3'
 ScriptText = $codePython
}

Invoke-VMScriptPlus @sCode

And the result.
Notice how the Invoke-VMScriptPlus function added the SheBang line.

Other

During the VMworld Breakout session, where this function was first demonstrated, we showed additional examples in the following video.

Since there are many Linux flavours out there, and since I obviously couldn’t test them all, I would appreciate it if you can send me feedback about which Linux flavours/versions work, and which don’t.

If there are requests for other languages, feel free to forward me your requests.

Enjoy!

OBN allows you to refer to a PowerCLI object by name, instead of passing an actual PowerCLI object. A classic example is “Get-VM -Datastore DS1“, where we are retrieving all VMs that live on a specific datastore. If one looks at the description of the Datastore parameter, it clearly states that a value of type StorageResource is expected, but we are able to provide the datastorename, a string. Under the cover, PowerCLI converts this string to the required PowerCLI object for the parameter.

When we are writing our own functions, it would be very handy to have the same functionality at our disposal. Define a parameter to be of the type of a PowerCLI object, but then be able to pass the name of the object, instead of the object itself.
The solution is here, with the MyOBN attribute. We now have the same functionality available, that was until now only available for PowerCLI cmdlets.

Update September 14th 2017

• Added support for VIServer
• Added support for arrays of objects
• Fixed an issue with the VirtualMachine object

Some History

To have a better understanding, I’ll show some example functions.

The first function, Invoke-Test1, uses a Datastore parameter, that accepts a PowerCLI Datastore object.

Function Invoke-Test1{
    param(
        [VMware.VimAutomation.ViCore.Types.V1.DatastoreManagement.Datastore]$Datastore
    )

    Process{
        $Datastore | Select Name,CapacityGB
    }
}

$dsName = 'DS1'
$ds = Get-Datastore -Name $dsName

Invoke-Test1 -Datastore $ds

If we execute this, we get the Name of the VMs that live on the datastore.

But if we try to use the name of the datastore, as we are used to doing with the Datastore parameter on the Get-VM cmdlet for example, we get an error.

To get our function to behave more like OBN on the PowerCLI cmdlets, we can define the parameter as a general PSObject object. And perform the type casting inside the function. Something like this.

Function Invoke-Test2{
    param(
        [PSObject]$Datastore
    )

    Process{
        if($Datastore -is [System.String]){
            $Datastore = Get-Datastore -Name $Datastore
        }

        Get-VM -Datastore $Datastore | select Name
    }
}

$dsName = 'DS1'
$ds = Get-Datastore -Name $dsName

Invoke-Test2 -Datastore $ds

Our function now works with a Datastore object

but also with a String.

Note that this way of simulating the OBN functionality is far from fool-proof. There is for example no strict type checking, we could pass a VirtualMachine object on the Datastore parameter, and our function would throw an error.

When I first read about Custom Attributes, in a blog post, named Powershell: Creating and using custom attributes, from Kevin Marquette, I immediately saw the potential to get one step closer to a proper OBN in my functions.

For now, I have implemented the Attribute as a class, which implies it’s use is limited to PowerShell v5 or higher.

The Code

#requires -Version 5.0

class MyOBN:System.Management.Automation.ArgumentTransformationAttribute
{
    [ValidateSet(
        'Cluster','Datacenter','Datastore','DatastoreCluster','Folder',
        'VirtualMachine','VirtualSwitch','VMHost','VIServer'
    )]
    [String]$Type

    MyOBN([string]$Type)
    {
        $this.Type = $Type
    }
    [object] Transform([System.Management.Automation.EngineIntrinsics]$engineIntrinsics,[object]$inputData)
    {
        if ($inputData -is [string])
        {
            if (-NOT [string]::IsNullOrWhiteSpace( $inputData ))
            {
                $cmdParam = "-$(if($this.Type -eq 'VIServer'){'Server'}else{'Name'}) $($inputData)"
                $sCmd = @{
                    Command = "Get-$($this.Type.Replace('VirtualMachine','VM')) $($cmdParam)"
                }
                return (Invoke-Expression @sCmd)
            }
        }
        elseif($inputData.GetType().Name -match "$($this.Type)Impl")
        {
            return $inputData
        }
        elseif($inputData.GetType().Name -eq 'Object[]')
        {
            return ($inputData | %{
                if($_ -is [String])
                {
                    return (Invoke-Expression -Command "Get-$($this.Type.Replace('VirtualMachine','VM')) -Name `$_")
                }
                elseif($_.GetType().Name -match "$($this.Type)Impl")
                {
                    $_
                }
            })
        }
        throw [System.IO.FileNotFoundException]::New()
    }
}

Annotations

Line 1: this code requires PowerShell v5 or up

Line 3: we base the MyOBN class on the ArgumentTransformationAttribute class. We inherit the Transform method from this class. Our override of the Transform method will contain the logic to simulate the OBN functionality.

Line 5-8: currently the Transform method in the MyOBN class supports these PowerCLI types.

Line 11: We need a way to tell the Transform method, which objecttype we want to use. That’s why we add a property Type to the MyOBN class.

Line 9-12: the class has a constructor which allows us to initiate the Type property.

Line 13-44: the Transform method override is where all the magic happens

Line 15: we need to detect if the parameter is a String, in other words if the parameter was passed as an “Object By Name”.

Line 21: with the help of the Type property we invoke the correct PowerCLI cmdlet to convert the String to a PowerCLI obejct

Line 26-29: if the value passed to the parameter was already a PowerCLI object, we just pass that object along

Line 30-42: These lines handle the request for an array of supported objects

Sample Runs

To use this new parameter attribute is quite straightforward. Make sure the MyOBN class is loaded (this can be through dot-sourcing, adding it to your profile or just including it in your .ps1 file).

Then add the MyOBN attribute like any other parameter attribute. Make sure to specify the “Type” for which you want MyOBN to work. Like this

function Invoke-Test3{
  [CmdletBinding()]
  param(
    [MyOBN('Datastore')]
    [VMware.VimAutomation.ViCore.Types.V1.DatastoreManagement.Datastore]$Datastore
  )

  Process{
    Get-VM -Datastore $Datastore
  }
}

We can now call our function with a Datastore object

but also with a String that contains the name of the Datastore.

And all that without adding any extra code in our function, besides the MyOBN attribute.

Enjoy!