The main reason to use Photon OS is that it is open-sourced, it has a small footprint and it is optimised for VMware vSphere.

Some Background

Does this mean that Photon OS is the ideal guest OS for our ‘cattle‘ stations where we want to run our PowerShell scripts?

Not really, the major flaw with Photon OS, imho, is that the available packages are limited and rather infrequently maintained.

In fact, until recently the available PowerShell package was an outdated, not supported anymore version. But the worst part of the deal, it didn’t even allow to perform an Install-Module. Which, again imho, is a basic concept in PowerShell. And which didn’t allow me to install, for example, VMware PowerCLI on a Photon station.

But then recently this happened!

The available PowerShell v6 TDNF package was upgraded to a currently supported version, and more importantly, the Import-Module cmdlet  worked! 

Allowing us to install VMware PowerCLI on a Photon instance.

Is this the end of all the PowerShell woes on Photon OS?

Not really! The PowerShell version in this TDNF package will also run out of support in a not so far future and a PowerShell TDNF package upgrade is (still) not automated and somewhat of a black art.

But for now, let’s be happy with what we have and ‘cloud-init’ our Photon ‘cattle’ station!

Issues and some Solutions

No UserData Property

While the Ubuntu cloud image, as we saw in Part 1 of the series, has the option to pass the user-data to cloud-init via an OVF property, this feature is unfortunately not available in the Photon OVA.

There are some solutions available.

• Roll your own OVA image
• Use the NoCloud option, which means creating an ISO containing the user-data and attaching this ISO to the instance before cloud-init runs.

Since I prefer to use the ‘standard’ OVA, I opt for creating such a seed ISO.

The Root Password

This is somewhat of a chicken-and-the-egg dilemma.

In the Photon OVA image, the root account is set to force a password change on first use. But unfortunately, this also disallows setting the root password via the user-data. See Issue#931 for more details.

Again, there are two options available.

• Roll your own OVA image
• Use a two-stage process, wherein the first stage we use William Lam‘s Set-VMKeystrokes function. Then in the second stage, we attach the seed ISO and start the instance. And thus kicking off the cloud-init process.

And again, since I prefer to use the ‘standard’ OVA, I opt for the Set-VMKeystrokes solution.

Install-CloudInitVM Revisited

function Install-CloudInitVM
{
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
  1.1 07/12/19  Added Photon deployment
                Added seed ISO option
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER CloudConfigISO
  The ISO seed file, containing the meta-data and user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.PARAMETER Photon
  Switch to indicate a Photon is deployed. This will add an
  additional step to reset the root password before starting
  the actual cloud-init based deployment
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\photon-hw13_uefi-3.0-9355405.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfigISO = '.\seed.iso'
    Credential = $cred
    Photon = $true
  }
  Install-CloudInitVM @sCloudInitVM
#>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [Parameter(ParameterSetName = 'YAML')]
    [string]$CloudConfig,
    [Parameter(ParameterSetName = 'SeedISO')]
    [string]$CloudConfigISO,
    [PSCredential[]]$Credential,
    [Switch]$Photon
  )

  $waitJob = (Get-Command -Name .\Wait-Job.ps1 ).ScriptBlock
  if ($PSCmdlet.ParameterSetName -eq 'YAML')
  {
    $userData = Get-Content -Path $CloudConfig -Raw
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"
  if ($Photon)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - This is a Photon deployment"
  }

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue -Verbose:$false
  if ($vm)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn')
    {
      Stop-VM -VM $vm -Confirm:$false -Verbose:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false -Verbose:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile -Verbose:$false
  if (-not $Photon)
  {
    $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
    $ovfProp.NetworkMapping.VM_Network.Value = $pgName
  }
  else
  {
    $ovfProp.NetworkMapping.None.Value = $pgName
  }
  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName -Verbose:$false
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName -Verbose:$false | Get-VMHost  -Verbose:$false | Get-Random
    OvfConfiguration = $ovfProp
    Verbose = $false
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync -Verbose:$false | Out-Null

  if ($Photon)
  {
    while (-not $vm.ExtensionData.Guest.GuestOperationsReady)
    {
      Start-Sleep 2
      $vm.ExtensionData.UpdateViewData('Guest')
    }

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Resetting password"

    $PswdIsReset = $false
    while (-not $PswdIsReset)
    {
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'root' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'changeme' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'changeme' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'Welcome2019!' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput 'Welcome2019!' -ReturnCarriage $true | Out-Null
      Start-Sleep 1
      Set-VMKeystrokes -VMName $VM.Name -StringInput "exit" -ReturnCarriage $true | Out-Null

      Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Testing new password"

      $sInvoke = @{
        VM = $vm
        ScriptType = 'bash'
        ScriptText = 'whoami'
        GuestUser = $Credential.GetNetworkCredential().UserName
        GuestPassword = $Credential.GetNetworkCredential().Password
        Verbose = $false
      }
      try
      {
        Invoke-VMScript @sInvoke -ErrorAction Stop | Out-Null
        $PswdIsReset = $true
      }
      catch [VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidGuestLogin]
      {
        Start-Sleep -Seconds 1
      }
      catch
      {
        Write-Error "Exception $($error[0].Exception.GetType().FullName)"
        return
      }
    }

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Shutdown VM"

    Shutdown-VMGuest -VM $vm -Confirm:$false -Verbose:$false | Out-Null
    while ($vm.ExtensionData.Runtime.PowerState -ne 'poweredOff')
    {
      Start-Sleep 2
      $vm.ExtensionData.UpdateViewData('Runtime.PowerState')
    }

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Attaching seed ISO"

    $isoName = Split-Path -Path $CloudConfigISO -Leaf
    $isoFile = "DS:\$($vm.ExtensionData.Summary.Config.VmPathName.Split(' ')[1].Split('/')[0])\$($isoName)"
    Get-PSDrive -Name DS -ErrorAction SilentlyContinue -Verbose:$false | Remove-PSDrive -Confirm:$false -Verbose:$false
    New-PSDrive -Location (Get-Datastore -VM $vm) -Name DS -PSProvider VimDatastore -Root '\' -Verbose:$false | Out-Null
    Get-Item -Path $CloudConfigISO |
    Copy-DatastoreItem -Destination $isoFile -Force -Confirm:$false -Verbose:$false
    $isoDSPath = $vm.ExtensionData.Summary.Config.VmPathName.Replace("$vmName.vmx", $isoName)
    Get-CDDrive -VM $vm -Verbose:$false | Set-CDDrive -IsoPath $isoDSPath -StartConnected:$true -Confirm:$false -Verbose:$false | Out-Null

    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
    Start-VM -VM $vm -Confirm:$false -Verbose:$false | Out-Null
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  if ($Photon)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Restart OS"
    $vmGuest = Stop-VMGuest -VM $vm -Confirm:$false -Verbose:$false
    while ($vmGuest.VM.ExtensionData.Runtime.PowerState -ne 'poweredOff')
    {
      Start-Sleep 2
      $vmGuest.VM.ExtensionData.UpdateViewData('Runtime.PowerState')
    }
    $vm = Start-VM -VM $vmName -Confirm:$false -Verbose:$false
    while ($vm.PowerState -ne 'PoweredOn')
    {
      Start-Sleep 2
      $vm = Get-VM -Name $VmName -Verbose:$false
    }
    while (-not $vm.ExtensionData.Guest.GuestOperationsReady)
    {
      Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for GuestOperations ready"
      Start-Sleep 2
      $vm.ExtensionData.UpdateViewData('Guest')
    }
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Eject CDROM"
    $sInvoke = @{
      VM = $vm
      ScriptType = 'bash'
      ScriptText = 'eject cdrom'
      GuestUser = $Credential.GetNetworkCredential().UserName
      GuestPassword = $Credential.GetNetworkCredential().Password
      Verbose = $false
    }
    Invoke-VMScript @sInvoke | Out-Null
    Get-CDDrive -VM $vm -Verbose:$false | Set-CDDrive -NoMedia -Confirm:$false -Verbose:$false | Out-Null
  }

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"
  Write-Verbose "`nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

Annotations

The v1.0 version of this function has already been annotated in Part 1 of this series. Those annotations will not be repeated here.

Line 68-71: To distinguish between the way the user-data is passed, the function now has two mutually exclusive parameters, CloudConfig and CloudConfigIso.

Line 73: Since Photon requires some special actions, the function now has a switch, obviously named Photon, that specifies if the cloud-init deployment is intended for a Photon instance or not.

Line 77-80: When the user-data is provided as a YAML file, the function reads the file.

Line 83-86: A small reminder in the verbose output that the function will handle this as a Photon deployment.

Line 102-110: The Ubuntu and the Photon OVA files have different OVF properties. This If-Then-Else handles the differences.

Line 126-199: This part of the function handles most of the peculiarities of a Photon deployment.

Line 136-175: This part handles the reset of the root password. As mentioned earlier, it uses William’s Set-VMKeystrokes function. I experienced intermittent race conditions in sending the keystrokes. Hence the loop with the retries. As a test, to check if the root password was reset, the command ‘whoami‘ is executed trough Invoke-VMScript on the instance.

Line 179-184: Before attaching the seed ISO, the Photon OS is shutdown gracefully. In a While-loop the script test until the instance is actually powered off.

Line 188-195: The seed ISO is copied to the VM’s folder, and then attached.

Line 198: The VM is started with the seed ISO attached. As a result, cloud-init will now use the YAML files it finds on the CD drive as datasource.

Line 216-221: Since the cloud-init implementation on Photon is not configured to use the power-state module, the test we used on the Ubuntu instance, can not be used for Photon. As a bypass, the function stops/starts the guest OS on the instance.

Line 235-244: The function uses the stop/start sequence to disconnect the seed ISO. To avoid having a question on the VM, the function first runs the ‘eject cdrom‘ command inside the instance.

The Seed ISO

As mentioned earlier, I prefer to use a seed ISO file to solve the issue of not having an OVF property for the user-data.

To create this seed ISO file, we need a Linux station that has the ‘genisoimage‘ command available. In the following sample script, I use an Ubuntu station just for that purpose.

$workVM = 'UbuntuPS'
$isoName = 'genisoimage-ps.iso'
$isoVolId = 'cidata'
$isoFiles = @{
    'meta-data' = 'meta-data.yaml'
    'user-data' = 'user-data.yaml'
}

$code1 = @'
sudo apt-get install genisoimage -y > null
genisoimage -output $isoName -volid $isoVolId -joliet -rock $($isoFiles.Keys -join ' ')
'@

$workCred = Get-VICredentialStoreItem -Host $workVM

$isoFiles.GetEnumerator() | ForEach-Object -Process {
    Copy-Item -Path $_.Value -Destination $_.Name
}

$sInvoke = @{
    VM = $workVM
    ScriptType = 'Bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($code1)
    GuestUser = $workCred.User
    GuestPassword = ConvertTo-SecureString -String $workCred.Password -AsPlainText -Force
    GuestOSType = 'Linux'
    Verbose = $true
    NoIPinCert = $true
    InFile = $isoFiles.Keys
    OutFile = $isoName
}
Invoke-VMScriptPlus @sInvoke

Remove-Item -Path @($isoFiles.Keys) -Confirm:$false
Move-Item -Path $isoName -Destination "..\$isoName" -Force -Confirm:$false

Annotations

Line 4-7, 16-18: The script converts (copies) the provided YAML files to standard named YAML files. This allows the script to be used with different versions of the data and only needs a change in the $isoFiles hash table.

Line 32: This script uses the latest version of my Invoke-VMScriptPlus function. The reason is twofold: the bash script is multi-line and the script used the InFile/OutFile parameters to copy the YAML files to the Linux box and the ISO file back to the calling station.

User-Data

The user-data that is used for a Photon based instance is, for now, rather simple and straightforward.

Note the installation of the tdnf PowerShell package (as mentioned earlier in this post) in the runcmd section.

#cloud-config
fqdn: PhotonPS.local.lab
timezone: Europe/Brussels
hostname: PhotonPS
write_files:
    - path: /etc/systemd/network/10-static.network
      permissions: 0644
      content: |
        [Match]
        Name=eth0

        [Network]
        Address=192.168.10.81/24
        Gateway=192.168.10.1
        DNS=192.168.10.2 192.168.10.3
        DHCP=no
        Domains=local.lab
        NTP=192.168.10.2 192.168.10.3
        LinkLocalAddressing=no
        LLDP=true
    - path: /etc/systemd/network/99-dhcp-en.network
      permissions: 0644
      content: |
        [Match]
        Name=e*

        [Network]
        DHCP=no
    - path: /etc/gnutls/default-priorities
      content: |
        SYSTEM=NONE:!VERS-SSL3.0:!VERS-TLS1.0:+VERS-TLS1.1:+VERS-TLS1.2:+AES-128-CBC:+RSA:+SHA1:+COMP-NULL
runcmd:
- tdnf distro-sync
- tdnf update -y
- tdnf install bindutils -y
- tdnf install powershell -y
- tdnf install gnuTLS -y
- touch /etc/gnutls/default-priorities
package_upgrade: true
package_reboot_if_required: true

Meta-data

If you plan on deploying multiple instance, just make sure that the instance-id has a unique value.

instance-id: iid-local01
local-hostname: photonps

Sample Run

For a Photon instance the function is called like this.

$vmName = 'PhotonPS'

# Get credential for logging on to the guest OS
$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = 'D:\Repository\VMware\Photon\photon-hw13_uefi-3.0-9355405.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfigISO = '..\genisoimage-ps.iso'
  Photon = $true
  Credential = $cred
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

The result of this, since we used the Verbose switch, looks like this.

Again, don’t give too much importance to the total execution time. This was a run in my lab environment, which has limited resources.

On a side-note, notice how the PowerCLI cmdlets used in the function all show this, imho, annoying ‘Finished execution‘ message.
I would love to be able to suppress these verbose messages from the PowerCLI cmdlets. That is why I launched PowerCLI Idea #225 “Get rid of the over-eager verbosity on PowerCLI cmdlets“.

Enjoy!

In this first part, we will introduce cloud-init and how you can use it from your PowerShell/PowerCLI scripts. Since the Ubuntu distribution is very popular, on-premises and in the cloud, this introduction will focus on that distro to demonstrate the concept. In the following parts, we will tackle Photon, containers and how to run your scripts on these stations.

Before diving into the technical stuff, first a reminder why we should treat our servers as cattle.

A list with some of the major arguments (at least for me).

• No maintenance required when using the latest distributions. Meaning no long-running updates and security patches before usage.
• No “less than 5% used” machines anymore in your environment.
• No risk of system tattooing from any previous usage.
• An easy way to test new versions of your scripts, the OS, PowerShell and PowerCLI.

Ingredients

To use cloud-init, the requirements, in the cases we will handle here, are minimal.

• An OVA image that is configured to use cloud-init
  • Several Linux distributions nowadays have such a ‘cloud’ image: Ubuntu, Photon…
• A datasource, which is the configuration data provided by the user to the cloud-init process, and which defines how the resulting station will be configured.
• A cloud, in it’s broadest sense, to run the VM. From the cloud-init documentation “… all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations.”
• A script to trigger and control it all.

Canonical, the company behind Ubuntu, maintains a document, named Cloud Instance Initialisation with cloud-init, that has a schematic that captures the cloud-init process perfectly.

If we annotate and update this schema for the usage we have in mind, it becomes like this.

The OVA File

Since our target cloud platform in this series is a vSphere environment, we will use OVA files as the source for the VM(s) we are going to deploy.

The Ubuntu OVA image that we will be using in this post is for Ubuntu Server 18.04 LTS, aka Bionic Beaver.

This OVA image allows using the OVFProperties as the datasource. Note that the user-data we pass in this way needs to be Base64 encoded.

The User-data

As mentioned in the previous section, in this Ubuntu image we will use the OVFProperties as the datasource for our user-data.

The content of the user-data is provided to the script as a YAML file. The syntax for such a file and the available is defined in the cloud-init Documentation.

The following is the sample YAML file we will use in the rest of this post.

#cloud-config
hostname: ubuntubionic
fqdn: ubuntubionic.local.lab
write_files:
- path: /etc/netplan/50-cloud-init.yaml
  content: |
    network:
     version: 2
     ethernets:
      ens192:
       addresses: [192.168.10.79/24]
       gateway4: 192.168.10.1
       dhcp6: false
       nameservers:
         addresses:
           - 192.168.10.2
           - 192.168.10.3
         search:
           - local.lab
       dhcp4: false
       optional: true
- path: /etc/sysctl.d/60-disable-ipv6.conf
  owner: root
  content: |
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv6.conf.default.disable_ipv6=1
runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt-get -y update
- add-apt-repository universe
- apt-get -y clean
- apt-get -y autoremove --purge
timezone: Europe/Brussels
system_info:
  default_user:
    name: default-user
    lock_passwd: false
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
disable_root: false
ssh_pwauth: yes
users:
  - default
  - name: luc
    gecos: LucD
    lock_passwd: false
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
chpasswd:
  list: |
    default-user:$6$aPp//e2ueP$ETEXcAhAyQuJ4qNCbqxmmSYGZbg2wFwpP/YITvoXdgxwZBnf32drePKi2OIn5fLqtH5pHO03yRdPXK3ToLG6b0
    luc:$6$kW1WwJ2K$M6415du1BZd.qt92SvR6X.RuyDhEZmgR4hz4NcKH9XHn2850Vc6zHpubXM6uUeqMUaJQ740ogROB74gfBEhn9.
    root:$6$Js9CVr06$br9qf0VxuBsdY7Vtg/0pk9jLlycYBDLVsvbKwLDleCK7dSDheOxWaFOWdjkiqSPRrWG./N8V5RgCVwugZGnTc1
  expire: false
package_upgrade: true
package_reboot_if_required: true
power_state:
  delay: now
  mode: reboot
  message: Rebooting the OS
  condition: if [ -e /var/run/reboot-required ]; then exit 0; else exit 1; fi

Annotations

Line 1: the user-data file always has to start with the line ‘# cloud-config’

Line 2-3: These lines follow the regular YAML syntax of key-value entries. The lines defines the hostname and the FQDN that the instance will get.

Line 4-26: The write-files section instructs cloud-init to create files, with the specified properties and content on the instance. In this example file the files are used to define a static IPv4 address and to disable IPv6. Which files need to be used is dependent on the OS that runs in the VM. Note that this example will use netplan for the network configuration.

Line 27-34: The runcmd section defines commands that will run on the instance during the first boot of the guest OS. In this example the commands are used to activate the network (it uses netplan), set some variables and set and update the application repositories.

Line 35: This line defines which timezone needs to be configured.

Line 36-40: An Ubuntu image comes configured with a default user. In the system_info section, the settings for this default user are specified.

Line 43-50: The users section allows to define additional users on the instance. Note that the default user needs to come first in this list.

Line 51-56: The chpasswd allows you to change and/or set the passwords for the users. The required format and how these entries are created will be discussed in one of the following sections.

Line 57-58: These lines request an upgrade of the packages present on the instance. And perform a reboot when one of the package upgrades would require that.

Line 59-63: The power-state section instructs cloud-init in which state the instance shall be left after cloud-init completes. In this example, the system is rebooted on condition that a specific file exists. This file is created by cloud-init when it has decided that a reboot of the instance is required.

Password encoding

In user-data, there are several places where a password can be provided. The value needs to be the hash of the password, not the password itself. With the help of an existing Linux box (UbuntuWork in this case), I use the following script to create such SHA-512 encoded password hashes. The script uses the mkpasswd command to generate the hash.

Automation rules!

function Get-PasswordHash
{
    param(
        [String]$VMName,
        [String]$GuestUser,
        [String]$GuestPassword,
        [String]$Password
    )

    $sInvoke = @{
        VM = $VMName
        ScriptType = 'bash'
        ScriptText = "mkpasswd -m SHA-512 $Password"
        GuestUser = $GuestUser
        GuestPassword = $GuestPassword
    }
    (Invoke-VMScript @sInvoke).ScriptOutput.Trim("n")
}

$sHash = @{
    VM = 'UbuntuWork'
    GuestUser = 'root'
    GuestPassword = 'VMware1!'
    Password = 'VMware1!'
}
Get-PasswordHash @sHash

This returns something like this.

A word of warning, such hashed password are not super-safe. A decent password cracker with sufficient compute power and a good wordlist can most probably crack this in less than one minute.

The Script

function Install-CloudInitVM
{
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
  #>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [string]$CloudConfig,
    [PSCredential[]]$Credential
  )

  $waitJob = (Get-Command -Name .\Wait-Job.ps1).ScriptBlock
  $userData = Get-Content -Path $CloudConfig -Raw

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue
  if ($vm)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn')
    {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile
  $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
  $ovfProp.NetworkMapping.VM_Network.Value = $pgName

  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName | Get-VMHost | Get-Random
    OvfConfiguration = $ovfProp
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"

  Write-Verbose "nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

Annotations

Line 52: To find out when cloud-init has finished it's run, the function uses a background job. This background job is stored in a separate .ps1 file. More on the content and purpose of that 'wait' script later.

Line 53: The user-data is read from the YAML file. It is important to use the Raw switch, otherwise, we would lose the line separators.

Line 60-68: If a VM with the same Displayname is already present, it will be stopped (when powered on) and removed.

Line 70-71: The user-data is assigned to the Common.user_data property of the OVF properties. The user-data needs to be converted to base64.

Line 72: The Ubuntu OVA allows to pass the Portgroup to which the VM shall be connected.

Line 74-83: The VM is installed from the OVA with the Import-VApp cmdlet.

Line 86: The VM is powered on. During the boot process, the first phase of cloud-init will run.

Line 90-98: The function uses a background job to check if the cloud-init process has completed. More on that later.

The Wait job

I decided to store the WaitJob script in a separate .ps1 file. Primarily because I can easily reuse it this way and it makes the code of scripts that use shorter.

The Wait Job is started as a background job. This allows the calling code to simply use the Wait-Job cmdlet.

[CmdletBinding()]
param(
    [string]$vmName,
    [string]$user,
    [string]$pswd,
    [string]$vcsaName,
    [string]$SessionId
)

$sleepTime = 5
Connect-VIServer -Server $vcsaName -Session $SessionId | Out-Null
$notFinished = $true
while ($notFinished)
{
    Try
    {
        $vm = Get-VM -Name $vmName -ErrorAction Stop
        while ($vm.PowerState -ne 'PoweredOn' -and -not $vm.ExtensionData.Guest.GuestOperationsReady)
        {
            Start-Sleep -Seconds $sleepTime
        }
        $fileExist = $false
        while (-not $fileExist)
        {
            $sInvoke = @{
                VM = $vm
                ScriptType = 'bash'
                ScriptText = '[ -f /var/lib/cloud/instance/boot-finished ] && echo "File exist"'
                GuestUser = $user
                GuestPassword = $pswd
                ErrorAction = 'Stop'
            }
            try
            {
                $result = Invoke-VMScript @sInvoke
                $fileExist = [Boolean]$result.ScriptOutput
            }
            catch
            {
                Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') Exception:tId: $($error[0].Exception.ErrorId)  Category: $($error[0].Exception.ErrorCategory)"
                Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff')ttLine: $($error[0].InvocationInfo.Line)"
            }
        }
        $sInvoke = @{
            VM = $vm
            ScriptType = 'bash'
            ScriptText = 'cat /var/lib/cloud/instance/boot-finished'
            GuestUser = $user
            GuestPassword = $pswd
        }
        $result = Invoke-VMScript @sInvoke
        $result.ScriptOutput
        $notFinished = $false
    }
    catch
    {
        Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') Exception:tId: $($error[0].Exception.ErrorId)  Category: $($error[0].Exception.ErrorCategory)"
        Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff')ttLine: $($error[0].InvocationInfo.Line)"
    }
    Start-Sleep -Seconds $sleepTime
}

Annotations

Line 7 + 11: In a background job we do not inherit any open connections to a vSphere Server. A script can use the SessionId of an open connection in the calling script, to open a connection, without having to provide credentials.

Line 18: Before using the call to Invoke-VMScript, the Wait Job makes sure the VM and the VMware Tools in there, are ready to receive such a call.

Line 28: Inside the guest OS, the function uses the simple bash expression to test for the presence of a file. The file /var/lib/cloud/instance/boot-finished is created by cloud-init when the process completes.

Line 44-52: The Wait Job function returns the content of the file to the caller.

Sample Run

With the above code, we have everything in place to deploy a VM from an Ubuntu OVA, with the guest OS configuration is done through cloud-init, based on a YAML file.

A typical call could look something like this.

$vmName = 'UbuntuBionic'

# Get credential for logging on to the guest OS

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = '.\bionic-server-cloudimg-amd64.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfig = '.\user-data-bionic.yaml'
  Credential = $cred
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

With the Verbose switch set to $true, the output of this call would look like this.

I highlighted with red boxes the verbose messages from the function. The text in the green box is the content of the /var/lib/cloud/instance/boot-finished file.

Do not look too much at the timings. This run happened in my lab environment, which has limited resources. The main reason for showing this is to demonstrate how easy such a cloud-init based deployment can be incorporated in your pipelines.

This concludes Part 1 in the cloud-init series. It provides some functions and scripts to deploy a VM, starting from an OVA file, with the configuration of the guest OS done through cloud-init.

In the upcoming parts in this series, I will show some more advanced Ubuntu deployments, show how cloud-init can be used with Photon and show how you can use these deployed VMs as your 'cattle'.

Enjoy!