Introduction

In Post 4 of the cloud-init series, we ran our PowerShell/PowerCLI scripts natively in the Guest OS of the instance.

That might have an impact in some situations. When the first script installs for example the VMware PowerCLI modules, these modules will stay installed in the Guest OS. Consider this as a kind of PowerShell tattooing.

Another task that might be somewhat of a challenge is testing your PowerShell scripts against multiple environments, different module versions, different OS versions… Again, when we make changes to the Guest OS, these changes might be hard to remove.

Besides deploying a new ‘cattle’ station for each possible combination, we can also run our scripts in Docker containers. Once the script is done, we throw away the container instance. And we are back to a pristine OS.

And as it turns out, containers also offer some new features that are harder, or even impossible, to implement when running locally in the Guest OS.

Since we will be using several docker commands in the following sections, it will be useful to have the Docker Reference documentation nearby.

Install Docker

Ubuntu

The Ubuntu OVAs we have been installing in the previous posts in this series, do not contain the docker package. We can easily include that in our YAML file under the runcmd section.

#cloud-config
hostname: ubuntubioniccontainer
fqdn: ubuntubioniccontainer.local.lab
write_files:
- path: /etc/netplan/50-cloud-init.yaml
  content: |
    network:
     version: 2
     ethernets:
      ens192:
       addresses: [192.168.10.84/24]
       gateway4: 192.168.10.1
       dhcp6: false
       nameservers:
         addresses:
           - 192.168.10.2
           - 192.168.10.3
         search:
           - local.lab
       dhcp4: false
       optional: true
- path: /etc/sysctl.d/60-disable-ipv6.conf
  owner: root
  content: |
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv6.conf.default.disable_ipv6=1
runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt -y autoremove
- apt-get update
- apt-get remove docker docker-engine docker.io
- apt install docker.io -y
- systemctl enable docker
- systemctl start docker
timezone: Europe/Brussels
system_info:
  default_user:
    name: default-user
    lock_passwd: false
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
disable_root: false
ssh_pwauth: yes
users:
  - default
  - name: luc
    gecos: LucD
    lock_passwd: false
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
chpasswd:
  list: |
    default-user:$6$2csGx3AAX0RPRMPB$85sEEZ76z0V93M9pvXGtTFrwlMOaV2VjRHB6IKPRrwc.4qEqcs2EnDzn8oUlcVVb9Biq.Pg7qQchLgNLuetkT0
    luc:$6$.4Kyvju2v5iR$bvZHOntkH9EqgGdMc7fKZKbB132LYHVIFgxN7Ho.mlMGloUayOaKijR6MvLp/ncf4SynTigjaOKGk11EF9gru/
    root:$6$m1TXY60tUAViCYb1$/w.CqV50.M0Nlt6mx2726.bxWZ7/TCqWjOOw4S3h1H8AN.SPWQnKoASXth2E3euBXwebU4Q7dpoGU1SiQontT1
  expire: false
package_upgrade: true
package_reboot_if_required: true

Annotations

Line 33-36: Install the docker package, start it and make sure it runs after a reboot.

Line 33: If there should already be a docker package installed, this line removes that package. If no docker package is installed, the line does nothing.

Photon

The Photon OVA we use comes with docker installed. No need for any additional commands in the YAML file.

Image microsoft/powershell

In the following sections we will run a number of scripts in the microsoft/powershell container on Ubuntu and Photon. Only when there is a difference between both platforms, will the text mention the platforms.

A Simple Run

Let’s try running a simple PowerShell script in a container on the station.

$vmName = 'ubuntubioniccontainer'
$code = @'
sudo docker run --rm microsoft/powershell:latest pwsh -C 'whoami;Get-Process;pwd'
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $code
    NoIPinCert = $true
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

Annotations

Line 3: The docker command we sent to the station. The details of the different parts in this line are explained in the following table.

The output will be something like this.

Notice how docker will search for and install the image (since it isn’t present). That is the output in the yellow box. The actual output of the commands we sent to pwsh, are in the green box.

The second time we run the same script, we will only see the actual output of the commands. And that’s because the image is already present.

Modules

Time to start diving into PowerCLI. For that we need to install the VMware PowerCLI modules of course.

$vmName = 'ubuntubioniccontainer'

$script = @'
$ProgressPreference = "SilentlyContinue"
Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
Get-Module -Name VMware* -ListAvailable
'@

$docker = @'
sudo docker run --rm microsoft/powershell:latest pwsh -C '$($script.Replace("r",'').Split("n") -join ';')'
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($docker)
    NoIPinCert = $true
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

Annotations

Line 3-7: We use a here-string for the script we want to execute under pwsh in the container.

Line 9-11: And another here-string for the docker command.

Line 10: The script is running on a Windows box, hence the <CR><LF> to <LF>. The second manipulation is to convert the array of lines to a single line, where the lines are separated with a semi-column. Note that the actual conversion does not happen here.

Line 20: The script uses the ExpandString method to perform the text manipulations we described in the comments on Line 10. Also note that we don’t have to escape the dollar-signs in the $script part, since the variable substitution will not go that deep.

The result (don’t mind the funny characters for now).

ScreenSpace

When we use all the defaults in the container, we seem to end up with a simple screen that has the default 80 x 25 dimensions. For better results, we can introduce bigger screens inside the container.

$vmName = 'ubuntubioniccontainer'

$script = @'
reset -w
$ProgressPreference = "SilentlyContinue"
Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
Get-Module -Name VMware* -ListAvailable
'@
$docker = @'
sudo docker run --rm --tty \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -C '$($script.Replace("`r",'').Split("`n") -join ';')'
'@

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($docker)
    NoIPinCert = $true
    Verbose = $true
}
Invoke-VMScriptPlus @sInvoke

Annotations

Line 4: The reset -w command is a fix for an issue that Docker seems to have when a pseudo-terminal (the –tty parameter) is used. With the reset -w command, the terminal will reread the content of setupterm and adjust the lines and columns accordingly.

Line 10: The –tty parameter allocates a pseudo-terminal to the container.

Line 11: The -e parameter is used to set a number of environment variables for the container. In this case, specify that the container shall use an xterm with dimensions 25 x 135.

The changed screen is visible in the output (compare it with the width of the previous output).

Volumes

With the method we just showed, we will have to incur the overhead of installing the VMware PowerCLI modules on each run, because we throw away the container after we used it.

But there is a solution for this, by using docker volumes. They are kept inside the guest OS of the instance and can be attached to each container we run.

$vmName = 'ubuntubioniccontainer'

$script = @'
reset -w
\`$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name VMware.PowerCLI -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
}
\`$WarningPreference = 'SilentlyContinue'
\`$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = \`$false
    DisplayDeprecationWarnings = \`$false
    Scope = 'User'
    Confirm = \`$false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $vcsaUser -Password $vcsaPswd | Out-Null
Get-VMHost
Disconnect-VIServer -Server $vcsaName -Confirm:\`$false
'@

$docker = @'
sudo docker volume create PCLIModules > /dev/null
cat > ~/script.ps1 << EOF
$($script.Split("r"))
EOF
sudo docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v ~:/Scripts \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -F /Scripts/script.ps1
rm ~/script.ps1
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$vcsaUser = $credVCSA.User
$vcsaPswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

#$credItem = Get-VICredentialStoreItem -Host $vmName
#$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))
$cred = New-Object System.Management.Automation.PSCredential ($vmUser, (ConvertTo-SecureString $vmPswd -AsPlainText -Force))

$dockerExpanded = $ExecutionContext.InvokeCommand.ExpandString($docker)
$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($dockerExpanded)
    NoIPinCert = $true
}
Invoke-VMScriptPlus @sInvoke

Line 5: We have some 'Inception' going on here, hence the '' sequence before the dollar sign. In line 50 we substitute the variables in the bash script itself. That way we can provide the script as an inline file in the bash script. But since bash also interprets that back-tick, we have to escape it. The back-tick itself is to avoid variable substitution that is done in line 55.

Line 24: We create a docker volume named PCLIModules. If that volume already exists, the bash script will just continue.

Line 25-27: Instead of having the PowerShell/PowerCLI script in the Command parameter of the pwsh command, we use an inline file. This effectively creates a .ps1 file in the home directory of the user in the guest OS.

Line 30: The Volume parameter on the docker run command maps the docker volume PCLIModules to the folder /root/.local/share/powershell/Modules in the container. With an Install-Module, the modules will be installed in this /root/.local/share/powershell/Modules , and thus be stored on the docker volume PCLIModules. This volume can be reused on subsequent runs of the container, eliminating the need to install the VMware PowerCLI modules on each run.

Line 31: On the docker run command we map the user's home directory (~) in the guest OS to a folder, named Scripts in the container.

Line 34: The pwsh command that runs in the container uses the File parameter to run the .ps1 file created in lines 25-27.

Line 35: The .ps1 file that was created before calling the docker run command, is now removed.

Line 55: The second substitution round. In this case we substitute primarily all non-escaped (back-tick) dollar sign in the PowerShell script (at this point in the script present as an inline file in the bash script).

The result when we run the script.

Notice how the ScriptText property contains the bash script as it is presented to the Guest OS of the instance. In there the dollar signs are escaped (with a back-slash) to avoid substitution by bash.

ANSI codes and Existing Scripts

In this section, we will tackle two subjects.

• Eliminate those 'funny' characters in the output
• How to run my existing script, similar to what we did in Part 4 - Running Scripts.

As it turns out, these 'funny characters' are ANSI escape codes. In the container, we are using a pseudo-terminal, and as a result, the OS in the container sends those ANSI escape codes to position the cursor, change the text colour, but also those '...' (three dots) when PowerShell has to cut short the value of a property.

For now, the only way if found to eliminate those ANSI escape codes, is to find and replace them.

For using your existing scripts in this setup, we adopt the same strategy as we did in Part 4 - Running Scripts.

Our existing script, in Sample-PCLI-Script3.ps1, looks like this.

Get-VMHost |
    Select Name, NumCpu, Version |
    ConvertTo-Html

And the script to run all this looks like this

$vmName = 'ubuntubioniccontainer'

$prefix = @'
reset -w
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name VMware.PowerCLI -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $vcsaUser -Password $vcsaPswd | Out-Null
'@
$script = Get-Content -Path .\Sample-PCLI-Script3.ps1
$suffix = @'
Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$docker = @'
sudo docker volume create PCLIModules > /dev/null
cat > ~/script.ps1 << 'EOF'
$($prefix.Split("r"))
$($script -join "n")
$($suffix.Split("r"))
EOF
sudo docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v ~:/Scripts \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -F /Scripts/script.ps1 > ~/result.txt
cat ~/result.txt
rm ~/result.txt
rm ~/script.ps1
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$vcsaUser = $credVCSA.User
$vcsaPswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$dockerExpanded = $ExecutionContext.InvokeCommand.ExpandString($docker)
$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($dockerExpanded)
    NoIPinCert = $true
    Verbose = $true
}
$result = Invoke-VMScriptPlus @sInvoke

# Remove ANSI Escape Sequences
($result.ScriptOutput.Split("n") |
    Foreach-Object -Process {
        ($_ -replace "\xe2\x80\xa6","…") -replace "\x1b\[\?[1-9]h|\x1b\[[1-9]n|\x1b=|\x1b\]0;|\x07|\x1b\]0;", ''
    }) -join "n"

Annotations

Line 3, 20, 21: We split the code into three parts. A fixed prefix, the code of our existing script and a suffix.

Line 27-31: We use the same concept of an inline file as we did in the previous section. The only difference is that we now have three parts instead of one.

Line 64-67: The ANSI escape sequences are removed from the output. The RegEx expression does not cover all possible ANSI escape sequences, but it did the job for all the ones I encountered. On the other hand, the RegEx expression can easily be expanded.

The result, which is an HTML file, looks like this.

The part where we filter away the ANSI code sequence is also used to translate some well-known characters. Remember the funny characters in the module list?

With the first -replace operator in the previous code, we can change this back to the character we are used to. The hex sequence 0xE2, 0x80, 0xA6 that get back is, in fact, the code for the horizontal ellipsis, the 3 horizontal dots PowerShell uses to indicate a cut off value.

There are most probably many other characters or character combinations that do not translate well, but at least we have a method to correct this.

InFile & OutFile

Using inline files and the stdout stream for input and results works in most cases, but it might make things needlessly complex in others. To avoid that, we can make use of the InFile and OutFile parameters on the Invoke-VMScriptPlus function. The parameters allows us to pass one or more files along with the script, and retrieve one or more files back from the environment where the script runs.

If we use InFile and OutFile together with containers, there is some Inception going on. The following schematic might make this a bit more visible.

As you can see, the InFile files are stored in the guest OS of the instance. And then through the Volume parameter on the docker command, passed along into the container.

The same goes for the OutFile files but in the reverse sense.

We are going to execute an existing script. Note how to make a small change in the path of the resulting file. We have to use the path to the Scripts folder, which will be mapped through the Volume parameter.

Get-VMHost |
Select Name, NumCpu, Version |
ConvertTo-Html | Out-File -FilePath \Scripts\report.html

Write-Host "Script completed"

And this is the script that will run that script in a container.

$vmName = 'ubuntubioniccontainer'
$oldScript = '.\Sample-PCLI-Script.ps1'
$tempScript = 'script.ps1'

$script = @'
reset -w
$ProgressPreference = "SilentlyContinue"
if(-not (Get-Module -Name VMware.PowerCLI -ListAvailable)){
    Install-Module -Name VMware.PowerCLI -AllowClobber -Force | Out-Null
}
$WarningPreference = 'SilentlyContinue'
$sConf = @{
    InvalidCertificateAction = 'Ignore'
    ParticipateInCeip = $false
    DisplayDeprecationWarnings = $false
    Scope = 'User'
    Confirm = $false
}
Set-PowerCLIConfiguration @sConf | Out-Null
Connect-VIServer -Server $vcsaName -User $vcsaUser -Password $vcsaPswd | Out-Null

####### My Script ######
$(Get-Content -Path $oldScript -Raw)
########################

Disconnect-VIServer -Server $vcsaName -Confirm:$false
'@

$docker = @'
sudo docker volume create PCLIModules > /dev/null
sudo docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v $(pwd):/Scripts \
    -e TERM=xterm -e LINES=25 -e COLUMNS=135 \
    microsoft/powershell:latest \
    pwsh -F /Scripts/$($tempScript)
'@

$vcsaName = 'vcsa.local.lab'
$credVCSA = Get-VICredentialStoreItem -Host $vcsaName
$vcsaUser = $credVCSA.User
$vcsaPswd = $credVCSA.Password

$credVM = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credVM.User, (ConvertTo-SecureString $credVM.Password -AsPlainText -Force))

$tempScript = 'script.ps1'
$ExecutionContext.InvokeCommand.ExpandString($script) | Out-File -LiteralPath $tempScript
$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $ExecutionContext.InvokeCommand.ExpandString($docker)
    NoIPinCert = $true
    InFile = $tempScript
    OutFile = 'report.html'
}
Invoke-VMScriptPlus @sInvoke

Remove-Item -LiteralPath $tempScript -Confirm:$false

Invoke-Item -Path .\report.html

Annotations

Line 2: The name of our script we want to execute. Note that we had to change the Path of the resulting file. This to allow the mapping through the volume parameter on the docker run command.

Line 3: A filename for the script we are going to run in the container. Ultimately this could be replaced by a randomly generated filename.

Line 5-27: The full script we are sending to the container. It is comprised of the Prefix, our original script and the Suffix. The Prefix and Suffix are the same as we have shown in previous sections. The only difference is that all three parts now come together in one here-string.

Line 23: We substitute this line with our original script later on with the call to ExpandString.

Line 29-38: The actual bash script, with the docker commands, that we are running inside the Guest OS of the instance.

Line 49: We expand the $script variable and save the resulting text to a file.

Line 54: Another call to ExpandString. This time to substitute the variables in the $docker variable.

Line 56: The script file we send along to the Guest OS.

Line 57: The output file we retrieve from the Guest OS.

The script's output looks like this. Notice that it only contains the result from the Write-Host line in our original script.

And the result, report.html, is displayed in a web browser through the Invoke-Item cmdlet.

Image vmware/powerclicore

Shortly after the new tdnf package for PowerShell was made available, a new version of the vmware/powerclicore docker image was created.

The difference with the microsoft/powershell image is the underlying OS, Photon v3R2 instead of Ubuntu 18.04.

Let's start with a simple script, similar to the one we used earlier.

$code = @'
docker run --rm microsoft/powershell:latest pwsh -C 'whoami;Get-Process;pwd'
'@

$credItem = Get-VICredentialStoreItem -Host $vmName
$cred = New-Object System.Management.Automation.PSCredential ($credItem.User, (ConvertTo-SecureString $credItem.Password -AsPlainText -Force))

$sInvoke = @{
    VM = $vmName
    GuestCredential = $cred
    ScriptType = 'bash'
    ScriptText = $code
    NoIPinCert = $true
    GuestOSType = 'linux'
}
Invoke-VMScriptPlus @sInvoke | select -ExpandProperty ScriptOutput

With the microsoft/powershell image this produces the following. We see the result of all three commands.

On the vmware/powerclicore image, the result is a bit different. The whoami output is there, but the output from the other two commands is missing.

As it turns out, there is an issue when passing commands to pwsh via the Command parameter. When using the File parameter everything seems to work well.

One difference to note, compared with the micorosft/powershell image, the vmware/powerclicore:latest image, and the Photon OS in that image, have minimal terminal types present. As a result, you will need to set the TERM environment variable to linux instead of xterm.

A typical call would look something like this.

docker volume create PCLIModules > /dev/null
docker run --rm \
    --tty \
    -v /PCLIModules:/root/.local/share/powershell/Modules \
    -v `$(pwd):/Scripts \
    -e TERM=linux -e LINES=25 -e COLUMNS=135 \
    vmware/powerclicore:latest \
    pwsh -F /Scripts/script.ps1

Image mcr.microsoft.com/powershell:preview

Note that there is also an image for the latest PS v7 Preview available. If you want to test/develop your scripts in this PS v7 Preview, this is a simple and easy method to do it.

All previous methods discussed in this post work against this Preview.

Enjoy!

p

In this first part, we will introduce cloud-init and how you can use it from your PowerShell/PowerCLI scripts. Since the Ubuntu distribution is very popular, on-premises and in the cloud, this introduction will focus on that distro to demonstrate the concept. In the following parts, we will tackle Photon, containers and how to run your scripts on these stations.

Before diving into the technical stuff, first a reminder why we should treat our servers as cattle.

A list with some of the major arguments (at least for me).

• No maintenance required when using the latest distributions. Meaning no long-running updates and security patches before usage.
• No “less than 5% used” machines anymore in your environment.
• No risk of system tattooing from any previous usage.
• An easy way to test new versions of your scripts, the OS, PowerShell and PowerCLI.

Ingredients

To use cloud-init, the requirements, in the cases we will handle here, are minimal.

• An OVA image that is configured to use cloud-init
  • Several Linux distributions nowadays have such a ‘cloud’ image: Ubuntu, Photon…
• A datasource, which is the configuration data provided by the user to the cloud-init process, and which defines how the resulting station will be configured.
• A cloud, in it’s broadest sense, to run the VM. From the cloud-init documentation “… all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations.”
• A script to trigger and control it all.

Canonical, the company behind Ubuntu, maintains a document, named Cloud Instance Initialisation with cloud-init, that has a schematic that captures the cloud-init process perfectly.

If we annotate and update this schema for the usage we have in mind, it becomes like this.

The OVA File

Since our target cloud platform in this series is a vSphere environment, we will use OVA files as the source for the VM(s) we are going to deploy.

The Ubuntu OVA image that we will be using in this post is for Ubuntu Server 18.04 LTS, aka Bionic Beaver.

This OVA image allows using the OVFProperties as the datasource. Note that the user-data we pass in this way needs to be Base64 encoded.

The User-data

As mentioned in the previous section, in this Ubuntu image we will use the OVFProperties as the datasource for our user-data.

The content of the user-data is provided to the script as a YAML file. The syntax for such a file and the available is defined in the cloud-init Documentation.

The following is the sample YAML file we will use in the rest of this post.

#cloud-config
hostname: ubuntubionic
fqdn: ubuntubionic.local.lab
write_files:
- path: /etc/netplan/50-cloud-init.yaml
  content: |
    network:
     version: 2
     ethernets:
      ens192:
       addresses: [192.168.10.79/24]
       gateway4: 192.168.10.1
       dhcp6: false
       nameservers:
         addresses:
           - 192.168.10.2
           - 192.168.10.3
         search:
           - local.lab
       dhcp4: false
       optional: true
- path: /etc/sysctl.d/60-disable-ipv6.conf
  owner: root
  content: |
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv6.conf.default.disable_ipv6=1
runcmd:
- netplan --debug apply
- sysctl -w net.ipv6.conf.all.disable_ipv6=1
- sysctl -w net.ipv6.conf.default.disable_ipv6=1
- apt-get -y update
- add-apt-repository universe
- apt-get -y clean
- apt-get -y autoremove --purge
timezone: Europe/Brussels
system_info:
  default_user:
    name: default-user
    lock_passwd: false
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
disable_root: false
ssh_pwauth: yes
users:
  - default
  - name: luc
    gecos: LucD
    lock_passwd: false
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
chpasswd:
  list: |
    default-user:$6$aPp//e2ueP$ETEXcAhAyQuJ4qNCbqxmmSYGZbg2wFwpP/YITvoXdgxwZBnf32drePKi2OIn5fLqtH5pHO03yRdPXK3ToLG6b0
    luc:$6$kW1WwJ2K$M6415du1BZd.qt92SvR6X.RuyDhEZmgR4hz4NcKH9XHn2850Vc6zHpubXM6uUeqMUaJQ740ogROB74gfBEhn9.
    root:$6$Js9CVr06$br9qf0VxuBsdY7Vtg/0pk9jLlycYBDLVsvbKwLDleCK7dSDheOxWaFOWdjkiqSPRrWG./N8V5RgCVwugZGnTc1
  expire: false
package_upgrade: true
package_reboot_if_required: true
power_state:
  delay: now
  mode: reboot
  message: Rebooting the OS
  condition: if [ -e /var/run/reboot-required ]; then exit 0; else exit 1; fi

Annotations

Line 1: the user-data file always has to start with the line ‘# cloud-config’

Line 2-3: These lines follow the regular YAML syntax of key-value entries. The lines defines the hostname and the FQDN that the instance will get.

Line 4-26: The write-files section instructs cloud-init to create files, with the specified properties and content on the instance. In this example file the files are used to define a static IPv4 address and to disable IPv6. Which files need to be used is dependent on the OS that runs in the VM. Note that this example will use netplan for the network configuration.

Line 27-34: The runcmd section defines commands that will run on the instance during the first boot of the guest OS. In this example the commands are used to activate the network (it uses netplan), set some variables and set and update the application repositories.

Line 35: This line defines which timezone needs to be configured.

Line 36-40: An Ubuntu image comes configured with a default user. In the system_info section, the settings for this default user are specified.

Line 43-50: The users section allows to define additional users on the instance. Note that the default user needs to come first in this list.

Line 51-56: The chpasswd allows you to change and/or set the passwords for the users. The required format and how these entries are created will be discussed in one of the following sections.

Line 57-58: These lines request an upgrade of the packages present on the instance. And perform a reboot when one of the package upgrades would require that.

Line 59-63: The power-state section instructs cloud-init in which state the instance shall be left after cloud-init completes. In this example, the system is rebooted on condition that a specific file exists. This file is created by cloud-init when it has decided that a reboot of the instance is required.

Password encoding

In user-data, there are several places where a password can be provided. The value needs to be the hash of the password, not the password itself. With the help of an existing Linux box (UbuntuWork in this case), I use the following script to create such SHA-512 encoded password hashes. The script uses the mkpasswd command to generate the hash.

Automation rules!

function Get-PasswordHash
{
    param(
        [String]$VMName,
        [String]$GuestUser,
        [String]$GuestPassword,
        [String]$Password
    )

    $sInvoke = @{
        VM = $VMName
        ScriptType = 'bash'
        ScriptText = "mkpasswd -m SHA-512 $Password"
        GuestUser = $GuestUser
        GuestPassword = $GuestPassword
    }
    (Invoke-VMScript @sInvoke).ScriptOutput.Trim("n")
}

$sHash = @{
    VM = 'UbuntuWork'
    GuestUser = 'root'
    GuestPassword = 'VMware1!'
    Password = 'VMware1!'
}
Get-PasswordHash @sHash

This returns something like this.

A word of warning, such hashed password are not super-safe. A decent password cracker with sufficient compute power and a good wordlist can most probably crack this in less than one minute.

The Script

function Install-CloudInitVM
{
  <#
.SYNOPSIS
  Deploy a VM from an OVA file and use cloud-init for the configuration
  .DESCRIPTION
  This function will deploy an OVA file.
  The function transfer the user-data to the cloud-init process on the VM with
  one of the OVF properties.
.NOTES
  Author:  Luc Dekens
  Version:
  1.0 05/12/19  Initial release
.PARAMETER OvaFile
  Specifies the path to the OVA file
.PARAMETER VmName
  The displayname of the VM
.PARAMETER ClusterName
  The cluster onto which the VM shall be deployed
.PARAMETER DsName
  The datastore on which the VM shall be deployed
.PARAMETER PgName
  The portgroupname to which the VM shall be connected
.PARAMETER CloudConfig
  The path to the YAML file containing the user-data
.PARAMETER Credential
  The credentials for a user in the VM's guest OS
.EXAMPLE
  $sCloudInitVM = @{
    OvaFile = '.\bionic-server-cloudimg-amd64.ova'
    VmName = $vmName
    ClusterName = $clusterName
    DsName = $dsName
    PgName = $pgName
    CloudConfig = '.\user-data.yaml'
    Credential = $cred
  }
  Install-CloudInitVM @sCloudInitVM
  #>

  [cmdletbinding()]
  param(
    [string]$OvaFile,
    [string]$VmName,
    [string]$ClusterName,
    [string]$DsName,
    [string]$PgName,
    [string]$CloudConfig,
    [PSCredential[]]$Credential
  )

  $waitJob = (Get-Command -Name .\Wait-Job.ps1).ScriptBlock
  $userData = Get-Content -Path $CloudConfig -Raw

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting deployment of $vmName"

  $start = Get-Date

  $vm = Get-VM -Name $vmName -ErrorAction SilentlyContinue
  if ($vm)
  {
    Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Cleaning up"
    if ($vm.PowerState -eq 'PoweredOn')
    {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -DeletePermanently -Confirm:$false
  }

  $ovfProp = Get-OvfConfiguration -Ovf $ovaFile
  $ovfProp.Common.user_data.Value = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($userData))
  $ovfProp.NetworkMapping.VM_Network.Value = $pgName

  $sApp = @{
    Source = $ovaFile
    Name = $vmName
    Datastore = Get-Datastore -Name $dsName
    DiskStorageFormat = 'Thin'
    VMHost = Get-Cluster -Name $clusterName | Get-VMHost | Get-Random
    OvfConfiguration = $ovfProp
  }
  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Importing OVA"
  $vm = Import-VApp @sApp

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Starting the VM"
  Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Waiting for cloud-init to finish"

  $User = $Credential.GetNetworkCredential().UserName
  $Password = $Credential.GetNetworkCredential().Password

  $sJob = @{
    Name = 'WaitForCloudInit'
    ScriptBlock = $waitJob
    ArgumentList = $vm.Name, $User, $Password, $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId
  }
  Start-Job @sJob | Receive-Job -Wait

  Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') - Deployment complete"

  Write-Verbose "nDeployment took $([math]::Round((New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds,0)) seconds"
}

Annotations

Line 52: To find out when cloud-init has finished it's run, the function uses a background job. This background job is stored in a separate .ps1 file. More on the content and purpose of that 'wait' script later.

Line 53: The user-data is read from the YAML file. It is important to use the Raw switch, otherwise, we would lose the line separators.

Line 60-68: If a VM with the same Displayname is already present, it will be stopped (when powered on) and removed.

Line 70-71: The user-data is assigned to the Common.user_data property of the OVF properties. The user-data needs to be converted to base64.

Line 72: The Ubuntu OVA allows to pass the Portgroup to which the VM shall be connected.

Line 74-83: The VM is installed from the OVA with the Import-VApp cmdlet.

Line 86: The VM is powered on. During the boot process, the first phase of cloud-init will run.

Line 90-98: The function uses a background job to check if the cloud-init process has completed. More on that later.

The Wait job

I decided to store the WaitJob script in a separate .ps1 file. Primarily because I can easily reuse it this way and it makes the code of scripts that use shorter.

The Wait Job is started as a background job. This allows the calling code to simply use the Wait-Job cmdlet.

[CmdletBinding()]
param(
    [string]$vmName,
    [string]$user,
    [string]$pswd,
    [string]$vcsaName,
    [string]$SessionId
)

$sleepTime = 5
Connect-VIServer -Server $vcsaName -Session $SessionId | Out-Null
$notFinished = $true
while ($notFinished)
{
    Try
    {
        $vm = Get-VM -Name $vmName -ErrorAction Stop
        while ($vm.PowerState -ne 'PoweredOn' -and -not $vm.ExtensionData.Guest.GuestOperationsReady)
        {
            Start-Sleep -Seconds $sleepTime
        }
        $fileExist = $false
        while (-not $fileExist)
        {
            $sInvoke = @{
                VM = $vm
                ScriptType = 'bash'
                ScriptText = '[ -f /var/lib/cloud/instance/boot-finished ] && echo "File exist"'
                GuestUser = $user
                GuestPassword = $pswd
                ErrorAction = 'Stop'
            }
            try
            {
                $result = Invoke-VMScript @sInvoke
                $fileExist = [Boolean]$result.ScriptOutput
            }
            catch
            {
                Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') Exception:tId: $($error[0].Exception.ErrorId)  Category: $($error[0].Exception.ErrorCategory)"
                Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff')ttLine: $($error[0].InvocationInfo.Line)"
            }
        }
        $sInvoke = @{
            VM = $vm
            ScriptType = 'bash'
            ScriptText = 'cat /var/lib/cloud/instance/boot-finished'
            GuestUser = $user
            GuestPassword = $pswd
        }
        $result = Invoke-VMScript @sInvoke
        $result.ScriptOutput
        $notFinished = $false
    }
    catch
    {
        Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff') Exception:tId: $($error[0].Exception.ErrorId)  Category: $($error[0].Exception.ErrorCategory)"
        Write-Verbose "$(Get-Date -Format 'HH:mm:ss.fff')ttLine: $($error[0].InvocationInfo.Line)"
    }
    Start-Sleep -Seconds $sleepTime
}

Annotations

Line 7 + 11: In a background job we do not inherit any open connections to a vSphere Server. A script can use the SessionId of an open connection in the calling script, to open a connection, without having to provide credentials.

Line 18: Before using the call to Invoke-VMScript, the Wait Job makes sure the VM and the VMware Tools in there, are ready to receive such a call.

Line 28: Inside the guest OS, the function uses the simple bash expression to test for the presence of a file. The file /var/lib/cloud/instance/boot-finished is created by cloud-init when the process completes.

Line 44-52: The Wait Job function returns the content of the file to the caller.

Sample Run

With the above code, we have everything in place to deploy a VM from an Ubuntu OVA, with the guest OS configuration is done through cloud-init, based on a YAML file.

A typical call could look something like this.

$vmName = 'UbuntuBionic'

# Get credential for logging on to the guest OS

$viCred = Get-VICredentialStoreItem -Host $vmName
$secPassword = ConvertTo-SecureString -String $viCred.Password -AsPlainText -Force
$cred = [Management.Automation.PSCredential]::new($viCred.User, $secPassword)

$sCloudInitVM = @{
  OvaFile = '.\bionic-server-cloudimg-amd64.ova'
  VmName = $vmName
  ClusterName = 'cluster'
  DsName = 'vsanDatastore'
  PgName = 'vdPg1'
  CloudConfig = '.\user-data-bionic.yaml'
  Credential = $cred
  Verbose = $true
}
Install-CloudInitVM @sCloudInitVM

With the Verbose switch set to $true, the output of this call would look like this.

I highlighted with red boxes the verbose messages from the function. The text in the green box is the content of the /var/lib/cloud/instance/boot-finished file.

Do not look too much at the timings. This run happened in my lab environment, which has limited resources. The main reason for showing this is to demonstrate how easy such a cloud-init based deployment can be incorporated in your pipelines.

This concludes Part 1 in the cloud-init series. It provides some functions and scripts to deploy a VM, starting from an OVA file, with the configuration of the guest OS done through cloud-init.

In the upcoming parts in this series, I will show some more advanced Ubuntu deployments, show how cloud-init can be used with Photon and show how you can use these deployed VMs as your 'cattle'.

Enjoy!