There are numerous articles and blog posts that describe how set up a Photon 2.0 VM, but most of these use the ovftool or the vSphere Web Client to install it. Followed by quite of a bit of editing config files, to have your Photon 2.0 VM running the way you prefer it.
With the latest version of my Invoke-VMScriptPlus function, you can now automate this entire process, the SDDC way! 

Update August 21st 2018

• Added “reboot” action
• Added optional Environment variables for the customisation scripts
• Added Folder field for template destination

Intro

As I briefly mentioned in the introduction paragraph, setting up a VM with the Photon OS 2.0, is a two-step process.

• Deploy the appliance
• Configure the OS

The first step is rather straight-forward with PowerCLI’s Import-VApp cmdlet.

The second step, configuring the Photon OS, is presented as a manual step in most of the articles I could find. Till now!

With my Invoke-VMScripPlus function, this step can now also be automated. And that automation step is driven by a JSON file. This JSON file is the part that you place under version control.

The following scenario, which I’m currently using, creates a VM with a configured Photon 2.0 guest OS. Then converts that VM into a Template, which can then be used to roll out your VMs.

Create a Photon VM Template

This step is fully automated and driven by a JSON configuration file. In a matter of minutes you can create a new Photon 2.0 template, without even logging on to the machine!

The JSON Configuration file

The objective to have all configuration parameters in a flat text file is of course to be able to easily place it under version control and to easily compare between versions.

The JSON file contains a number of Level-1 entries, each specifying part of the final configuration of the Photon Template.

• vSphere: describes the Location for the template
• Template: characteristics of the Template
• Network: the network configuration inside the guest OS
• Account: the current and new password for the root account
• Proxy: the proxy settings, if present
• Docker: the docker configuration
• Code: a collection of customisation scripts

vSphere

• VMHost: a cluster or an ESXi node where the VM will be installed. If you specify a cluster, the script takes a random ESXi node in that cluster.
• Storage: a datastorecluster or datastore where the VM will be installed.
• OvaPath: where the script can find the Photon 2.0 ova file. Note the escaped back-slashes in the path.

Template

• TemplateName, TemplateNote: self explanatory
• Folder: a VM type folder where the template shall be stored
• NumCpu, MemoryGB, HarddiskGB: self explanatory
• GuestID: has to be other3xLinux64Guest for Photon 2.0
• CreateTemplate: when set to false, the script will not power off the VM, nor will it convert the VM into a Template. This can be used will fine-tuning the customisation scripts (see later).
• CustomisationCode: the names of the customisation scripts that will be executed once the VM is installed. See later in the Code section.

Network

Since most of the customization of the Photon guest OS was done when we created the Template, this step requires very little extra customization steps.

I assume these settings all speak for themselves. Nothing out of the ordinary, just you regular network settings. Perhaps one point to note, when you have multiple values for a field, separate them with a space.

Account

Again, quite straight-forward. The account, and the old and new password.

Proxy

When your environment into which you are installing the VM is located behind a proxy, you can specify the proxy details in this section. This is required to get the Photon repositories working (tdnf), and also the docker repositories,

Docker

Only one setting, the default port to access docker.

Code

Under the Code section, you can define a number of customisation scripts, that will be executed inside the Photon 2.0 guest OS. Each entry has a two fixed subfields.

• Name: the name for this particular customisation script. This is the name that is used in the Template section, in CustomisationCode field.
• Script: the actual customisation script. Due to restrictions in the JSON syntax, the script must be provided as an array of strings. Each line of the script is an element in this array
• Environment: an array with environment variables that will be passed to the environment where the customisation script is executed.

In the example above, the customisation script is a one-liner. Hence one string in the Code array. In this example, the Code contains the line to pull the docker container that contains VMware PowerCLI.

There are of course multi-line customisation scripts. In that case the array contains multiple elements. The following example is an excerpt of the docker customisation script. As you notice, the code also contains comment lines and empty lines.

Since we do not want to hard-code values in our customisation scripts, we use a PowerShell style notation to refer to variables defined in other sections in the JSON file.

In line 126 of excerpt above, we use the expression $($paramData.Docker.Port) to refer to the port number we defined in the Docker section. 

To avoid that you have to type in your customisation scripts in this specific JSON format, the following short script can help you convert your regular bash script to JSON notation.

function Convert-Script2Json
{
  [cmdletbinding()]
  param(
    [Parameter(Mandatory = $true)]
    [string]$ScriptName,
    [Parameter(Mandatory = $true)]
    [string[]]$Script
  )

  $obj = New-Object PSObject -Property @{
      Name = $ScriptName
      Script = $script.Split("n") | %{$_.TrimEnd("r")}
  }
  $obj | ConvertTo-Json
}

A sample run shows you how you could use the above code.

@'
# Update packages
# Separate to avoid script ending with VMware Tools update

tdnf update -y
'@

Convert-Script2Json -ScriptName packages -Script $code

Will result in the following output, which you can copy/paste into your JSON file.

The New-PhotonTemplate function

Before you can run the New-PhotonTemplate function, you will need two functions that are used in the function. These are

• Invoke-VMScriptPlus from my Invoke-VMScriptPlus v2 post.
• Set-VMKeystrokes, from William Lam’s Automating VM keystrokes using the vSphere API & PowerCLI 

You have multiple options to make these two functions available to the New-PhotonTemplate function.

• Add them to the .ps1 file where you copied the New-PhotonTemplate function
• Store the two functions in separate .ps1 files, and dot-source these .ps1 files before calling the New-PhotonTemplate function
• Create a module, and add the two functions to that module

The Code

function New-PhotonTemplate {
<#
.SYNOPSIS
 Create a Template with the Photon 2.0 guest OS
.DESCRIPTION
 This function will create a new Template with the Photon 2.0
 guest OS. The parameters and the configuration scripts are passed via
 a JSON file.
.NOTES
 Author:  Luc Dekens
 Version:
 1.0 11/08/18  Initial release
 1.1 16/08/18  Create function
               Added support for Reboot action
               Added support for environment variable
 1.2 17/08/18  Added Folder (in JSON file) support
              
.PARAMETER JSONPath
 The location of the JSON file with the configuration parameters
.PARAMETER LogFile
 Optional file for capturing logging information.
 This includes the output of the customisation scripts
.EXAMPLE
 New-PhotonTemplate -JSONPath .\photon.json
.EXAMPLE
 New-PhotonTemplate -JSONPath .\photon.json -LogFile .\photon.log
#>

  [cmdletbinding()]
  param(
    [Parameter(Mandatory = $true)]
    [string]$JSONPath,
    [string]$LogFile
  )

  function Write-ToLog {
    [CmdletBinding()]
    param(
      [string]$Category,
      [string]$Text
    )

    $timeStamp = (Get-Date).ToString('yyyyMMddThhmmss')
    $script = Split-Path -Path $myInvocation.ScriptName -Leaf
    $user = $env:UserName

    $line = "$timeStamp - $script - $user - [$Category] $Text"
    if ($LogFile) {
      $line | Out-File -FilePath $LogFile -Append
    }
    else {
      $line
    }
  }

  function Invoke-VMReboot {
    [CmdletBinding()]
    param(
      [string]$VMName
    )

    $vmGuest = Get-VMGuest -VM $VMName

    Stop-VMGuest -VM $vm -Confirm:$false | Out-Null
    while ($vmGuest.VM.PowerState -ne 'PoweredOff') {
      Start-Sleep 1
      $vmGuest = Get-VMGuest -VM $vmGuest.VM
    }

    $vmGuest = Start-VM -VM $vmGuest.VM -Confirm:$false | Get-VMGuest
    while ($vmGuest.VM.PowerState -ne 'PoweredOn' -or -not $vmGuest.ExtensionData.GuestOperationsReady) {
      Start-Sleep 1
      $vmGuest = Get-VMGuest -VM $vmGuest.VM
    }
  }

  # Get Parameters

  $paramDataJSON = Get-Content -Path $JSONPath | Out-String
  $paramData = ConvertFrom-Json -InputObject $paramDataJSON

  # Determine location for template

  $obj = Get-Inventory -Name $paramData.vSphere.VMHost
  if ($obj -is [VMware.VimAutomation.ViCore.Types.V1.Inventory.Cluster]) {
    $obj = Get-VMHost -Location $obj | Get-Random
  }
  $esx = $obj

  $dsc = Get-DatastoreCluster -Name $paramData.vSphere.Storage -ErrorAction SilentlyContinue
  if ($dsc) {
    $ds = Get-Datastore -RelatedObject $dsc | Get-Random
  }
  else {
    $ds = Get-Datastore -Name $paramData.vSphere.Storage
  }

  # Clean up eventual artifacts

  if ($vm = Get-VM -Name $paramData.Template.TemplateName -ErrorAction SilentlyContinue) {
    if ($vm.PowerState -eq [VMware.VimAutomation.ViCore.Types.V1.Inventory.PowerState]::PoweredOn) {
      Stop-VM -VM $vm -Confirm:$false | Out-Null
    }
    Remove-VM -VM $vm -Confirm:$false -DeletePermanently
    Write-ToLog -Category Info -Text "Removed VM $($vm.Name)"
  }
  if ($template = Get-Template -Name $paramData.Template.TemplateName -ErrorAction SilentlyContinue) {
    Remove-Template -Template $template -DeletePermanently -Confirm:$false
    Write-ToLog -Category Info -Text "Removed Template $($template.Name)"
  }

  # Import Appliance

  $ovfParm = Get-OvfConfiguration -Ovf $paramData.vSphere.OvaPath
  $ovfParm.NetworkMapping.None.Value = $paramData.Network.PortGroup
  $sVApp = @{
    Name              = $paramData.Template.TemplateName
    Source            = $paramData.vSphere.OvaPath
    OvfConfiguration  = $ovfParm
    VMHost            = $esx
    Datastore         = $ds
    DiskStorageFormat = [VMware.VimAutomation.ViCore.Types.V1.VirtualDevice.VirtualDiskStorageFormat]::Thin
    Confirm           = $false
  }
  if($paramData.Template.Folder){
    $sVApp.Add('InventoryLocation',(Get-FolderByPath -Path $paramData.Template.Folder))
  }
  Write-ToLog -Category Info -Text "Importing vApp..."
  $vm = Import-VApp @sVApp
  Write-ToLog -Category Info -Text "Imported vApp $($sVApp.Name)"

  # Configure VM

  $vm = Get-VM -Name $paramData.Template.TemplateName

  $sSet = @{
    VM       = $vm
    NumCpu   = $paramData.Template.NumCpu
    MemoryGB = $paramData.Template.MemoryGB
    Notes    = $paramData.Template.TemplateNote.Replace('#timestamp#', (Get-Date -Format 'dd/MM/yyyy HH:mm'))
    Confirm  = $false
  }
  Set-VM @sSet > $null
  Get-HardDisk -VM $vm | Set-HardDisk -CapacityGB $paramData.Template.HarddiskGB -Confirm:$false  > $null
  Get-FloppyDrive -VM $vm | Remove-FloppyDrive -Confirm:$false > $null
  Write-ToLog -Category Info -Text "Configured VM $($vm.Name)"

  Start-VM -VM $vm > $null
  Write-ToLog -Category Info -Text "Power on VM $($vm.Name)"

  while ($vm.ExtensionData.Guest.ToolsRunningStatus -ne [VMware.Vim.VirtualMachineToolsRunningStatus]::guestToolsRunning -or
    $vm.ExtensionData.Runtime.PowerState -ne [VMware.Vim.VirtualMachinePowerState]::poweredOn) {
    Start-Sleep 2
    $vm.ExtensionData.UpdateViewData()
  }
  Write-ToLog -Category Info -Text "VM $($vm.Name) is powered on"

  # Change Photon OS default root password
  # Thanks to William Lam for his Set-VMKeystrokes function
  # See https://www.virtuallyghetto.com/2017/09/automating-vm-keystrokes-using-the-vsphere-api-powercli.html

  Write-ToLog -Category Info -Text "Change password for user $($paramData.Account.User)"

  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.User -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.OldPassword -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.OldPassword -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.NewPassword -ReturnCarriage $true | Out-Null
  Set-VMKeystrokes -VMName $VM -StringInput $paramData.Account.NewPassword -ReturnCarriage $true | Out-Null

  Set-VMKeystrokes -VMName $VM -StringInput "exit" -ReturnCarriage $true | Out-Null

  $vm = Get-VM -Name $paramData.Template.TemplateName

  # Run all customisation scripts

  foreach ($code in $paramData.Template.CustomisationCode) {
    if ($code -eq 'reboot') {
      Write-ToLog -Category Info -Text "Rebooting VM $($vm.Name)..."
      Invoke-VMReboot -VMName $vm.Name
      Write-ToLog -Category Info -Text "Rebooted VM $($vm.Name)"
    }
    else {
      $codeObj = $paramData.Code.Where{$_.Name -eq $code}
      $sInvoke = @{
        VM               = $vm
        ScriptType       = 'Bash'
        ScriptText       = $ExecutionContext.InvokeCommand.ExpandString(($codeObj.Script -join "`r`n"))
        GuestUser        = $paramData.Account.User
        GuestPassword    = ConvertTo-SecureString -String $paramData.Account.NewPassword -AsPlainText -Force
        GuestOSType      = 'Linux'
      }
      if($codeObj.Environment){
        $sInvoke.Add('ScriptEnvironment',$codeObj.Environment.foreach{$ExecutionContext.InvokeCommand.ExpandString($_)})
      }
      Write-ToLog -Category Info -Text "Running $code"
      $result = Invoke-VMScriptPlus @sInvoke
      if ($LogFile) {
        $result | Out-File -FilePath $LogFile -Append
      }
      else {
        $result
      }
    }
  }

  # Create Template

  if ($paramData.Template.CreateTemplate) {
    $vm = Get-VM -Name $paramData.Template.TemplateName

    if ($vm.PowerState -eq [VMware.VimAutomation.ViCore.Types.V1.Inventory.PowerState]::PoweredOn) {
      Write-ToLog -Category Info -Text "Stopping VM $($vm.Name) ..."
      Shutdown-VMGuest -VM $vm -Confirm:$false > $null
      while ($vm.PowerState -ne [VMware.VimAutomation.ViCore.Types.V1.Inventory.PowerState]::PoweredOff) {
        Start-Sleep 3
        $vm = Get-VM -Name $paramData.Template.TemplateName
      }
      Write-ToLog -Category Info -Text "VM $($vm.Name) stopped"
    }

    Write-ToLog -Category Info -Text "Convert VM $($vm.Name) to Template"
    Set-vm -VM $vm -ToTemplate -Confirm:$false > $null
  }
}

Annotations

Line 36-54: An inline function to have all messages in the same format.

Line 56-75: A reboot the guest OS function. Called when the “reboot” instruction is encountered. See later.

Line 79-80: These lines read the JSON file and convert the data to a PowerShell object

Line 84-88: The vSphere.VMHost field can contain a cluster or an ESXi node. These lines handle those options. In case of a cluster, the function selects a random ESXi node from the cluster.

Line 90-96: The vSphere.Storage field can contain a DatastoreCluster or a Datastore. These lines handle those options. In case of a datastorecluster, the function selects a random datastore from the datastorecluster.

Line 100-110: These lines check if there is already a Template or a VirtualMachine with a name as specified in Template.TemplateName. If there is, the Template or VirtualMachine are removed.

Line 114-130: A straight-forward method to import the Photon OVA as defined in vSphere.OvaPath.

Line 134-146:  Further HW configuration of the VM

Line 148-156: The new VM is powered on, and the function waits till the VMware Tools are responding

Line 162-170: The function changes the root password with the new password defined in Account.NewPassword. For this action the code uses the Set-VMKeystrokes function. William describes this ingenious method in his Automating VM keystrokes using the vSphere API & PowerCLI post, well worth a read!

Line 176-204: The code now runs all the customisation script that are defined in Template.CustomisationCode. The actual customisation scripts are defined in the Code section of the JSON file.

Line 177-182: With reserved name “reboot” you can request a reboot of the guest OS. This is executed in the sequence order with the customisation scripts as defined in the Template.CustomisationCode field.

Line 187,193: With the ExpandString method the script expands all $paramData occurrances in the customisation scripts with the actual values. This allows us to keep the customisation script general and not coded for a specific instance.

Line 192-194: If the customisation script contains environment variables in the Code.Environment field, these are environment variables are passed along with the customisatoin script to the Invoke-VMScriptPlus function.

Line 208-223: Based on the Template.CreateTemplate value (true or false) the VM is converted into a Template. One reason not to convert the VM to a Template, could be that your debugging one or more of the customisation scripts.

Sample Run

Now we have everything in place to create a Photon 2.0 template, based on the  configuration settings and customisation scripts in our JSON file. Since it can be useful to review the output, especially of the customisation scripts, we use the LogFile parameter to specify a file where the function can write the output.

New-PhotonTemplate -JSONPath .\LabParams.json -LogFile C:\Temp\photon-template.log

When we call the New-PhotonTemplate with the LogFile parameter, there will be no regular output on the PowerShell console.

The logfile contains useful information, especially if we add statements to our customisation scripts to retrieve information on the components. This excerpt shows some information that was obtained in the docker customisation script.

When all goes well, the New-PhotonTemplate will have created a Template with the Photon 2.0 guest OS.

Known Issues

open-vm-tools

When you use one of the customisation scripts to upgrade the Photon components (command tdnf upgrade), and the open-vm-tools are in there, the returned information of the script might not be complete. You will miss the ScriptOutput and the ExitCode.

As a workaround, upgrade the open-vm-tools separately before upgrading any of the other components.

Files

Attached to this post are some files that will help you get started. There is an empty, skeleton JSON file and two Photon customisation scripts, one for the network and one for docker.

Next

In Part 2 of this series I will be showing how the Template we just created, can be used to roll out any number of VMs.

Enjoy!

]]> https://www.lucd.info/2018/08/14/deploy-photon-2-0-part-1/feed/ 5 https://www.lucd.info/2017/01/10/upgrade-appliance-ovftools/ https://www.lucd.info/2017/01/10/upgrade-appliance-ovftools/#comments Tue, 10 Jan 2017 18:43:44 +0000 http://www.lucd.info/?p=5518 Appliances are hot!

Each appliance, delivered as an OVA or OVF, can have one or more properties attached to it. These properties are mostly used to configure the appliance. In PowerCLI we have the Get-OvfConfiguration cmdlet. It returns the user configurable properties from an OVF or OVA file in a hash table.

But what about upgrading an appliance? Most, if not all, of the Ovf Properties are already entered for the older version of the appliance. Can’t we use that information to upgrade the appliance? And avoid having to retype all that information?

And to take away the suspense, of course, that can be done with a bit of PowerShell!

The OvfTools

function Get-OvfVm
{
  [cmdletbinding()]
  param(
    [Parameter(Mandatory=$true)]
    [PSObject]$VM
  )

  Process{
    if($VM -is [STring]){
      $VM = Get-VM -Name $VM
    }
    $result = New-Object PSObject -Property @{
      VmName = $vm.Name
      OvfProperties = @()
    }

    $vm.ExtensionData.Config.VAppConfig.Property | 
    where{$_.UserConfigurable} | %{
      $obj = [ordered]@{
        Name = $_.Id
        Value = $_.Value
        Type = $_.Type
        Label = $_.Label
        Description = $_.Description
      }
      $result.ovfProperties += New-Object PSOBject -Property $obj
    }
    $result
  }
}

function Update-OvfVm
{
  [cmdletbinding()]
  param(
    [Parameter(Mandatory=$true)]
    [PSObject]$VM,
    [Parameter(Mandatory=$true)]
    [String]$Path,
    [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
    [PSObject]$OvfProperties,
    [PSCredential[]]$Credential,
    [Parameter(Mandatory=$true)]
    [PSOBject]$VMHost,
    [Parameter(Mandatory=$true)]
    [PSObject]$Datastore,
    [String]$Suffix = 'new'
  )

  Process{
    if($VM -is [String]){
      $VM = Get-VM -Name $VM
    }
    if($VMHost -is [String]){
      $VMHost = Get-VMHost -Name $VMHost
    }
    if($Datastore -is [String]){
      $Datastore = Get-Datastore -Name $Datastore
    }

    $OVfHash = @{}
    foreach($prop in $OvfProperties.OvfProperties){
      if($OVfHash.ContainsKey($prop.Name)){
        Write-Error -Message "Name $($prop.Name) appears twice!"
      }
      else{
        if($prop.Type -eq 'password'){
          $OVfHash.Add($prop.Name,($Credential | where{$prop.Name -eq $_.UserName} | select -ExpandProperty Password))
        }
        else{
          $OVfHash.Add($prop.Name,$prop.Value)
        }
      }
    }
    $sVApp = @{
      Source = $Path
      Name = "$($VM.Name)_$($Suffix)"
      OvfConfiguration = $OVfHash
      VMHost = $VMHost
      Datastore = $Datastore
    }
    Import-VApp @SVApp
  }
}

Annotations

Line 18: The vApp properties can be found under the VAppConfig property in a VAppConfigInfo object.

Line 19: In the current version of the function we only look at the UserConfigurable properties.

Line 43: The Update-OvfVm function can accept multiple PSCredential objects. And in this way can handle multiple Ovf Properties of type password.

Line 52-60: Poor man’s version of OBN.

Line 68-70: The lines that handle Ovf Properties of type password.

Line 83: Use the Import-VApp cmdlet with the self-constructed OvfConfiguration hash table to import the new appliance.

Sample Run with OvfTools

A recent Appliance that is seeing quite a bit of updates, is the OpBot appliance from Opvizor. See my ChatOps Bot post for more information and further links.

The OpBot appliance has quite a number of properties that need to be provided on installation.

Once installed you can check these Ovf Properties via the vApp Options page. In the case of the OpBot appliance, this also contains the Slack integration token.

With the help of the OvfTools functions, I’m going to upgrade my OpBot v2.0 installation to the newer OpBot v2.0.1 version. And that without having to retype all the Ovf Properties.

The OvfTools functions assume that the version from which you want to copy the Ovf Properties is installed. The appliance doesn’t need to be running. First let’s have a look at what the Get-OvfVm function returns.

Get-OvfVm -VM esx1Bot | Select -ExpandProperty OvfProperties

And all the properties we saw earlier via the vApp Options page, are returned.

Password property

There is one kind of property that we can’t retrieve this way, these are the properties of type Password. To circumvent that issue, I have written the Update-OvfVm function in such a way that the user can provide these passwords via a separate parameter. Consider this a security measure.

In short, the function caller needs to created a PSCredential object, where the User is the Name of the Ovf Password property.

First determine the password (1) properties, and find the propertyname (2).

In the script create a PSCredential object (3) where the username is equal to the propertyname (2). Pass the PSCredential object(s) on the call to the Update-OvfVm function (4).

The complete script to update the OpBot VM from v2.0 to v2.0.1 looks like this.

$ovfPath = 'C:\Temp\opbot_2.0.1\OpBot_2.0.1.ovf'
$vmName = 'esx1bot'
$suffix = '2.0.1'
$esxName = 'esx1*'
$dsName = 'DS1'

$credProperty = 'vmware_password'
$pswd = 'Whatever'

$secPswd = ConvertTo-SecureString -String $pswd -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential ($credProperty,$secPswd)

Get-OvfVm -VM $vmName | 
Update-OvfVm -VM $vmName -Path $ovfPath -Credential $cred -VMHost $esxName -Datastore $dsName -Suffix $suffix

After the script is executed, the new version of the OpBot is deployed. The Ovf Properties are all copied. The switch from the old to the new appliance with some PowerCLI cmdlets should be trivial.

Enjoy!