In my homelab I try to automate as much as possible. I have a number of scripts and functions that help me setting up test environments in my homelab. Since I got quite a number of requests on this, I decided to start a series on my homelab tools.
One of the tasks I automated is the provisioning of VMs. Quite easy when you have a vCenter that manages the cloning process. But in layer 1 of my homelab I’m running ESXi standalone on a PaaS provided server, so no vCenter.
With the Copy-DatastoreItem cmdlet it is easy to clone the files of a VM, but this kind of copy doesn’t know about the different types of VirtualDisk you can have in a VM. As a result, your Thin vdisks become Thick vDisks in the clone. The function in this post avoids that problem by using the VirtualDiskManager for copying the VMDK.
Note that there are a couple of prerequisites: the master VM needs to be powered off and have no snapshots. And till now I only tested this with VMs that run a Windows guest OS.
The Script
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 |
function Clone-MasterVM { <# .SYNOPSIS Clone a VM without using vCenter .DESCRIPTION The function will clone a VM to a specified datastore. Optionally the new VM will be registered and powered on. .NOTES Author: Luc Dekens .PARAMETER MasterName The name of the VM that will be cloned .PARAMETER CloneName The name of the VM that is cloned .PARAMETER DatastoreName The name of the datastore where the clone will be stored .PARAMETER Register Register the clone on the vSphere server .PARAMETER PowerOn Power on the clone. Can only be used when the Register switch is selected. .EXAMPLE PS> Clone-MasterVM -MasterName M1 -CloneName Srv1 -DatastoreName DS1 #> param( [string]$MasterName, [string]$CloneName, [string]$DatastoreName, [switch]$Register, [switch]$PowerOn ) $vm = Get-VM -Name $MasterName if($vm.ExtensionData.Snapshot -or $vm.PowerState -eq "PoweredOn"){ Write-Error "The VM should be powered off and have no snapshots" return } $si = Get-View ServiceInstance $vdkMgr = Get-View $si.Content.virtualDiskManager $fileMgr = Get-View $si.Content.FileManager $dcMoRef = (Get-Datacenter -VM $vm).ExtensionData.MoRef $srcDSName = $vm.ExtensionData.Config.Files.VmPathName.Split(']')[0].Trim('[') $srcDS = Get-Datastore -Name $srcDSName $srcDSBrowser = Get-View $srcDS.ExtensionData.Browser $tgtDS = Get-Datastore -Name $DatastoreName $spec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec # Create folder for new VM $fileMgr.MakeDirectory("[$DatastoreName] $CloneName",$dcMoRef,$false) # Copy vDisks $spec.Query = New-Object VMware.Vim.VmDiskFileQuery $qResult = $srcDSBrowser.SearchDatastore("[$srcDSName] $MasterName",$spec) if($qResult.File){ $qResult.File | %{ $srcPath = "$($qResult.FolderPath)/$($_.Path)" $tgtPath = $srcPath.Replace($srcDSName,$DatastoreName).Replace($MasterName,$CloneName) $hd = $vm.ExtensionData.Config.Hardware.Device | where {$_ -is [VMware.Vim.VirtualDisk] -and $_.Backing.FileName -eq $srcPath} $controller = $vm.ExtensionData.Config.Hardware.Device | where {$_.Key -eq $hd.ControllerKey} $vDiskSpec = New-Object VMware.Vim.VirtualDiskSpec $vDiskSpec.adapterType = &{ if($controller -is [VMware.Vim.VirtualLsiLogicController] -or $controller -is [VMware.Vim.VirtualLsiLogicSASController] -or $controller -is [VMware.Vim.ParaVirtualSCSIController]){ [VMware.Vim.VirtualDiskAdapterType]::lsiLogic } elseif($controller -is [VMware.Vim.VirtualBusLogicController]){ [VMware.Vim.VirtualDiskAdapterType]::busLogic } else{ [VMware.Vim.VirtualDiskAdapterType]::ide } } $vDiskSpec.diskType = &{ if($hd.Backing.eagerlyScrub){[VMware.Vim.VirtualDiskType]::eagerZeroedThick} elseif($hd.Backing.thinProvisioned){[VMware.Vim.VirtualDiskType]::thin} else{[VMware.Vim.VirtualDiskType]::thick}} $vdkMgr.CopyVirtualDisk($srcPath,$dcMoRef,$tgtPath,$dcMoRef,$vDiskSpec,$null) } } # Copy other VM files $dsDestination = New-PSDrive -Location $tgtDS -Name dest -PSProvider VimDatastore -Root '\' Get-ChildItem -Path "vmstore:\ha-datacenter\$srcDSName\$MasterName" | where {"vmdk","log","vmsd" -notcontains $_.Name.Split('.')[1]} | %{ # The copy is done in a foreach loop, to bypass a Copy-DatastoreItem bug with pipeline Copy-DatastoreItem -Item $_ ` -Destination ("dest:\$CloneName\" + $_.Name.Replace($MasterName,$CloneName)) -Confirm:$false } # Update VMX file Copy-DatastoreItem -Item "dest:\$CloneName\$($CloneName).vmx" -Destination $env:Temp\t.vmx $text = Get-Content -Path $env:Temp\t.vmx | %{$_.Replace($MasterName,$CloneName)} $text | Set-Content -Path $env:Temp\t.vmx Copy-DatastoreItem -Item $env:Temp\t.vmx -Destination "dest:\$CloneName\$($CloneName).vmx" -Confirm:$false # Update VMXF file Copy-DatastoreItem -Item "dest:\$CloneName\$($CloneName).vmxf" -Destination $env:Temp\t.vmxf $text = Get-Content -Path $env:Temp\t.vmxf | %{$_.Replace($MasterName,$CloneName)} $text | Set-Content -Path $env:Temp\t.vmxf Copy-DatastoreItem -Item $env:Temp\t.vmxf -Destination "dest:\$CloneName\$($CloneName).vmxf" -Confirm:$false if($Register){ New-VM -VMFilePath "[$DatastoreName] $CloneName\$($CloneName).vmx" | Out-Null if($PowerOn){ Start-VM -VM $CloneName -ErrorAction SilentlyContinue Get-VMQuestion -VM $CloneName | Set-VMQuestion -DefaultOption -Confirm:$false } } Remove-PSDrive -Name dest -Confirm:$false } |
Annotations
Line 24-30: The definition of the parameters for the function. Note that the function doesn’t use any of the advanced features, nor pipeline input. Since I use this function in my homelab, I found functionality more important than good coding style 😉 And the function name itself doesn’t comply with the approved verbs either.
Line 33-36: The function tests if the master VM is powered on or has snapshots. In that case the function returns immediately. It seems that copying a VM that has snapshots, can have an impact on the functioning of the cloned VM (see KB1000936).
Line 38-40: Get the managers via the ServiceInstance. Remember, no vCenter !
Line 42-46: Retrieve and/or compose the values that are required later in the function.
Line 50: Create the datastore folder that will hold the cloned VM
Line 47,53-54: The function uses the HostDatastoreBrowser to find all VMDK files.
Line 59-80: These lines make sure that the VirtualDisk type of each vDisk in the master VM, is also used in the corresponding vDisk in the cloned VM.
Line 81: The actual copy of the vDisk from the master to the clone.
Line 88: There is no need to copy the .vmdk (already done), the .log and the .vmsd (no snapshots remember) files to the clone.
Line 89: It seems that some PowerCLI builds have a problem when the Item object is passed over the pipeline to the Copy-DatastoreItem cmdlet. To bypass this problem, the function uses a ForEach loop and specifies the Item object on the cmdlet explicitly.
Line 95,101: To update the .vmx and .vmxf files, the function copies both files to a Temp folder on the station where the script is running. This is to bypass the “cannot open file” problem when updating a file in place.
Line 110: The ESXi server will see that the clone has been moved or copied when we try to start the VM. The Set-VMQuestion selects the “copied” answer, which is the default.
Sample Usage
Just
The function is quite easy to use.
1 |
Clone-MasterVM -MasterName W2K12Core_Base -CloneName Server1 -DatastoreName datastore2 -Register -PowerOn |
Note that the cloned VM will not be customised !
It will have the same configuration as the master VM from which it was cloned. The guest OS customisation, as can be done through the OSCustomizationSpec when there is a vCenter, is the subject for another post in this series.
What I normally do is to prepare my master with everything that I want to have on the cloned VMs. And then I run sysprep with the Shutdown option.
When the cloned VM is started, it will start with the mini-setup.
Enjoy !
Andrey Agra
Hi, This script is very useful to me but doesn’t work. When I’m trying to use it, this error below happens in line 86:
Get-ChildItem : Cannot find path ‘\LastConnectedVCenterServer\ha-datacenter\DataStore2\Win10 10 Client’ because it does not exist.
At line:86 char:3
+ Get-ChildItem -Path “vmstore:\ha-datacenter\$srcDSName\$MasterName” …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (\LastConnectedV…ndows 10 Client:String) [Get-ChildItem], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand
Do you have any idea?
LucD
Hi,
I would need to know a bit more about the environment in which you are trying to run the script.
Feel free to use the Contact Form to send me some more information (and some eventual screenshots).
From the error I get the impression you are connected to a vCenter, while the script assumes you are connected directly to an ESXi node.
cloudinmyhead
This worked perfect ! I have tried others code but they didn’t work for me. This was perfect.
I did have to look up how to use the switch since I don’t write functions yet in PS.
For others like myself add these to the end of your “Clone-MasterVM” command to register the VM, and or power it on.
-Register -PowerOn
YVS Kumar
Hi LucD – I’m trying to run this script on esxi 6.0, does this script works on 6.0 , do you have any latest script for 6.0.
Exception calling “MakeDirectory” with “3” argument(s): “Current license or ESXi version prohibits execution of the requested
operation.”
LucD
From the error message it looks as if you might be using a so-called “free ESXi” license. This license has some limitations, one of them being that you cannot make changes to the environment with PowerCLI. You can only use PowerCLI in “read mode” I’m afraid.
jvschelt
scripted this to run from a ESXi box, see
https://communities.vmware.com/message/2470141?tstart=0#2470141
LucD
Although I normally don’t condone running scripts in the ESXi BusyBox, this is a nice port and handy for a lab environment.
Deepak George
Do you any possible ways to do a storage vMotion of VM from one datastore to another without using the vCenter.?
William
I love the idea of the script, however, when I try and execute it nothing happens. No errors, no warnings, no activity in the VMware console. This is what I’m using to execute:
.\vmclone.ps1 Clone-MasterVM -MasterName W2K12Core_Base -CloneName Server1 -DatastoreName datastore2 -Register -PowerOn
Help is greatly appreciated
LucD
Hi William, the script is a function, did you call the function in that .ps1 file ?
There is a sample call of the function near the end of this post.
JDShots
How do I specify the server to perform this procedure in, and the credentials to do so?
LucD
@JDShots, you will need to be connected to an ESXi server.
You can do that with
Connect-VIServer -Server esx1.lucd.info -User root -Password password
Of course you will need to change the hostname and the credentials to reflect your environment.
JDShots
Is there a way to download your script?
LucD
Hi JDShots, when you double-click in the code block, all the code will be highlighted. You should then be able to do a copy/paste from the browser to your editor.
Would you prefer to have all the scripts available as file downloads as well ?
akamac
Hello, Luc! Thanks for the post. I’m trying to write my own version with hot cloning capabilities.
My question is about the VM config changes between the snapshots are created. Where can I track these changes? vmsd file contains only hard disk drive information. TIA
LucD
Hi, you could use the events as I described in Events – Part 3 : Auditing VM device changes
Frank
Thanks for the script. Is this also a supported method for cloning a VM in a production environment? Thanks!
admin
Hi Frank, I’m not sure that this method is officially supported by VMware.
Although it uses public API and does an edit of the VMX file, I’m not sure what the official stance of VMware on this is.
It would be best to ask for confirmation from VMware Support before you consider using this in a production environment.
Sanshis
useful script for work in Home!!
just realize Service Instance is bigger thing thn i thought if all its contents can be use with standalone esxi as it can be use with vCenter; Can it be???
–Regards
Sanshis
admin
Thanks.
The ServiceInstance is indeed available on any vSphere server, beit a vCenter or an ESXi server.
Note that some properties will not be available for all vSphere servers. For example the OptionManager.
Francois-Xavier Cat
Awesome script LucD, this is so useful! will give it a try this week
LucD
Thanks, I hope to post some more articles in this series in the coming weeks.