VMX Raiders Revisited

This afternoon news of a new (and interesting) blog post from Julian Wood hit the Twitter-verse.

He took part from the logic of my Raiders of the Lost VMX script and used the relatively new VMFilePath parameter of the New-VM cmdlet, to shorten my script considerable. Well done, great job !

But I was intrigued by a remark in Julian’s post. He was obviously not too fond of using MoRefs in a script. But in Julian’s final script I still saw a Get-View cmdlet and the use of the HostDatastoreBrowser. Remembering a remark from one of my co-authors on the PowerCLI book, that the Get-View cmdlet would disappear, once the PowerCLI snapin would be sufficiently evolved, I decided to tackle Julian’s script and give it another simplification treatment and make it more PowerCLI 😀

The script

Annotations

Line 9: I added a fail-safe, where I avoid that .vmx files that belong to already registered VMs, would be used again. This is done by adding the VMX-path of all registered VMs on the datastore to a hash table.

Line 10: The VMX-path is retrieved for each registered VM and added to the hash table

Line 13: To avoid the use of the HostDatastoreBrowser object, and make it more pure PowerCLI, I use the datastore provider that comes with PowerCLI. The big disadvantage of this is that the datastore provider is way slower than the HostDatastoreBrowser methods.

Line 14-15: With datastore provider the script retrieves all .vmx files that are not in a snapshot folder and that do not belong to a registered VM

Line 16: Remove the datastore provider

Line 19-21: Register all .vmx files that were found

Enjoy.

37 Comments

    Frands

    Hello LucD

    Thanks for the script.
    It worked like a charm on my home lab. (vCenter 6.7 and ESXi 6.7 U2)
    The only ting I changed was the Cluster and Get-Cluster.
    I switched that out with Datacenter and Get-Datacenter since I am not using clusters on my lab.

    Thanks

    michael

    Thanks LucD, You the man!

    I had the issue described here : https://kb.vmware.com/s/article/2146168

    After vCenter reboot, VMs re-added to inventory using this code showed up as “ds:///vmfs/volumes/datastore/vm_name”.

    The KB article recommends naming the VM is all cases when adding to inventory using VPX file path.

    I added a simple line to set the VM name based on the VPX file name which works in our environment.

    foreach($VMXFile in $unregistered) {
    $VMName = ($VMXFile.name).split(“.”)[0]
    New-VM -VMFilePath $VMXFile.DatastoreFullPath -VMHost $ESXHost -Location $VMFolder -name $VMName -RunAsync
    }

      LucD

      Thanks for sharing that Michael

    Patrick GINHOUX

    Hi LucD,

    Because I have to move a lot of VMs from an old Datastore and old ESXi hosts to new ones, I searched for a PowerCLI script to register all the VMs after all the files be transferred to the new datastore.

    So the one here, and it has registered all the VMs successfully in the target folder and for the VM host specified in the script.

    But it has also registered the existing 2 VMs and also the vCSA a second time. This is probably normal because these 2 VMs and the vCSA don’t belong to the same folder.
    Please correct me if I’m wrong.

    There is another issue for new registered VM with a strange name : ‘prdfbch’, for which there is no .vmx file on the new DS.

    Hopefully, I was able to remove all of the above from the inventory.

    At least, in the target folder, I can see 2 VMs with the ‘inaccessible’ state but with a VM name that looks like a Datastore path :

    ds:///vmfs/volumes/5c7fe08a-13ad5e16-2031-e4434b2a3be4/Tempora (1)
    ds:///vmfs/volumes/5c7fe08a-13ad5e16-2031-e4434b2a3be4/Temporary Tes

    The Get-VM -Name * command doesn’t list these 2 strange VMs, so that I don’t know how to remove it from the inventory.

    Do you have an idea why they have been created like that and then how to remove it ?

    Thanks in advance for the help

    Regards
    Patrick

      LucD

      The script finds all .VMX files and then checks the vCenter to which you are connected if the corresponding VM is registered there.
      If it can’t find the VM, it register the VMX file.
      Could it be that the same VMX appears multiple times on the datastores?

        Patrick GINHOUX

        Thanks for you answer.

        Now I’m quite sure that these VMX didn’t exist multiple times on the datastore but I have no way to prove it.

        The only thing I can add, is that 2 VMs were active when I started the script.
        If I examine the DS and do a search now for one of the VM with “niuadm1.vmx”, the file browser returns these files :
        vm-pcoe-niuadm1.vmx~
        vm-pcoe-niuadm1.vmx.lck
        vm-pcoe-niuadm1.vmx
        Could it be possible that the script failed to identify the .vmx for the above VM ?

        Otherwise, any idea about the strange VMs ?

    Simon Lee

    Hi,

    Have you come across an issue with using this script when adding over 400 VMs to the inventory?
    We have in the past used a similar script without issue, however on vCenter 6.7 when the script adds the VMs it causes the vCenter services to crash out. Is this due to the -RunAsSync?
    We are using our script to run a DR test.

    Thanks,
    Simon

      LucD

      Hi Simon,
      No, haven’t seen that issue I’m afraid.

      Would you mind sending me some more info on what your script looks like?
      And which errors you are getting?

      You can use the Contact Form to send attachments if you prefer.

        Simon Lee

        Hi,

        The issue wasn’t to do with the script. It was to do with a content library item being present on the replicated VM. The item was an ISO mounted to the CD-ROM, the total length of the path (including the VMware UIDs, etc) was over 255 characters, which was causing the vCenter services to completely crash.

        Thanks anyway 🙂

        Simon

    albvar

    I converted this to a function, see below. It’s very slow and would not use in a production environment with a lot of assets under management. Currently seeking a way to do this same functionality but getting rid of the slow points (Get-ChildItem -Recurse) and replacing them with calls to Get-View.

    function Find-UnregisteredVM
    {
    [CmdLetBinding()]
    param
    (
    [object[]]
    [Parameter(Mandatory = $true)]
    $Cluster,

    [object[]]
    $DataStore = [string]::Empty,

    [ValidateNotNullOrEmpty()]
    [string]
    $MatchPattern = ‘*.vmx’
    )

    begin
    {
    $stopWatch = [Diagnostics.Stopwatch]::StartNew()
    $startMsg = (‘{0} Start: {1}’ -f (Get-Date), $MyInvocation.MyCommand, $env:USERNAME)
    Add-Type -AssemblyName VMware.Vim
    Add-Type -AssemblyName VMware.VimAutomation.ViCore.Impl
    }

    process
    {
    foreach ($clustr in $Cluster)
    {
    if (-not ($clustr -is [VMware.VimAutomation.Types.Cluster]))
    {
    $clustr = Get-Cluster -Name $clustr
    }

    if (-NOT $PSBoundParameters.DataStore)
    {
    $esxiHost = $null
    $esxiHost = $clustr | Get-VMHost | Where-Object ConnectionState -EQ Connected | Select-Object -First 1

    $DataStore = @($esxiHost | Get-DataStore)
    Write-Verbose -Message (‘Processing {0} host selected {1}’ -f $clustr, $esxiHost)
    Write-Verbose -Message (‘Datastore param not specified, scanning all[{0}] DataStores for cluster {1}’ -f $DataStore.Count, $clustr)
    }

    foreach ($store in $DataStore)
    {
    if (-not ($store -is [VMware.VimAutomation.Types.DatastoreManagement.DatastoreItem]))
    {
    $store = Get-Datastore -Name $store
    }

    Write-Verbose -Message (‘Processing datastore {0}’ -f $store)

    try
    {
    $null = New-PSDrive -Name datastore -Location $store -PSProvider VimDatastore -Root ‘\’ -ErrorAction Stop
    try
    {
    $unregistered = $null
    $unregistered = @(Get-ChildItem -Path datastore: -Recurse -ErrorAction Stop)
    Write-Verbose -Message (‘A total of {0} files were found under {1}’ -f $unregistered.Count, $store)
    $unregistered = $unregistered | Where-Object -FilterScript {
    $_.FolderPath -notmatch ‘.snapshot’ -and $_.Name -like $MatchPattern
    }
    Write-Verbose -Message (‘A total of {0} items found matching our “{1}” filter’ -f $unregistered.Count, $MatchPattern)

    $unregistered
    }
    catch
    {
    $msg = (‘Unable to recursive list contents of my datastore {0} {1}’ -f $store, $_)
    Write-Error $msg
    }
    }
    catch
    {
    $msg = (‘Unable to create a VimDataStore for {0} {1}’ -f $store, $_)
    Write-Error $msg
    }
    finally
    {
    if (Test-Path -Path datastore:)
    {
    Remove-PSDrive -Name datastore
    }
    }
    }#foreach DataStore
    }#foreach Cluster
    }#process
    End
    {
    $endMsg = (‘End: {0} {1} execution time: {2} seconds’ -f $MyInvocation.MyCommand,
    $env:USERNAME, $stopWatch.Elapsed.TotalSeconds)

    Write-Verbose -Message ($startMsg)
    Write-Verbose -Message (‘{0} {1}’ -f (Get-Date), $endMsg)
    $stopWatch.Stop()
    }#End
    }

    Christoph

    I’m quite new to scripting and maybe I made a mistake how I used it. It took me a while to figure it out but I had to make small change to make that script work in my invironment to find unregistered vms.

    Change in Line 15:

    From:
    where {$_.FolderPath -notmatch “.snapshot” -and $_.Name -like “*.vmx” -and !$registered.ContainsKey($_.Name)})

    To:
    where {$_.FolderPath -notmatch “.snapshot” -and $_.Name -like “*.vmx” -and !$registered.ContainsKey($_.DatastoreFullPath)})

    I made some more changes because I just need to know unregistered vms, I don’t want to register them.

    JHoSee

    Thank you so much for this script! It helped me recover over 500 VMs lost from inventory. Much appreciated…

    Almero

    Hi Guys , I need to register certain VMs to DR Tests from CSV. Testing script before I run in real DR .

    I am having issues with VMFilePath varaible .

    My CSV DRVMs.csv >
    [LOCAL] TESTvm1/TESTvm1.vmx
    [LOCAL] TESTvm2/TESTvm2.vmx
    [FAKESAN1] TESTvm_onSAN/TESTvm_onSAN.vmx

    My Ps1 >
    add-pssnapin VMware.VimAutomation.Core

    $ESXHost = “192.168.22.101”
    $VMXFiles = Import-Csv “C:\Elmo\Ps\Shop\DR\DRVMs.csv”

    Connect-VIServer $ESXHost

    ForEach ($VMXFile in $VMXFiles)

    {
    New-VM -VMFilePath $VMXFile -vmhost $ESXHost
    }

    But I get the following Error > I am missing something obvious here 🙂

    New-VM : 2014/08/22 10:28:53 AM New-VM ‘@{[LOCAL] TESTvm1/TESTvm1.vmx=[LOCAL] TESTvm2/TESTvm2.vmx}’ is not valid datastore path.
    At C:\ELMO\PS\Shop\DR\ELMO Start DR VMs.ps1:27 char:16
    + New-VM <<<< -VMFilePath $VMXFile -vmhost $ESXHost
    + CategoryInfo : InvalidArgument: (:) [New-VM], InvalidArgument
    + FullyQualifiedErrorId : Common_DatastorePathHelper_ValidateDatastorePath_InvalidPath,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewVM

      LucD

      Hi Almero,
      You seem to be passing 2 VMX files on the VMFilePath parameter.
      Can you check your CSV file ?

      Almero

      LucD , I got it working , sorry for hassle .

      Added VMXFILE to top of my CSV .
      Added this line , even though I still dont understand , but it all works now .

      $vms= $VMXFile.VMXFile
      New-VM -VMFilePath $vms -vmhost $ESXHost

        LucD

        Great, glad you got it working

    ShaharF

    hi
    i’ve change the datastore (2nd line) to select all datastores.
    if it works – maybe give it as an option?
    $Datastores = @(Get-Cluster -Name “Cloud Cluster” | Get-VMHost | Get-Datastore | where {$_.Type -eq “VMFS” -or “NFS”} | % {$_.Name})

    ShaharF

    kris

    Hi Luc
    How would you take input from csv file when csv file looks like this—-Thanks for your help in advance.
    vm,vmx
    0609455774-WinXPx32,[VM Datastore2] 0609455774-WinXPx32_1/0609455774-WinXPx32.vmx
    0187541347-Win7,[VM Datastore2] 0187541347-Win7_1/0187541347-Win7.vmx
    1240043273-IIS,[VM Datastore2] 1240043273-IIS_1/1240043273-IIS.vmx
    0663697505-win2k8mssql,[VM Datastore2] 0663697505-win2k8mssql_1/0663697505-win2k8mssql.vmx
    0504883356-Win7,[VM Datastore2] 0504883356-Win7_1/0504883356-Win7.vmx
    0692920522-IIS,[VM Datastore2] 0692920522-IIS_1/0692920522-IIS.vmx
    1612413179-Dev,[VM Datastore2] 1612413179-Dev_1/1612413179-Dev.vmx
    0829482277-mssql2005,[VM Datastore2] 0829482277-mssql2005_1/0829482277-mssql2005.vmx
    1443716454-win2k8mssql,[VM Datastore2] 1443716454-win2k8mssql_1/1443716454-win2k8mssql.vmx

      LucD

      Hi Kris, not sure I understand the question.
      This script looks for unregistered VMs, by searching for VMX files on the datastore(s).
      What is in the CSV file, a list of VMX files of unregistered VMs ?

        Kris

        Luc
        My question was that I have list of VM’s in a csv file and I want to extract vmx file path information for these specific vm’s in csvfile not for the whole datacenter.
        What I want to do is something like
        $vm= import-csv “c:\temp\vm1.csv -useculture | % {
        $vms=get-vm $_.”name”
        foreach($vm in $vms)
        {Get-View -ViewType virtualmachine | % { $_.Config.Files.VmPathName }}

        It does not read the vm name from .csv file and, I was wondering if you could help me in sorting this out.

    saito

    when try to run script or create a new vm I get error:

    “New-VM The specified path is not correct. Element ‘inMaintenanceMode’ doesn’t exist.”

    I have tried many things but no chance.

    any idea?

      LucD

      Hi Saito, could it be that the ESXi host or the datastore is placed in maintenance ?
      It could help to debug if I could see the content of the $VMXFile variable in the last foreach loop.
      Try changing that part like this, and let me know what the content of $VMXFile looks like.

      ....
      foreach($VMXFile in $unregistered) {
      $VMXFile
      New-VM -VMFilePath $VMXFile.DatastoreFullPath -VMHost $ESXHost -Location $VMFolder -RunAsync
      }

        murat

        hi lucD,
        i have same problem as i post before comunity forum. SR didn\’t solve my problem.
        PowerCLI C:\\> new-vm -VMHost desxi01.xyz.local
        cmdlet New-VM at command pipeline position 1
        Supply values for the following parameters:
        Name: sdfsdfsd
        new-vm : 2.02.2017 09:46:52 New-VM The specified path is not correct. Element \’inMaintenanceMode\’ doesn\’t exist.
        At line:1 char:1
        + new-vm -VMHost desxi01.xyz.local
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : NotSpecified: (:) [New-VM], ViError
        + FullyQualifiedErrorId : Client20_MoServiceImpl_GetViNetView_ViError,VMware.VimAutomation.ViCore.Cmdlets.Comman
        ds.NewVM

          erkan

          Hello murat,
          did you find a solution ?

      erkan

      hello saito,
      i am having the same problem.
      and cant find any solution.
      do you remember how had you solve this problem ?

    Randy

    I’m attempting migrate VMs (remove, re-register) from one vCenter to another (both share the same, multiple datastores) by iterating an array of VM names I supply using $arrVMNames = Get-Content (“C:\scripts\vmnames.txt”).

    The problem I’m having is saving the vmx file path to a variable and using it: New-VM -VMFilePath $VMPaths -VMHost $esxhost

    Can you elaborate on how to save & reuse the file path correctly in this manner?

    Thanks!

    djr

    Hi, is there a way to randomize the host the vm’s are registered on. in your example they all get registered on the first node? unless I am mistaking… We have an 8 node cluster, so it would be great if it could register to any of the 8 nodes.

      LucD

      Hi djr, everything is poshible.
      Seriously, with the Get-Random cmdlet you can easily accomplish that.
      The code becomes something like this

      $Cluster = "MyCluster"
      $Datastores = "MyDS"
      $VMFolder = "MyFolder"

      $clusterObj = Get-Cluster -Name $Cluster

      foreach($Datastore in Get-Datastore $Datastores) {
      # Collect .vmx paths of registered VMs on the datastore
      $registered = @{}
      Get-VM -Datastore $Datastore | %{$_.Extensiondata.LayoutEx.File | where {$_.Name -like "*.vmx"} | %{$registered.Add($_.Name,$true)}}

      # Set up Search for .VMX Files in Datastore
      New-PSDrive -Name TgtDS -Location $Datastore -PSProvider VimDatastore -Root '\' | Out-Null
      $unregistered = @(Get-ChildItem -Path TgtDS: -Recurse | `
      where {$_.FolderPath -notmatch ".snapshot" -and $_.Name -like "*.vmx" -and !$registered.ContainsKey($_.Name)})
      Remove-PSDrive -Name TgtDS

      #Register all .vmx Files as VMs on the datastore
      foreach($VMXFile in $unregistered) {
      $ESXHost = Get-VMHost -Location $clusterObj | Get-Random
      New-VM -VMFilePath $VMXFile.DatastoreFullPath -VMHost $ESXHost -Location $VMFolder -RunAsync
      }
      }

      See also the 3th tip in my article in PowershellMagazine.

    Brent McDonald

    I get this error when adding VMs to a folder called _Templates. Each of my data centers have a folder called this, and they are both managed by the same vCenter. Is there away to work around this?

    New-VM : 3/16/2012 2:09:49 PM New-VM The specified parameter ‘Location’ expects a single value, but your name criteria ‘_Template
    s’ corresponds to multiple values.
    At C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\scripts\_add.ps1:20 char:12
    + New-VM <<<< -VMFilePath $VMXFile.DatastoreFullPath -VMHost $ESXHost -Location $VMFolder -RunAsync
    + CategoryInfo : InvalidResult: (System.Collecti…dObjectInterop]:List`1) [New-VM], VimException
    + FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_MoreResultsThanExpected,VMware.VimAutomation.ViCore.Cmdlets.Commands
    .NewVM

      LucD

      Hi Brent,
      Yes, you can change line 3 into

      $VMFolder = Get-Folder -Name "MyFolder" -Location (Get-Datacenter -Name MyDC)

      That way, you will avoid having an array, but you will have the single folder with that name in that specific datacenter.

    Conrad

    LucD, I am having an issue with this script…

    The names that go into the registered are in the form
    [Volume1] vmname/vmname.vmx

    When later that is .containskey against the names that come out of the $unregistered, it doesn’t find a match where it should… the names it is trying to match the above to look like
    vmname.vmx

    Which straight up
    [Volume1] vmname/vmname.vmx -ne vmname.vmx

    I have tried wildcards…but .containskey doesn’t seem to like that very much

    I might be missing something but I have checked all the possibilities I can think of…
    Thoughts?

      Chris Brinkley

      Fixed by changing line 12 to this:
      Get-VM -Datastore $Datastore | %{$_.Extensiondata.LayoutEx.File | where {$_.Name -like “*.vmx”} | %{$registered.Add($_.Name.split(‘/’)[-1],$true)}}

      It’s comparing apples and oranges, I just changed apples to oranges by adding this bit:

      .split(‘/’)[-1]

    kenny

    This is fantastic!

    Doing migrations with data centers and srdf etc, this will work wonders!

    Thanks very much Luc!

    Julian Wood

    Luc, as always rising to the challenge!

    I didn’t even think of using the new functionality to “map” a drive to a datastore and then search within it.

    This is a fantastic addition and far simpler than delving into Get-View!

    Having MoRefs and Get-View is great to be able to access the entire API but it’s not really native PowerCLI and can be confusing. As the cmdlets are developed, I suppose more API functionality will be added to PowerCLI which just makes it easier to understand.

    PowerCLI just keeps getting better!

    Thanks

      LucD

      @Julian, thanks but it was you that used the VMFilePath parameter first 🙂

      Btw the datastore provider is terribly slow compared to the HostDatastoreBrowser search.
      Will be better in the next build I hope.

    Shawn Cannon

    I just tried to run this on my NFS Datastore and it would not work. Do I still need to use your older VMX method to import from NFS Datastores?

      LucD

      Hi Shawn, something apparently went wrong during the copy of the code.
      I corrected that and tested against an NFS datastore. It seems to work correctly now.
      Let me know when you have the time to test on your side, if you also see success.

      The original Raiders script still has it’s value.
      It offers way more possibilities than this short script.

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.