Orphaned files and folders – Spring cleaning

In our PowerCLI book we presented a Delete-Harddisk function in Chapter 7.

One of our readers asked if that function could be used to remove orphaned VMDK files from one or more datastores. Now unfortunately that is not the case since the function we presented in chapter 7 uses the ReconfigVM_Task method to remove the harddisk.

In the PowerCLI Community there are some thread that provide scripts to report on orphaned VMDK files, but most of these are quite old.

So I decided to write a new script that would report on orphaned folders and VMDK files and that would have an option to remove these folders and files.

Update March 19th 2013

The following changes were made to the function

  • Fixed the problem where a VMDK in the root of a datastore, which you shouldn’t have of course, would erase the complete datastore content (ouch). Thanks Olivier
  • Added WhatIf functionality. You can now check what the Delete switch will do before actually removing files and folders.
  • Added details for lone VMDK. See the properties of the returned objects.

Update May 22nd 2011

Two corrections in this new version of the script.

  • NiTRo reported a problem with guests that had only a .vmdk, and not their .vmx, file located on the datastore(s) to be cleaned
  • The COS became a virtual machine with ESX 4.x and thus has a .vmdk file as well. Although the COS files should be located on a local datastore, the script now ignores COS .vmdk files when it encounters them on the datastore(s) to be cleaned.

The Script

Annotations

Line 22, 91, 110, 116: These handle the WhatIf switch

Line 26: The function accepts datastore by name or as a DatastoreImpl object. Hence the PSObject type. There has to be a Datastore argument !

Line 39-41: Cheap OBN implementation.

Line 42: The test makes sure only VMFS datastores that are shared are handled.

Line 43-48: The function creates a hash table with all the files that are used by the virtual machines on the datastore. The $fldList hash table contains the folders and the $hdList hash table contains all the disk related files (they have the diskDescriptor or the diskExtent type).

Line 49-54: The Get-Template cmdlet doesn’t have a Datastore parameter, that’s why this block uses a different method of determining if there are Templates on a specific datastore.

Line 58-75: The parameters for the SearchDatastoreSubFolders method are prepared and at the end the method is called.

Line 77-105: The returned results from the SearchDatastoreSubFolders method are analysed.

Line 87,103: The Extents property will contain all related filenames of the VMDK file. For example, a file called MyVM.vmdk will have an extent called MyVM-flat.vmdk.

Line 89-93: If the Delete switch is used, the script tests if the WhatIf switch was also specified. If not, the VMDK file is deleted.

Line 109: This condition handles the case when there is an orphaned VMDK file in the root of the datastore.

Line 110: For an orphaned VMDK in the root of the datastore, the DeleteFile method is used.

Line 118: For a folder with orphaned VMDK files, the DeleteDatastoreFile method is used

Sample runs

The function will only scan shared VMFS datastores for orphaned folders and files.

This call will display all found folders and files and will not delete them.

demo2

When you add the Delete switch, the function will return the same information, but the folders and files will be deleted.

When you combine the Delete switch with the WhatIf switch, not files or folders will be deleted. But the function will show what actions will be taken without

demo1

You can specify more than one value with the Datastore parameter, the function will investigate all the datastores you specified.
You can pass the datastores by name

or by object

And the function can of course also be used in a pipeline construction

Make absolutely sure that you know what you are doing before using the Delete switch.

Use the WhatIf switch before actually deleting anything !

Enjoy !

133 Comments

    Syed Quadri

    Hi @LucD,

    Thank you for this amazing function. I ran this against one of our datastores without the delete switch of course and it presented a bunch of files that are actually associated with some of our templates. Does this not work with templates?

    Thanks,
    Syed

      LucD

      It should handle Templates.
      In line 49 the script gets all the Templates on the Datastore, and marks all the VMDK in the Template as “used”.

      You would need to check which VMDK the Template is actually pointing to, and which VMDK are erroneously included in the list.
      Can you share an example with some these VMDK (and include the full path)?

        Syed Quadri

        Hi LucD,

        Thanks for your reply. So I found that some (Maybe all) of these templates that are being erroneously included in the list are located within a folder called “Template” in the datastore.

        See the example below.

        Name : AVALANCHE-4.45-TEMPLATE.vmdk
        Extents : [ISOs-Templates] Template/AVALANCHE-4.45-TEMPLATE/AVALANCHE-4.45-TEMPLATE-flat.vmdk
        Thin : True
        Folder : [ISOs-Templates] Template/AVALANCHE-4.45-TEMPLATE/
        Size : 1829765120
        CapacityKB : 2000000

        What if: Performing the operation “Remove Folder” on target “[ISOs-Templates] Template/AVALANCHE-4.45-TEMPLATE/”.

        Another question I have is if there’s a way to exclude the iso image files from this list as we store a lot of iso images in our datastore.

        Thanks,
        Syed

          LucD

          I suspect that the issue is due to the fact that these VMDK are not in a standard location.
          The VMDK for a VM are normally located in a folder [datastore] .
          In your case, the VMDK are located in a deeper folder, that might be the issue.
          I’ll have to do some testing.

    Arthur

    I had to add if clause after these lines:
    elseif ($folder.File | where { “cos.vmdk”, “esxconsole.vmdk” -notcontains $_.Path }) {
    $folder.File | % {

    if clause:
    if (!$hdList.ContainsKey($folder.FolderPath + $_.Path)) { … }

    because some used vmdk files were included …

      LucD

      That would mean there are valid VMDK files in folders that are not found via Get-VM or Get-Template.
      Do you have VMDK in separate folders, outside the VM’s folder?

    Karl

    Hi, I am really liking this script along with many others you have put together. We have a platform that’s in a bit of a state following some erratic Zerto edployments and as a result have a lot of “zombie” disks being reported by the script. I’ve been bashing my head against the table trying to get the findings to output to gridview or even a csv file. Are you able to suggest a solution?

    What is interesting is the difference in files found by the script compared to the vHealth output from an RVTools export (as in RVTools reports very few in comparison).

    Thanks in advance 🙂

      LucD

      Since this is a function you could just “pipe” the output to Export-Csv or Out-GridView.
      With the PassThru switch you can even do both in 1 line.

      Remove-OrphanedData -Datastore DS1 | Out-GridView -PassThru | Export-Csv -Path .\report.csv -NoTypeInformation -UseCulture

      To compare 2 files I would advise having a look at the Compare-Worksheet function included in the ImportExcel module.

        Karl

        That’s awesome, thank you, I was looking at a way more complex ways of doing deep in the bowels of the script/module.

        I wasn’t aware of the PassThru option, I’ll be making use of that in some other scripts, very handy, thank you.

    sekhar

    Hi LucD,

    How to remove vmfile/folders in datastore. I removed VM from inventory and now i want to delete it from datastore. how can i do it with powercli/powershell?

    Please help me out.

    Thanks,
    Sekhar

      LucD

      The function has a Delete switch, which will delete the discovered orphaned files.
      But make sure that the files the function discovers are orphaned (first run without the Delete switch).

    Luca

    Can anyone explain how to use the script? do you need to build a module or install it somehow so you can run the command?

      LucD

      Hi,

      The PS engine needs to “know” the function before you can call it.
      The easiest way IMHO is to store the function in a .ps1 file and then dot-source that .ps1 file.

      • Copy the function code in a .ps1 file, for example Remove-OrphanedData.ps1
      • Save the file in the current directy
      • Dot-source the .ps1 file (Note the space between the 2 dots)

        . .\Remove-OrphanedData.ps1
      • You can now call the function as documented in the Sample Runs section in the post.

      I hope this helps.

    norman

    @LucD
    Hi Luc,

    The updated script does not support datastores with the same name.

    if(!$Datastore){
    $Datastore = Get-Datastore
    }
    else{
    $Datastore = $Datastore | %{
    if($_.GetType().Name -eq “string”){
    Get-Datastore -Name $_
    }
    else{
    $_
    }
    }
    }
    foreach($ds in $Datastore){
    if($ds.Type -eq “VMFS” -and $ds.ExtensionData.Summary.MultipleHostAccess){

      LucD

      Hi Norman,
      I’m not sure how you can have two datastores that have exactly the same name?
      Are these in different vCenters?
      Luc

        norman

        @LucD
        There are two datacenters in one vCenter. Each datacenter got exactly the same name datastore. Thanks much.

          LucD

          Hi Norman,
          Yes, that is indeed an issue.

          I could update the code to add an additional parameter like Datacenter.

          But an easier option, without needing to change the code, would be to pass a Datastore object to the code, instead of the Datastorename as a string.
          The function does accept pipeline input.

          For example:

          Get-Datastore | Remove-OrphanedData

          or


          Get-Datacenter -Name xyz | Get-Datastore | Remove-OrphanedData

          I hope that helps.

            norman

            Hi LucD,

            That works fine. Thanks a lot.

            norman

            Hi LucD,
            Sorry, it did not work. I tried: “Get-Datastore xxx | Remove-OrphanedData” and then tried “Get-Datacenter -Name xyz | Get-Datastore xxx | Remove-OrphanedData”. The result is all the vmdk for both Datacenters in the list.

            norman

            @LucD
            Hi LucD,
            I tried both. But after running the script,it list all vmdk in the datastore. And suggestion? Thanks much.

              LucD

              Hi Norman,
              I ran some more tests in my lab, but I seem to be unable to reproduce what you are seeing.

              When I run

              Get-Datacenter -Name xyz | Get-Datastore -Name xxx

              it returns exactly 1 datastore.

                norman

                Hi LucD,
                Yes. But it list all vmdk in the one datastore when you got 2 datastores with the same name in different datacenter. It works when distinct name of database in different datacenter. And suggestion? Thanks much.

                norman

                Hi LucD,

                if you have 2 same name datastores in your lab, you can get all the vmdk by running the script “Get-Datacenter -Name xyz | Get-Datastore -Name xxx”

                  LucD

                  I’m sorry, I can’t replicate what you are seeing.

                    norman

                    Hi LucD,

                    Could you please explain a bit about how you find the Orphaned disk in your script? The logic in your script.
                    Thanks much.

                      LucD

                      I think the Annotations in the post explain that.

                    norman

                    Hi LucD,

                    I think this following line is not necessary.

                    where {“cos.vmdk”,”esxconsole.vmdk” -notcontains $_.Path}

                      LucD

                      It is, otherwise those system process-related VMDKs will also be listed.
                      But feel free to leave that line out.

    Alona

    Hi Luc,

    I am trying to produce a similar outcome mostly around orphan files and your scrip helped me a great deal by showing a way to browse datastore files.

    I was trying to unravel the syntax and tried on a couple of environments however for some reason whenever I am attempting to call searchdatastoresubfolders task it returns error
    PowerCLI C:\> (get-view (get-datastore| select -first 1).id).browser.searchdatastoresubfolders()
    Method invocation failed because [VMware.Vim.ManagedObjectReference] does not contain a method named ‘searchdatastoresubfolders’.
    At line:1 char:1
    + (get-view (get-datastore| select -first 1).id).browser.searchdatastoresubfolders …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : MethodNotFound

    I tried searching for the error but was not able to find anything at all:(
    Would you be able to point me in the right direction please?

      Alona

      I figured it has to be a view of a view %_%
      (get-view (get-view -viobject “datastorename”).browser).SearchDatastoreSubFolders($RootPath, $searchSpec)

      I wish I could explain why…

    Jason

    “get-datastore | remove-orphaneddata” results in:

    Get-View : 6/26/2017 8:51:33 AM Get-View View with Id ‘FileManager-FileManager’ was not found on the server(s).
    At line:30 char:16
    + $fileMgr = Get-View FileManager
    + ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : ObjectNotFound: (:) [Get-View], VimException
    + FullyQualifiedErrorId : Core_GetView_WriteNotFoundError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInter
    op.GetVIView

    vSphere PowerCLI 6.3 Release 1 with ESXi 5.5.0, 4722766

    mkellerman

    Is there a way to Create a VM from these orphaned VMs? Would love for them to re-appear in vCenter in ‘Recovered’ folder, so other users can decide if they should truly be deleted or not?

      LucD

      You can use the script in my Raiders of the Lost VMX post for that.

    Frederik Bisback

    @LucD
    Hello Luc, This has solved my question. Nagios check is working.

    Frederik Bisback

    Dear Luc,
    Thanks already for the script. I’m using the script to create a NAGIOS check.
    But because of NetApp VMware backup I have always a folder with .snapshot on the datastore.
    Is there a possibility to exclude a folder?

    Thanks for the feedback.

    Regards,

    Frederik Bisback

      LucD

      Hi Frederik,
      Thanks.
      Interesting use case for the script.

      On line 77, you can try to filter out these .snapshot folders as follows:

      foreach($folder in ($searchResult | where{$_.FolderPath -notmatch '\.snapshot'})){

      Let me know if that works.
      Luc

        AndJul

        Hi LucD.
        Thanks for your script.
        I need to exclude some files for the search and delete. For example 00000*.vmdk or hbrdisk.RDID*.
        I’ve tried to modify the parameters and search information in the web but I don´t now how to do it.
        Can you help me to do it?
        Thanks for your greta job and regards.

          LucD

          You can add a filter condition to the file selection lines.
          Line 80

          if($file.Path -notmatch "00000.*vmdk|hbrdisk.RDID" -and !$hdList.ContainsKey($folder.FolderPath + $file.Path)){

          and line 97

          elseif($folder.File | where {$file.Path -notmatch "00000.*vmdk|hbrdisk.RDID" -and "cos.vmdk","esxconsole.vmdk" -notcontains $_.Path}){

            AndJul

            Hi LucD.
            Thank you very much for your help.
            After replace the lines that you suggest me and add Write-Host “Path: $($_.Path)” in the line 99 to view the output, hbr files continue showing.

            For example.
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6241101.93521559482246.vmdk
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6289430.212374729256072.vmdk
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6198356.148153045212617.vmdk
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6235009.274341275858148.vmdk
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6271975.267797771358014.vmdk
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6222648.258000086367830.vmdk
            Path: hbrdisk.RDID-6efe0398-37ce-438d-8fdb-221a3707f810.6265819.37192913259666.vmdk

            Snapshots too.

            Path: o_vApp1_VM1 (4bf6120c-9c1c-403f-af73-29a1e35e28a4)_1-000002.vmdk

            Regards.

              LucD

              I would need to see exactly what you changed.

                AndJul

                Here you have the modifications:

                foreach($folder in $searchResult){
                if($fldList.ContainsKey($folder.FolderPath.TrimEnd(‘/’))){
                foreach ($file in $folder.File){
                if($file.Path -notmatch “00000.*vmdk|hbrdisk.RDID” -and !$hdList.ContainsKey($folder.FolderPath + $file.Path)){
                New-Object PSObject -Property @{
                Folder = $folder.FolderPath
                Name = $file.Path
                SizeKB = $file.FileSize
                CapacityKB = $file.CapacityKB
                Thin = $file.Thin
                vcenter = $vcenter
                Extents = [string]::Join(‘,’,($file.DiskExtents))

                }
                if($Delete){
                If ($PSCmdlet.ShouldProcess(($folder.FolderPath + ” ” + $file.Path),”Remove VMDK”)){
                $dsBrowser.DeleteFile($folder.FolderPath + $file.Path )
                }
                }
                }
                }
                }
                elseif($folder.File | where {$file.Path -notmatch “00000.*vmdk|hbrdisk.RDID” -and “cos.vmdk”,”esxconsole.vmdk” -notcontains $_.Path}){
                $folder.File | %{
                Write-Host “Path: $($_.Path)”
                New-Object PSObject -Property @{
                Folder = $folder.FolderPath
                Name = $_.Path
                SizeKB = $_.FileSize
                CapacityKB = $_.CapacityKB
                Thin = $_.Thin
                vcenter = $vcenter
                Extents = [String]::Join(‘,’,($_.DiskExtents))
                }
                }

                Regards

                  LucD

                  The ElseIf is incorrect, my bad.
                  Try with these lines instead.

                  } elseif ($folder.File | where{ “cos.vmdk”, “esxconsole.vmdk” -notcontains $_.Path }) {
                  $folder.File | Where-Object { $_.Path -notmatch “00000.*vmdk|hbrdisk.RDID”} | ForEach-Object {

                    AndJul

                    Perfect LucD, now works fine.
                    Thank you very much for your help.
                    Regards.

                      AndJul

                      One question more.
                      When I use : Remove-OrphanedData -Datastore $ds
                      the filter created works fine.
                      However, when use
                      Remove-OrphanedData -Datastore $ds -Delete -whatif, indicate that all orphaned will be deleted. Filter is not working with -Delete.
                      I have reviewed the “delete” field and I think that should works fine, I don’t know what to modify.
                      Any suggest solving it?
                      Regards and thank you very much for your patience.

                      LucD

                      That was done on purpose.

                      The function checks if the Folder belongs to a registered VM.
                      If that is not the case, then all files in that Folder are possibly orphaned.
                      That includes, besides the VMDK files, also files like for example the VMX file of an orphaned VM.
                      The function would then only delete the VMDK files of that orphaned VM, and not the other files.

                      Another issue might be that a user has created a Folder with a number of VMDKs for attaching/detaching vDisks to a VM.
                      Those shall not be deleted of course.

                      I left it to the user to decide if a full Folder can be removed or not.
                      It would be hard to program that logic.

                      If you still want to remove the VMDK files that are considered ‘orphaned’, you can add some lines to the ElseIf (which handles orphaned Folders).
                      Something like this

                      } elseif ($folder.File | where{ "cos.vmdk", "esxconsole.vmdk" -notcontains $_.Path }) {
                      $folder.File | Where-Object { $_.Path -notmatch "00000.*vmdk|hbrdisk.RDID"} | ForEach-Object {
                      Write-Host "Path: $($_.Path)"
                      New-Object PSObject -Property @{
                      Folder = $folder.FolderPath
                      Name = $_.Path
                      SizeKB = $_.FileSize
                      CapacityKB = $_.CapacityKB
                      Thin = $_.Thin
                      vcenter = $vcenter
                      Extents = [String]::Join(',', ($_.DiskExtents))
                      }
                      if ($Delete) {
                      If ($PSCmdlet.ShouldProcess(($folder.FolderPath + " " + $file.Path), "Remove VMDK")) {
                      $dsBrowser.DeleteFile($folder.FolderPath + $_.Path )
                      }
                      }
                      }
                      }

                      But you should be absolutely sure that you know what you are doing!

    Steve

    Luc,

    thank you for writing this. Since we started using NetBackup for backing up directly the virtual machines, I’ve noticed a massive increase of orphaned snapshots. The question I have is how could I rename the files 1st, as a safety precaution, before rerunning the script a day or 2 later for the deletion.

    Nico

    Hi LucD, nice work.
    I’ve changed some lines in your script to speed up the time execution (Get-View instead of Get-Datastore for example).
    I’m using your script to list orphaned vmdk (also ignoring -ctk.vmdk), but not to delete them.

    I’m facing a random error in the $rootpath, where it stores 2 datastore’s names instead of only one.

    I can’t figure out why. I’ve set $rootpath = $null at the beginning of my foreach loop, still remains.
    So I’ve found that its comes from the $ds variable that sometimes becomes an array of two datastores…
    Did you ever experience this ?

    TIA,
    Nico

      Nico

      I’ve changed $rootPath variable to take $strDatastoreName, so I’m not encoutering this error anymore.

      I still have one question : is it possible to speed up the function : SearchDatastoreSubFolders() that takes between 400 and 120000ms to end…

      I’ve tried to use some get-view ou get-childitem without success.

      If you have any idea 🙂
      TIA

        Alexey

        Hi Nico,
        It looks like I’m having the same or similar issue. If you will be able to solve the issue could you share updated script pls?
        Cheers
        Alex.

    chris86

    Still works in 2016 with VC 5.5!

    Thanks LucD! Great Work!

    Chris

    Still works in 2016 with VC 5.5!

    Thanks LucD!

    Alexey

    Hi Luc,
    First of all thanks for the script.

    But some reason getting next error then running it in production environment:

    Exception calling “SearchDatastoreSubFolders” with “2” argument(s): “Invalid datastore path ‘[SITENAME:SAN-30-01]’.”
    At C:\PS\Remove-OrphanedData.ps1:76 char:9
    + $searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchS …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : VimException

    Any clues about the reason?
    Cheers!

    Dean132

    Hi great script, is there a way to move the orphaned vmdks instead of deleting them, and what about a renaming option do you think that would be possible? I’d like to be able to run the script and then move all of the orphaned vmdk files to a new datastore, keep them there for a short time and then manually perform a delete.

    Dean132

    Hi LucD just wanted to say thanks very much for this script. I have run this and I do appear to be getting some false positives. Is there a way you can run this by searching for vmdks of a name.

    I tried running this

    Get-Harddisk -Datastore DatastoreName | WHERE { $_.Filename -like “*testname*” } | Remove-OrphanedData

    but it shows no results, and I know for a fact there are 100s.

    Luke Murray

    Hey – awesome script! Just started to use it on our 5.1 environment – running it manually at the moment per datastore.

    We have SQL servers that reference VMDK files held on another datastore and these are detecting as “orphaned” by this script. Something I do not indeed want to delete. How do I go about adding exclusions to this.. to ignore say VMs with a certain name format.

      LucD

      Hi Luke,
      What do you mean with “referenced” ?
      Are these VMDK connected to the VM ?

      Drop me an email if you want
      Luc

    Tobias Heyl

    Dear Luc,

    I’m running this in a 5.1 environment and the script does nothing in my case. I call the script and it’s finished within a second, not showing any error or problem. It’s just back to the command line with the blinking cursor.

    In my case I tried:
    PowerCLI C:\Users93373\Documents\PowerShell> .\Remove-OrphanedData.ps1
    PowerCLI C:\Users93373\Documents\PowerShell> .\Remove-OrphanedData.ps1 -Datastore cld_ISOs
    PowerCLI C:\Users93373\Documents\PowerShell> .\Remove-OrphanedData.ps1 -Datastore cld_2T_000
    etc.

    All other operations using other scripts work fine, I’m connected to the vCenter with that same PowerCLI window and can do simple other tasks. Executing your script, though, does just nothing which I don’t understand 😐 can’t trace it when there is no error given …

    Have you ever heard of this or experienced the same issue on your end?
    Anyway thanks for your effort putting this together!

    Best regards
    Tobias

      LucD

      Hi Tobias,
      Sorry to hear that.

      Just to make sure, in the .ps1 file you are calling, you have copied the function itself, but also, at the end, a line where you call the function (like in the Sample Runs in the post ?

    Colin

    Thanks for a great script Luc, it really helped me out.

      LucD

      Thanks, much appreciated.

    Guilherme Neto

    Just writing here to thank you for the script, very helpful indeed.

    joachim

    hi luc,

    at an vcenter 5.1 env with two clusters inside, 18 tb disk and 370 vm´s on it, i get the following error, using your VMDK-orphaned-2 or raiders scripts, any idea ??

    Ausnahme beim Aufrufen von “SearchDatastoreSubFolders” mit 2 Argument(en): “Fe
    hler beim Kommunizieren mit dem Remoteserver.”
    Bei E:\Archiv\VmWare\script\GetOrphanedVmdks-ECT.ps1:80 Zeichen:69
    + $searchResult = $dsBrowser.SearchDatastoreSubFolders <<<< ($r
    ootPath, $searchSpec)
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

      Daniel Jensen

      Yea I solved it by using the method I have described here:
      https://blog.dhjensen.dk/2015/08/orphaned-vmdk-search-return-exception.html

        LucD

        Great, thanks for sharing that !

    James

    Luc,
    Your script detected and deleted all orphaned .vmdk files that show as type “Virtual Disk”. I also have some orphaned .vmdk files that are corrupt and show as type “File”, and the script ignored those. Is there a way to add functionality to detect and delete corrupted vmdk files, as well?

      LucD

      Hi James,
      My function uses the SearchDatastoreSubFolders_Task method to find files. Unfortunately that method accepts only a specific set of Query types. See FileQuery and the available extensions.
      I think something might be possible through the PowerCLI Datastore Provider though.
      Are these corrupted files still having the extension .vmdk ?

    Adam

    I tried running this script on a 5.5 vCenter with PowerCLI 5.5 on an NFS volume and it comes back with nothing.. Not sure if im doing something wrong.. Essentially the command looks like it runs fine but just completes and goes to the prompt.

    Steve Jones

    Can this be modified to ignore the vmdk’s created by a snapshot? The script seems to tag those as orphaned when they’re really not.

    Thanks.

    Steve

    Steve Jones

    We have a problem where we tell people to clean up their old VM’s and they just remove them from inventory instead of deleting them so that they can keep them around for later use. This is a software development cluster with 100’s of VM’s so it’s difficult to go through all of them by hand. This script does most of what we need it to do but it doesn’t seem to catch the case where someone adds a disk to a VM but has it on a different datastore. Is that the case or am I not using the script correctly? So far I’ve only ran “Remove-OrphanedData -Datastore myDataStore”.

    Thanks.

      LucD

      Hi Steve,
      No, you are right.
      The script is datastore-centric, in other words it starts from the datastore(s), not from the VMs I’m afraid.
      But that is a good idea, I’ll see what I can put together.

        LucD

        Hi Steve,
        Another good suggestion, keep them coming.
        I’ll definitely have to do a new version of this script.

    Reza

    Luc,

    Thanks for the script it’s greatly appreciated.

    -Reza

    Krishna

    Does the above script needs to be saved by the name Remove-OrphanedData.ps1 ?
    Does “Remove-OrphanedData -Datastore DS1” refers to name of the file or the function module?
    If it is referred to function module,how does the power shell environment know about the whole code?

      admin

      Hi Krishna, you can save the function in a .ps1 file with whatever name you like.
      The line
      Remove-OrphanedData -Datastore DS1

      is a sample call of the function.

      For that to work the function needs to be know in your PowerShell session. There are several ways to do this.
      1) Store the function and the call in the same .ps1 file. Then call the .ps1 file from the PS prompt
      2) Dot-source the .ps1 file that contains the function with

      . ./myfile.ps1

      Note that there is a blank between the 2 dots !
      Then call the function from the PowerShell prompt.

      Does this make it any clearer ?

    Vamsi

    Luc, This is one hell of a script. Superb.

    Tim

    Ok, please forgive my stupidity, how do I modify this to put in the correct values? I want to run this in my environment but what variables do I put in so that it looks at my datastores and does not delete anything unless I say ok?

      LucD

      @Tim, there are some samples in the Sample usage section in the post.
      As long as you don’t add the -Delete switch, the function will not remove any files.

    Michael

    Nice work on the update, Luc.
    I found this script very useful. Would it be possible to add exemptions for vmdks that are not attached to a VM and orphaned on purpose?

    Duncan Clark

    Hi Luc,

    Firstly great work on this and all your other scripts. Is this script meant to detect orphaned folders too? Just trying to work out if its something I’m doing or if it’s just not in the design of the script in its current form.

    I have seen Orphaned folders, without VMDKs, not being detected by the script.

    IonutN

    Hi,

    This is great script, but i’m having trouble modifying it to include more data like: the timestamp of the vmdk file (this would help to know if the vm has not been used for a long time) or the timestamp of the oldest and newest file in the folder where the vmdk was found.

    do you have a piece of code that can help with that?

    thanks,
    ionut

    Olivier D

    If there is an orphaned vmdk file at the root level (that should never happen …) , the script try to suppress the whole datastore (in this case the parent directory is the datastore itself)

    Olivier

      LucD

      Thanks for finding that potential danger in the script.
      I’ll add a test to avoid that.

    Brandon Seymour

    I get this in the report “WARNING: 2 columns do not fit into the display and were removed.” and it cuts off the names of the files. Can anyone help with this?

    alex

    Does anyone have the same skript rebuild with the vSphere Perl SDK?

      admin

      @Alex, I don’t know I’m afraid.
      But why don’t you raise the question in the VMTN CLI Community ?

      Eugenio

      Hi,

      Did someone find it for vSphere Perl SDK? I am looking in the VMTN community but no luck.

      Thank you,

      Regards

        LucD

        Hi Eugenio,
        I’m afraid my Perl knowledge is too limited 🙂
        Perhaps someone else has something ?

    MarkD

    @LucD
    I realized it was just the folder search portion that was returning the NetApp snapshots and, I don’t need that portion of the script so I commented it out. If you wish to tweak the script for others, the path is typically like this:
    .snapshot/nightly.0/VMNAME/

    Now that I was getting the results I wanted, I attempted to run the delete as follows:
    Remove-OrphanedData -Datastore DATASTORENAME -delete

    However, I get the following error:

    Exception calling “DeleteFile” with “1” argument(s): “Cannot delete file [DATASTORENAME] VM/VM-000003.vmdk”
    At C:\snapremove\Remove-OrphanedData.ps1:94 char:40
    + $dsBrowser.DeleteFile <<<< ($folder.FolderPath + $file.Path)
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

    Any suggestions?

    MarkD

    I’m scanning NFS vols (removed the option for VMFS) and trying to get the script to ignore our NetApp snapshot files in the results. Something like
    where ($folder.FolderPath + $file.Path) -notcontains “*snapshot*”

    The NetApp snapshot files always start with .snapshot/ in the path

    I’m not having much luck despite trying different methods. Any suggestions?

    And thanks for all your scripting work!

      LucD

      @MarkD, I’m afraid the script in its current form only does VMFS datastores.
      I’ll see if I can include NFS datastores as well.

    MarkD

    I ran the script and it only detected orphaned deltas on HBA attached disk. It didn’t detect any of the deltas on NFS (and we have lots of them). Any idea why this would be the case? PS 4.1.1 VC5.0 ESXi5.0U2
    Thanks for all your scripting work Luc!

      LucD

      @MarkD, can you give me an example of such a NetApp snapshot file ?
      Something like ?

      [datastore] .snapshot/VM1/VM1.vmdk

      Or is the path different ?

    Richard

    I have tried this with powercli 5.1 and it does nothing. Are you considering to do a new version for powercli 5.0 or 5.1?

    Thanks.

      LucD

      @Richard, just did some tests with PowerCLI 5.1 and the function seems to be reporting all the orphaned folders and VMDK files.
      Are you sure there are some orphaned folders or VMDK files on the datastore you ran the function against ?
      Perhaps describe how you tested so I can replay that in my test environment.

    Meeks

    Hey Christian,One thing to make not of thoguh. Even thoguh you can upgrade your VMFS3 datastores to VMFS5 it will retain the existing blocksize of the VMFS3 datastore. This becomes a problem if you want to do Storage vMotions across datastores with different block sizes as you’ll see a performance hit. Also this really becomes a problem if you mix datastores with different block sizes inside of a Storage DRS Cluster. Just something to keep in mind. It might be a good idea once the upgrade of VMFS3 to VMFS5 is complete to create a new VMFS5 datastore and migrate all your VMs off of the upgraded VMFS datastore. Thanks for the great article!!

    Gram

    Hi Luc, script works great. Is there a parameter show size in MB or GB and export search results to CSV so I can get it by email? Kind regards

    Nicky

    Cheers Luc, where will the Delete parameter be added in your script, line 003 is it? Is it added already?@LucD

      LucD

      @Nicky, the Delete switch is already in the function.
      When you call the function, you can specify the Delete switch, in that case the files will actually be removed. If you do not specify the Delete switch, the function will only list the orphaned files and folders.
      The Sample runs section has examples.

    Basavaraja K Ayyanagoudar

    Can you please tell me how to run this from client machine? I have Vcenter in different machine and connecting that by vcenter client so how to run this script from client machine and delete all unused cmx/vmdk/ files.

      LucD

      Hi, there are several ways you can do this.
      One of the simplest is as follows:

    1. Save the function to a file with a filetype of .ps1
    2. Open the PowerCLI prompt on your PC
    3. Connect to the vCenter with the Connect-VICenter cmdlet
    4. dot-source the .ps1 file. That way the function is know in your PowerCLI session
    5. . ./yourfile.ps1

    6. Call the function. See the Sample runs in the post.
    7. Remove-OrphanedData

      I hope this helps you to use the function.

    Nicky

    Hey Luc, I just wanted to display the orphaned folders or disks but don’t want to delete them? Is it possible? Cheers!

      LucD

      Hi NIcky, yes, if you don’t add the Delete parameter, the function will just display the orphaned files and folders. See also the Sample Runs section at the end of the post.

    Rick

    @LucD Good point, I agree.

    Rick

    The lines:

    if($ds.GetType().Name -eq “String”){
    $ds = Get-Datastore -Name $ds
    }

    should come before:

    foreach($ds in $Datastore){

    because it is possible for datastores in an enterprise to have the same name, thus this will assign a collection of objects instead of a single object and the rest of the loop won’t handle it properly.

      LucD

      @Rick, you have a point there. The current version of the script doesn’t handle datastores with the same name.
      But I think a better solution would be to do

      if(!$Datastore){
      $Datastore = Get-Datastore
      }
      else{
      $Datastore = $Datastore | %{
      if($_.GetType().Name -eq "string"){
      Get-Datastore -Name $_
      }
      else{
      $_
      }
      }
      }
      foreach($ds in $Datastore){
      if($ds.Type -eq "VMFS" -and $ds.ExtensionData.Summary.MultipleHostAccess){
      ...

      That way you can pass names, but also Datastore objects through the Datastore parameter.
      Let me know if that works for you, and I will update the script.

    Karl

    @LucD

    Does seem strange, as no one else has said the same, it did delete folders, but it deleted folders that contained files, not the empty folders, I thought this script would delete empty (apart from phd) folders, and leave the folders that still contained virtual machines…

      LucD

      @Karl, so you’re saying that the function deleted a folder that still had active VMDK files in there ?
      Or where there other files in the folder ? Could you tell me what type ?

    Karl

    @LucD

    Hey, Thanks LucD, I got the script working, though, maybe I am missunderstood, but it didn’t go and delete all the empty folders from my datastore, it deleted files from folders that still contained files? (luckly I didn’t need them) I thought it would removed the folder and the phd folder that was left after a machine deletion, guess I was wrong.

    thanks 🙂

      LucD

      @Karl, strange. The function should also remove the folders.

    karl

    Hi, I am trying to use your script, to remove empty folders on my datastores, when I am trying to run the script via PowerShell (running command ‘Remove-OrphanedData’ i get the error –
    The term ‘Remove-OrphanedData’ is not recognized as the name of a cmdlet, function, script file, or check the spelling of the name, or if a path was included, verify that the path is correct and try again
    At line:1 char:20
    + Remove-OrphanedData <<<<
    + CategoryInfo : ObjectNotFound: (Remove-OrphanedData:String) [], CommandNotFoundExcept
    + FullyQualifiedErrorId : CommandNotFoundException

    any idea what I am doing wrong? thanks

      LucD

      Hi Karl, before calling a function, you have to make the function known to PowerShell. You can save the function in a .ps1 file (for example orphan.ps1) and then dot-source the file.

      . .\orphan.ps1

      Note that there is a blank between the 2 dots.
      Now the function is known to your PowerShell session and you can call it by it’s name.

      Remove-OrphanedData
      .

    mblake4u

    Hi LucD,

    Just an observation – I’ve got 3 vCenters which all use the same SAN NFS shares. When querying one vCenter, the folders relating to the other 2 vCenters will appear to have orphaned disks (as they obviously aren’t registered on the first vCenter).

    I’m new to PowerCLI but I guess a way around this would be to use arrays or objects and make comparisons between the 3 vCenters – as I’m new, I might just start off with a manual reconciliation.

    Thanks for the script!

    Roderick

    Hi Luc,
    Trying to run the script on vSphere5 vCenter, getting this error:

    The script ‘orphaned_files.ps1’ cannot be run because the following snap-ins th
    at are specified by the “#requires” statements of the script are missing: VMwar
    e.VimAutomation.Core (Version 4.1).

    > Get-PSSnapin vmware.vimautomation.core | select Name,version

    Name Version
    —- ——-
    VMware.VimAutomation.Core 5.0.0.0

    Suggestions?

    Grtz
    Roderick

      LucD

      @Roderick, that is a problem with the #requires line.
      It allows you to specify the exact version of the snapin, but not this version or higher.
      You can safely take out the #requires line at the top.

    Simon

    Hu Luc

    How would I change this to produce a report, maybe in CSV format ?

    And can I assume that this will not detect snapshot VMDK files as orphans ?

    Cheers

    Simon

    Mark

    Can someone post the initial part of this script? There must be connection strings etc that is missing here.

    Thanks
    Mark

      LucD

      @Mark, you only need to connect to your vCenter and then call the function.

      Connect-VIServer -Server your-vCenter -User your-account -Password your-password

      You can use the account with which you are logged, provided that account has the required permissions.

      Connect-VIServer -Server your-vCenter

    sysadmin

    @sysadmin
    Thanks LucD , Its works well now.

    Thanks a lot… it really helped a lot.

    sysadmin

    @LucD

    Thanks Luc for your assistance , I am running with old version

    Name Version
    —- ——-
    VMware.VimAutomation.Core 4.0.1.0

    Will get it corrected now.

    sysadmin

    @LucD

    Thanks Luc for your quick response…. when I am running
    ” Add-PSSnapin VMware.VimAutomation.Core ” its giving me below mentioned error.

    PS P:\> Add-PSSnapin VMware.VimAutomation.Core
    Add-PSSnapin : Cannot add Windows PowerShell snap-in VMware.VimAutomation.Core because it is already added. Verify the name of the snap-in and try again.
    At line:1 char:13
    + Add-PSSnapin <<<< VMware.VimAutomation.Core
    + CategoryInfo : InvalidArgument: (VMware.VimAutomation.Core:String) [Add-PSSnapin], PSArgumentException
    + FullyQualifiedErrorId : AddPSSnapInRead,Microsoft.PowerShell.Commands.AddPSSnapinCommand

    Please assist….

      LucD

      @sysadmin, it looks as if you’re using an older PowerCLI version.
      You can check with

      Get-PSSnapin vmware.vimautomation.core | select Name,version

      The #requires line at the beginning of the script says it need at least PowerCLI 4.1. The reason is that the script uses some features that were introduced in PowerCLI 4.1

    sysadmin

    Hi Luc ,

    Sorry I am new to scripting world , how should I run this script ?

    I am trying to call this function , everytime I am getting below error. Please help

    + . <<<< 'C:\Documents and Settings\ankitjain\Desktop\PS Script\orphan-disk.ps1'
    + CategoryInfo : ResourceUnavailable: (orphan-disk.ps1:String) [], ScriptRequiresException
    + FullyQualifiedErrorId : ScriptRequiresMissingPSSnapIns

    The script 'orphan-disk.ps1' cannot be run because the following snap-ins that are specified by the "#requires" statements of the script are missing: VMware
    .VimAutomation.Core (Version 4.1).
    At line:1 char:2
    + . <<<< 'C:\Documents and Settings\—-\Desktop\PS Script\orphan-disk.ps1'
    + CategoryInfo : ResourceUnavailable: (orphan-disk.ps1:String) [], ScriptRequiresException
    + FullyQualifiedErrorId : ScriptRequiresMissingPSSnapIns

      LucD

      @sysadmin, it looks like the PowerCLI snapin wasn’t loaded when you executed the script.
      Do a

      Add-PSSnapin VMware.VimAutomation.Core

      first.

    Nicholas

    Running this returned some good hits, but also a multitude of false positives. A few examples are similar to the ones below. The script output was the following. But a quick look in SSH shows the truth. Any idea why?

    Name :
    CapacityKB :
    Thin :
    Extents :
    Folder : [ARRAY 02381 – LUN 25] VM-Disks/ICSS/WPQMA310/

    /vmfs/volumes/498ae5a6-b8d1bed9-5115-00215ab17094/VM-Disks # cd ICSS/
    /vmfs/volumes/498ae5a6-b8d1bed9-5115-00215ab17094/VM-Disks/ICSS # ls
    WPQMA310 WPQMD310 WPQMW310
    /vmfs/volumes/498ae5a6-b8d1bed9-5115-00215ab17094/VM-Disks/ICSS # cd WPQMA310
    /vmfs/volumes/498ae5a6-b8d1bed9-5115-00215ab17094/VM-Disks/ICSS/WPQMA310 # ls -l
    -rw-r–r– 1 root root 37 Jun 16 16:38 WPQMA310-155e0bfd.hlog
    -rw——- 1 root root 2147483648 Apr 14 01:14 WPQMA310-155e0bfd.vswp
    -rw——- 1 root root 171798691840 Jun 18 07:12 WPQMA310-flat.vmdk
    -rw——- 1 root root 8684 May 13 01:51 WPQMA310.nvram
    -rw——- 1 root root 503 Oct 28 2010 WPQMA310.vmdk
    -rw-r–r– 1 root root 0 Feb 10 01:09 WPQMA310.vmsd
    -rwxr-xr-x 1 root root 4600 Jun 15 17:50 WPQMA310.vmx
    -rw-r–r– 1 root root 1572 Jun 15 17:50 WPQMA310.vmxf
    -rw——- 1 root root 214748364800 Jun 18 05:32 WPQMA310_1-flat.vmdk
    -rw——- 1 root root 505 Oct 28 2010 WPQMA310_1.vmdk
    -rw——- 1 root root 12884901888 Jun 18 07:13 WPQMA310_2-flat.vmdk
    -rw——- 1 root root 526 Feb 10 01:10 WPQMA310_2.vmdk
    -rw——- 1 root root 8589934592 Mar 21 19:57 WPQMA310_3-flat.vmdk
    -rw——- 1 root root 471 Mar 21 19:57 WPQMA310_3.vmdk
    -rw-r–r– 1 root root 170612 May 27 06:35 vmware-92.log
    -rw-r–r– 1 root root 170730 May 27 22:35 vmware-93.log
    -rw-r–r– 1 root root 170711 May 27 22:37 vmware-94.log
    -rw-r–r– 1 root root 170736 Jun 10 17:12 vmware-95.log
    -rw-r–r– 1 root root 170427 Jun 10 17:12 vmware-96.log
    -rw-r–r– 1 root root 192775 Jun 16 16:38 vmware-97.log
    -rw-r–r– 1 root root 190043 Jun 17 00:31 vmware.log

      LucD

      @Nicholas, when the script only returns a folder, that means that the script didn’t find a registered VM that points to that folder.
      You can have a look inside the .vmx file, find the Display value and check if the name is a registered VM.

    M

    Awesome script… Luc,

    However, when run this script, its not pulling up all data, as expected … the following paramaters are blank…whenever i run this script…

    Name :
    CapacityKB :
    Thin :
    Extents :
    Size :

    But its identifing folder name correctly…. Need your help…Luc

      LucD

      @M, that happens when the script encounters folder that is not related to any known guest at all. This can be caused by a remove of a guest with the “Remove from Inventory” option. In that case the guest is removed from vCenter but the folder and files are kept on the datastore.

    NiTRo

    @LucD
    Thanks Luc, it works very nice indeed 🙂
    Very nice piece of script !

    NiTRo

    @LucD
    Thanks luc, now the code annotations is wrong (bad line numbers) 🙂
    Is there a specific limitation to VMFS or it could be extend to nfs ?

      LucD

      @NiTRo, the line numbers got confused after the copy/paste of the script. That should be fixed now.

      There is no specific reason why it would only support VMFS datastores. The reason I put the test in line 47, is because I only have guests on VMFS datastores and as such I couldn’t test the proper functioning of the script against NFS datastores.

      If you change line 47 from

      if($ds.Type -eq "VMFS" -and $ds.ExtensionData.Summary.MultipleHostAccess){

      into

      if($ds.ExtensionData.Summary.MultipleHostAccess){

      you should be able to test this.
      Don’t use the Delete switch before you are sure you get the correct files and folders back from the script.
      Let me know if this works for NFS datastores and I will update the script to make it more general.

    NiTRo

    Something’s wrong for me Luc, it reports a folder from an existing vm. the home datastore of this vm is located on an nfs datastore an the reported folder contains a vmdk atached to this vm but located on a vmfs datastore. you can easily reproduce that issue.

      LucD

      Thanks for finding this bug.
      I published a new version on May 22nd 2011, and I think this solved the specific problem you have seen.

    Suresh

    Fantastic work, Luc! Yes, as you said most of the orphaned vmdk file reporting scripts found in net were old. And i was the user who asked this, not sure whether only me 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.