Orphaned Files Revisited

In my Orphaned files and folders – Spring cleaning post from way back, I provided a script to find orphaned VMDKs. This week there was a post in the VMTN PowerCLI Community that had a request to find all orphaned files. Time for a revisit of my old post!

 

file-orphan

I took my old script, massaged it a bit and gave it a more contemporary look and feel.
Just for info, the SearchDatastoreSubFolders method is relatively slow. So scanning a couple of datastores for orphaned files might take a bit of time. Be patient 🙂

The Script

Annotations

Line 28-57: The script constructs the SearchDatastoreSubFolders parameter in the Begin block. This parameter will be the same for all datastores.

Line 56: To get maximum detail for the returned files, the script uses all available FileQuery variations

Line 62-64: Cheap OBN implementation

Line 67: The script only looks at shared VMFS datastores

Line 71: This hash table will be used to determine which file/folder is orphaned or not. First all the files the method fins are placed in the hash table. Then the files that belong to VMs and Templates are removed. Finally some system files are removed. What is left are orphaned files.

Line 76: Get all the files on the datastore with the SearchDatastoreSubFolders method

Line 77-87: Enter the files in the hash table

Line 83-85: Take care of the folder entries. If a file inside a folder is encountered, the script also removes the folder itself from the hash table

Line 90-96: Remove all files that belong to registered VMs from the hash table

Line 99-105: Remove all files that belong to registered Templates from the hash table

Line 108-111: Remove system files. For now these all files that are located in folders that start with a dot, or in a folder named vmkdump.

Line 114-128: If there are entries left in the hash table, collect more information and create an ordered object containing that information.

Sample Usage

The use of this function is rather straightforward, call the function with a datastorename or a datastore object.

or

The function also accepts the datastore objects via the pipeline. You can do

The objects that are returned by the function, depending on the type of file, contain information about the file. The following sample output shows how some properties are specific to certain file types.

orphan-1

If you want to create an orphaned files report for multiple datastores, it is handy to capture the results for each datastore in a separate worksheet in an Excel spreadsheet.

The following sample code uses Doug Finke‘s ImportExcel module to accomplish that.

The resulting spreadsheet has all the orphaned files, with a separate worksheet for each datastore that has orphaned files.

orphan-2

Enjoy!

 

37 Comments

    Thomas

    First off, thanks for this function – will edefinately come in handy.

    I do however experience a slight issue, in that I’m getting no output.

    PowerShel: 5.1.18362.145
    VMware PowerCLI 11.2.0 build 12483598

    When running the function:

    Get-VmwOrphan -datastore fast03
    VERBOSE: 6/12/2019 8:36:42 AM Get-Datastore Finished executio

    And that’s it. What am I missing?

    Running Get-Datastore -name “fast03” shows expected output, so I know I’m connected and using a valid datastore.

    Thanks

      LucD

      Hi Thomas,
      Could it be that there are no orphaned files on that datastore?
      As a test, can you create a dummy VM on that datastore and then unregister it.
      When you then call the function, the files of that dummy VM should be shown.

        Thomas

        Hi

        Still doesn’t show anything. Wouldn’t the script output “”No orphaned files found on $($ds.Name).” if nothing had been found? After hitting “Enter”, i only the above verbose output, and the command finished instantly.

        It’s like it never really initiates or starts to run.

          LucD

          Hi Thomas,
          Sorry to hear that.
          Could it be that this datastore does not pass the test in line 67 (VMFS and shared and accessible)?

            Thomas

            Hi

            You hit the nail! It’s not a VMFS datastore at all (NFS 3).

            Thanks for the assistance.

              LucD

              You can still use the same logic, just remove the test for the VMFS type and the shared storage.

    Prashant

    Thanks for the excellent script. The sample code with ImportExcel function is missing operators. Please modify the sample code with ImportExcel function as:
    ====
    $reportName = ‘C:\temp\VMW-orphan-report-MYCluster.xlsx’
    foreach($ds in (Get-Cluster -Name MYCluster | Get-Datastore | Get-VmwOrphan |
    Group-Object -Property {$_.Folder.Split(‘]’)[0].TrimStart(‘[‘)})){
    $ds.Group | Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow
    }
    ====

    The output is listing Templates as being orphaned and is also picking up VMware-*.log files from actual VMs that exist. A sample output is attached:

    Name Folder Size
    vmware.log [MY_VMDATASTORE_05] MY-VMW-123-02/vmware.log 308926
    UBUNTU-18.vmtx [MY_VMDATASTORE_05] UBUNTU-18/UBUNTU-18.vmtx 2686
    vmware-7.log [MY_VMDATASTORE_05] MY-VMW-123_restored/vmware-7.log 1161458
    vmware-1.log [MY_VMDATASTORE_05] VCSA01/vmware-1.log 267696
    vmware-6.log [MY_VMDATASTORE_05] MY-vm-1234_restored/vmware-6.log 724546
    windows-2012-r2.vmtx [MY_VMDATASTORE_05] windows-2012-r2/windows-2012-r2.vmtx 2542
    vmware-1.log [MY_VMDATASTORE_05] MY-Pilot-02/vmware-1.log 397817

      LucD

      Thanks for the feedback.

      I’m not sure what you mean with “missing operators”?
      That sample seems to work for me, and the code you included looks exactly as what I have, except for the Path of the file.

      The script indeed also lists templates, but only when they are orphaned. In other words not registered in the vCenter.

      The vmware.log files are not automatically removed when the logging cycles. So you could have for example 5 vmware.log type files attached to the VM, while there can be other vmware.log type files that are not connected to the VM anymore. In other words orphaned.

    Scott

    For some reason mine is listing Templates as being orphaned even though they aren’t. It is also picking up VMware-*.log files from actual VMs that exist. Does it not filter out Templates? Not sure on why the log files are showing up either.

      LucD

      Normally it checks if the files it finds belong to a registered VM. If they don’t, it reports the file as orphaned.
      Can you send me an example (you can use the Contact form)?

    Peter

    Hi Luc,

    I have placed your code – from 1st row to 138th – and then we pasted the following from 139th:
    $reportName = ‘C:\orphan-report.xlsx’

    foreach($ds in (Get-Cluster -Name LABMHM | Get-Datastore | Get-VmwOrphan |
    Group-Object -Property {$_.Folder.Split(‘]’)[0].TrimStart(‘[‘)})){
    $ds.Group | Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow

    The whole code(from st row to 144) was saved as Get-VmwOrphan.ps1
    There is an error message when it runs
    PS C:\> .\Get-VmwOrphan.ps1
    WARNING: The ‘Accessible’ property of Datastore type is deprecated. Use the ‘State’ property instead.
    Exception calling “Add” with “2” argument(s): “Az elemet korábban már felvették. Kulcs a szótárban: „[CX4_Medium] log” ; Felvétel alatt álló kulcs: „[CX4_Medium] Log”.”
    At C:\Get-VmwOrphan.ps1:80 char:13
    + $fileTab.Add($key,$file)
    + ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException

    In english: Exception calling “Add” with “2” argument(s): “Item has already been added. Key in dictionary: „[CX4_Medium] log” ; Key being added: „[CX4_Medium] Log”.”

    PowerCLI Version
    —————-
    VMware PowerCLI 11.0.0 build 10380590
    —————
    Component Versions
    —————
    VMware Cis Core PowerCLI Component PowerCLI Component 11.0 build 10335701
    VMware VimAutomation VICore Commands PowerCLI Component PowerCLI Component 11.0 build 10336080

    Your assistant and cooperation is highly appreciacted in this matter!

    Sincerely,
    Peter

      LucD

      Hi Peter,
      Can you give it a try with

      $reportName = 'C:\orphan-report.xlsx'

      Get-Cluster -Name LABMHM | Get-Datastore -PipelineVariable ds |
      ForEach-Object -Process {
      Get-VmwOrphan -Datastore $_
      } |
      Group-Object -Property {$_.Folder.Split(']')[0].TrimStart('[')} |
      Select -ExpandProperty Group |
      Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow

      You can ignore the Warning.

    Mariano

    Hello, please help as I get the following error:

    PowerCLI C:\> Import-Module C:\Users\user\Desktop\Orphaned_VM_Script.ps1
    PowerCLI C:\> Remove-OrphanedData -Datastore wrp-archive
    Get-View : 15/02/2019 2:46:50 PM Get-View View with Id ‘FileManager-FileManager’ was not found on the server(s).
    At C:\Users\user\Desktop\Orphaned_VM_Script.ps1:30 char:12
    + $fileMgr = Get-View FileManager
    + ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : ObjectNotFound: (:) [Get-View], VimException
    + FullyQualifiedErrorId : Core_GetView_WriteNotFoundError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInter
    op.GetVIView

      LucD

      I’m not sure why you are trying to do an Import-Module of the .ps1 file?
      When you want use a function, store the code in a .ps1 file.
      At the end of that file, add the line that calls the function.
      Now from a PowerShell prompt, invoke the .ps1 file, by typing it’s path

    Rajug VCP

    Thank you LuCD,
    am lil confused. I just used the main script and ran in Powercli its nothing happning.
    do I need to copy the other lines also with main script?
    foreach($ds in (Get-Cluster -Name MyCluster | Get-Datastore | Get-VmwOrphan |
    Group-Object -Property {$_.Folder.Split(‘]’)[0].TrimStart(‘[‘))){
    $ds.Group | Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow
    }

      LucD

      Hi Rajug,
      Yes, the function in itself will not do anything.
      You will have to call the function.
      You can place that call in the same .ps1 file as the one you placed the function in.
      The code you included is such a call to the function.

    ANkoji

    Hi Lucd,

    Where I need to copy below lines in Script for output

    $reportName = ‘C:\orphan-report.xlsx’
    foreach($ds in (Get-Cluster -Name MyCluster | Get-Datastore | Get-VmwOrphan |
    Group-Object -Property {$_.Folder.Split(‘]’)[0].TrimStart(‘[‘))){
    $ds.Group | Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow
    }

      LucD

      The Get-VmwOrphan is a function.
      You can copy the complete function to a .ps1 file, and then place your lines of code at the bottom of your .ps1 file (outside the function).

    RegReg

    very good thank you

    Pete

    Great script, thanks! I used installed the ImportExcel module and tried your sample code and got an error. It looks like it’s missing a } at the end of line 4 before the final 2 closing parenthesis. Line 4 should be as follows. At least that’s how it worked for me.

    Group-Object -Property {$_.Folder.Split(‘]’)[0].TrimStart(‘[‘)})){

    Jon

    Can folders be excluded? When I modify for including NFS datastores, NetApp snapshots directory (.snapshot) becomes visible too and the scan sorts through all of that as well. Slows down the scan unnecessarily. Thoughts?

      LucD

      Hi Jon,
      The SearchDatastoreSubFolders method doesn’t have any options to exclude specific folders I’m afraid.
      The only option I can think of is to call the Get-VmwOrphan function and then filter the results with a Where-clause on the Folder property.
      Luc

    Dario

    Hi Luc,

    there seems to be a error when using this script with PowerCLI 6.5:
    C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Scripts\Initialize-PowerCliEnvironment.ps1
    WARNING: The ‘Accessible’ property of Datastore type is deprecated. Use the ‘State’ property instead.
    Exception calling “SearchDatastoreSubFolders” with “2” argument(s): “Invalid datastore path ‘[s-infra-00-001]’.”
    At D:\sw\Script\PowerShell\VMWare\Get-Orphaned-Files-2.1.ps1:68 char:1
    + $searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec) | S …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : VimException

    And in your sample code at the bottom of the post there is a “}” missing on line 3.

    JD

    Hey LucD

    Just getting your script to run after hunting around

    I get the following error:
    PowerCLI C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI> Get-VmwO
    rphan -Datastore V-Cluster2-DISKa
    WARNING: The ‘Accessible’ property of Datastore type is deprecated. Use the
    ‘State’ property instead.
    Unable to find type [ordered]: make sure that the assembly containing this type
    is loaded.
    At C:\Users\jdesmarais\Documents\WindowsPowerShell\Modules\MyFunctions\MyFuncti
    ons.psm1:103 char:17
    + $obj = [ordered] <<<< @{
    + CategoryInfo : InvalidOperation: (ordered:String) [], RuntimeEx
    ception
    + FullyQualifiedErrorId : TypeNotFound

      JD

      I figured it out, I needed powershell 3 for windows 7.

    JerH

    Thanks for taking the time and effort to create these scripts. I’m really new to using PS scripts in VMware. I’m assuming somewhere along the way, I need to specify which vCenter to run this on? Can you help with the proper language for that? Also, I get an error with I try to run it with the export-excel function. Do I just paste that code into the end of your script? Maybe I need to download Doug Finkel’s code to make it work? Sorry for the annoying newb questions, but I’m not getting anywhere trying to figure it out on my own…

    Dennis Zimmer

    Great Script Luc! I would change line 67 to avoid an error for datastores that can’t be accessed.
    if($ds.Type -eq “VMFS” -and $ds.ExtensionData.Summary.MultipleHostAccess -and $ds.Accessible){

    That script will definitely makes its way into our OpBot repository!

      LucD

      Great suggestion Dennis.
      The code has been updated.

        Vladimir

        Hi LucD! In order to include NFS shared volumes, I would like to change line like 67 this

        if((($ds.Type -eq “VMFS”) -or ($ds.Type -eq “NFS”)) -and $ds.ExtensionData.Summary.MultipleHostAccess -and $ds.State -eq “Available”) {

        (already included ds.State instead deprecated ds.Accessible property)

    Dennis Zimmer

    Amazing script Luc, that will find its way into our OpBot function library. Small improvement, that the script runs through even if some datastores are not accessible:
    Line 61
    if($ds.Type -eq “VMFS” -and $ds.ExtensionData.Summary.MultipleHostAccess){

    I would change to
    if($ds.Type -eq “VMFS” -and $ds.ExtensionData.Summary.MultipleHostAccess -and $ds.Accessible){

    Todd Scalzott

    Thanks so much for this great script, Luc.

    Are you missing “$_.Group |” prior to “Export-Excel” in your sample utilizing the ImportExcel module?

    Thanks,
    Todd

      LucD

      Thanks Todd, your assumption is correct, well spotted.
      I updated the sample code.

    Stu Green

    Hi Luc,

    Thanks for this script 🙂

    I added this filter which I found to give the output in Byte\KB\MB\GB\TB\PB format

    https://mjolinor.wordpress.com/2012/04/18/powershell-byte-count-formatting-simplified/

    Cheers!

    Ritam9

    Hi Luc,
    Thanks again for your wonderful scripts. Is this has to be run from the Powershell only ? I am not able to run from powercli . Can you provide the script which can be run from the powercli. Also does it avoid snapshot files / ctk files ?

    PowerCLI D:\> powercliversion

    PowerCLI Version
    —————-
    VMware vSphere PowerCLI 5.1 Release 1 build 793510
    —————
    Snapin Versions
    —————
    VMWare AutoDeploy PowerCLI Component 5.1 build 768137
    VMWare ImageBuilder PowerCLI Component 5.1 build 768137
    VMware License PowerCLI Component 5.1 build 669840
    VMware vSphere PowerCLI Component 5.1 build 793489

      LucD

      Hi Ritam9,
      You have to download the function, under The Script, from the post, and store it in a .ps1 file.
      Now, at the end of the file, you will have to call the function. I have some examples of that in the Sample Usage.
      Once you have all that in the .ps1 file, save the file and from the PowerCLI prompt you can call the .ps1 file.
      Note that you will have to connect (Connect-VIServer) first to a vSphere Server (vCenter or ESXi node).
      I hope that helps.
      Luc

    Tony

    Thanks for another great script. Note, in your example section you have Get-VmwOrphaned -Datastore DS1, which should be updated to Get-VmwOrphan

      LucD

      Thanks Tony.
      And thanks for spotting that!

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.