Orphaned Files Revisited

In my Orphaned files and folders – Spring cleaning post from way back, I provided a script to find orphaned VMDKs. This week there was a post in the VMTN PowerCLI Community that had a request to find all orphaned files. Time for a revisit of my old post!

I took my old script, massaged it a bit and gave it a more contemporary look and feel.
Just for info, the SearchDatastoreSubFolders method is relatively slow. So scanning a couple of datastores for orphaned files might take a bit of time. Be patient 🙂

The Script

function Get-VmwOrphan{
<#
.SYNOPSIS
Find orphaned files on a datastore
.DESCRIPTION
This function will scan the complete content of a datastore.
It will then verify all registered VMs and Templates on that
datastore, and compare those files with the datastore list.
Files that are not present in a VM or Template are considered
orphaned
.NOTES
Author:  Luc Dekens
.PARAMETER Datastore
The datastore that needs to be scanned
.EXAMPLE
PS> Get-VmwOrphan -Datastore DS1
.EXAMPLE
PS> Get-Datastore -Name DS* | Get-VmwOrphan
#>
[CmdletBinding()]
param(
[parameter(Mandatory=$true,ValueFromPipeline=$true)]
[PSObject[]]$Datastore
)
Begin{
$flags = New-Object VMware.Vim.FileQueryFlags
$flags.FileOwner = $true
$flags.FileSize = $true
$flags.FileType = $true
$flags.Modification = $true
$qFloppy = New-Object VMware.Vim.FloppyImageFileQuery
$qFolder = New-Object VMware.Vim.FolderFileQuery
$qISO = New-Object VMware.Vim.IsoImageFileQuery
$qConfig = New-Object VMware.Vim.VmConfigFileQuery
$qConfig.Details = New-Object VMware.Vim.VmConfigFileQueryFlags
$qConfig.Details.ConfigVersion = $true
$qTemplate = New-Object VMware.Vim.TemplateConfigFileQuery
$qTemplate.Details = New-Object VMware.Vim.VmConfigFileQueryFlags
$qTemplate.Details.ConfigVersion = $true
$qDisk = New-Object VMware.Vim.VmDiskFileQuery
$qDisk.Details = New-Object VMware.Vim.VmDiskFileQueryFlags
$qDisk.Details.CapacityKB = $true
$qDisk.Details.DiskExtents = $true
$qDisk.Details.DiskType = $true
$qDisk.Details.HardwareVersion = $true
$qDisk.Details.Thin = $true
$qLog = New-Object VMware.Vim.VmLogFileQuery
$qRAM = New-Object VMware.Vim.VmNvramFileQuery
$qSnap = New-Object VMware.Vim.VmSnapshotFileQuery
$searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
$searchSpec.details = $flags
$searchSpec.Query = $qFloppy,$qFolder,$qISO,$qConfig,$qTemplate,$qDisk,$qLog,$qRAM,$qSnap
$searchSpec.sortFoldersFirst = $true
}
Process{
foreach($ds in $Datastore){
if($ds.GetType().Name -eq "String"){
$ds = Get-Datastore -Name $ds
}
# Only shared VMFS datastore
if($ds.Type -eq "VMFS" -and $ds.ExtensionData.Summary.MultipleHostAccess -and $ds.State -eq 'Available'){
Write-Verbose -Message "$(Get-Date)`t$((Get-PSCallStack)[0].Command)`tLooking at $($ds.Name)"
# Define file DB
$fileTab = @{}
# Get datastore files
$dsBrowser = Get-View -Id $ds.ExtensionData.browser
$rootPath = "[" + $ds.Name + "]"
$searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec) | Sort-Object -Property {$_.FolderPath.Length}
foreach($folder in $searchResult){
foreach ($file in $folder.File){
$key = "$($folder.FolderPath)$(if($folder.FolderPath[-1] -eq ']'){' '})$($file.Path)"
$fileTab.Add($key,$file)
$folderKey = "$($folder.FolderPath.TrimEnd('/'))"
if($fileTab.ContainsKey($folderKey)){
$fileTab.Remove($folderKey)
}
}
}
# Get VM inventory
Get-VM -Datastore $ds | %{
$_.ExtensionData.LayoutEx.File | %{
if($fileTab.ContainsKey($_.Name)){
$fileTab.Remove($_.Name)
}
}
}
# Get Template inventory
Get-Template | where {$_.DatastoreIdList -contains $ds.Id} | %{
$_.ExtensionData.LayoutEx.File | %{
if($fileTab.ContainsKey($_.Name)){
$fileTab.Remove($_.Name)
}
}
}
# Remove system files & folders from list
$systemFiles = $fileTab.Keys | where{$_ -match "] \.|vmkdump"}
$systemFiles | %{
$fileTab.Remove($_)
}
# Organise remaining files
if($fileTab.Count){
$fileTab.GetEnumerator() | %{
$obj = [ordered]@{
Name = $_.Value.Path
Folder = $_.Name
Size = $_.Value.FileSize
CapacityKB = $_.Value.CapacityKb
Modification = $_.Value.Modification
Owner = $_.Value.Owner
Thin = $_.Value.Thin
Extents = $_.Value.DiskExtents -join ','
DiskType = $_.Value.DiskType
HWVersion = $_.Value.HardwareVersion
}
New-Object PSObject -Property $obj
}
Write-Verbose -Message "$(Get-Date)`t$((Get-PSCallStack)[0].Command)`tFound orphaned files on $($ds.Name)!"
}
else{
Write-Verbose -Message "$(Get-Date)`t$((Get-PSCallStack)[0].Command)`tNo orphaned files found on $($ds.Name)."
}
}
}
}
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

function Get-VmwOrphan{

.SYNOPSIS

Find orphaned files on a datastore

.DESCRIPTION

This function will scan the complete content of a datastore.

It will then verify all registered VMs and Templates on that

datastore, and compare those files with the datastore list.

Files that are not present in a VM or Template are considered

orphaned

.NOTES

Author: Luc Dekens

.PARAMETER Datastore

The datastore that needs to be scanned

.EXAMPLE

PS> Get-VmwOrphan -Datastore DS1

.EXAMPLE

PS> Get-Datastore -Name DS* | Get-VmwOrphan

[CmdletBinding()]

param(

[parameter(Mandatory=$true,ValueFromPipeline=$true)]

[PSObject[]]$Datastore

)

Begin{

$flags = New-Object VMware.Vim.FileQueryFlags

$flags.FileOwner = $true

$flags.FileSize = $true

$flags.FileType = $true

$flags.Modification = $true

$qFloppy = New-Object VMware.Vim.FloppyImageFileQuery

$qFolder = New-Object VMware.Vim.FolderFileQuery

$qISO = New-Object VMware.Vim.IsoImageFileQuery

$qConfig = New-Object VMware.Vim.VmConfigFileQuery

$qConfig.Details = New-Object VMware.Vim.VmConfigFileQueryFlags

$qConfig.Details.ConfigVersion = $true

$qTemplate = New-Object VMware.Vim.TemplateConfigFileQuery

$qTemplate.Details = New-Object VMware.Vim.VmConfigFileQueryFlags

$qTemplate.Details.ConfigVersion = $true

$qDisk = New-Object VMware.Vim.VmDiskFileQuery

$qDisk.Details = New-Object VMware.Vim.VmDiskFileQueryFlags

$qDisk.Details.CapacityKB = $true

$qDisk.Details.DiskExtents = $true

$qDisk.Details.DiskType = $true

$qDisk.Details.HardwareVersion = $true

$qDisk.Details.Thin = $true

$qLog = New-Object VMware.Vim.VmLogFileQuery

$qRAM = New-Object VMware.Vim.VmNvramFileQuery

$qSnap = New-Object VMware.Vim.VmSnapshotFileQuery

$searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec

$searchSpec.details = $flags

$searchSpec.Query = $qFloppy,$qFolder,$qISO,$qConfig,$qTemplate,$qDisk,$qLog,$qRAM,$qSnap

$searchSpec.sortFoldersFirst = $true

}

Process{

foreach($ds in $Datastore){

if($ds.GetType().Name -eq "String"){

$ds = Get-Datastore -Name $ds

}

# Only shared VMFS datastore

if($ds.Type -eq "VMFS" -and $ds.ExtensionData.Summary.MultipleHostAccess -and $ds.State -eq 'Available'){

Write-Verbose -Message "$(Get-Date)`t$((Get-PSCallStack)[0].Command)`tLooking at $($ds.Name)"

# Define file DB

$fileTab = @{}

# Get datastore files

$dsBrowser = Get-View -Id $ds.ExtensionData.browser

$rootPath = "[" + $ds.Name + "]"

$searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec) | Sort-Object -Property {$_.FolderPath.Length}

foreach($folder in $searchResult){

foreach ($file in $folder.File){

$key = "$($folder.FolderPath)$(if($folder.FolderPath[-1] -eq ']'){' '})$($file.Path)"

$fileTab.Add($key,$file)

$folderKey = "$($folder.FolderPath.TrimEnd('/'))"

if($fileTab.ContainsKey($folderKey)){

$fileTab.Remove($folderKey)

}

# Get VM inventory

Get-VM -Datastore $ds | %{

$_.ExtensionData.LayoutEx.File | %{

if($fileTab.ContainsKey($_.Name)){

$fileTab.Remove($_.Name)

}

# Get Template inventory

Get-Template | where {$_.DatastoreIdList -contains $ds.Id} | %{

$_.ExtensionData.LayoutEx.File | %{

if($fileTab.ContainsKey($_.Name)){

$fileTab.Remove($_.Name)

}

# Remove system files & folders from list

$systemFiles = $fileTab.Keys | where{$_ -match "] \.|vmkdump"}

$systemFiles | %{

$fileTab.Remove($_)

}

# Organise remaining files

if($fileTab.Count){

$fileTab.GetEnumerator() | %{

$obj = [ordered]@{

Name = $_.Value.Path

Folder = $_.Name

Size = $_.Value.FileSize

CapacityKB = $_.Value.CapacityKb

Modification = $_.Value.Modification

Owner = $_.Value.Owner

Thin = $_.Value.Thin

Extents = $_.Value.DiskExtents -join ','

DiskType = $_.Value.DiskType

HWVersion = $_.Value.HardwareVersion

}

New-Object PSObject -Property $obj

}

Write-Verbose -Message "$(Get-Date)`t$((Get-PSCallStack)[0].Command)`tFound orphaned files on $($ds.Name)!"

}

else{

Write-Verbose -Message "$(Get-Date)`t$((Get-PSCallStack)[0].Command)`tNo orphaned files found on $($ds.Name)."

}

Annotations

Line 28-57: The script constructs the SearchDatastoreSubFolders parameter in the Begin block. This parameter will be the same for all datastores.

Line 56: To get maximum detail for the returned files, the script uses all available FileQuery variations

Line 62-64: Cheap OBN implementation

Line 67: The script only looks at shared VMFS datastores

Line 71: This hash table will be used to determine which file/folder is orphaned or not. First all the files the method fins are placed in the hash table. Then the files that belong to VMs and Templates are removed. Finally some system files are removed. What is left are orphaned files.

Line 76: Get all the files on the datastore with the SearchDatastoreSubFolders method

Line 77-87: Enter the files in the hash table

Line 83-85: Take care of the folder entries. If a file inside a folder is encountered, the script also removes the folder itself from the hash table

Line 90-96: Remove all files that belong to registered VMs from the hash table

Line 99-105: Remove all files that belong to registered Templates from the hash table

Line 108-111: Remove system files. For now these all files that are located in folders that start with a dot, or in a folder named vmkdump.

Line 114-128: If there are entries left in the hash table, collect more information and create an ordered object containing that information.

Sample Usage

The use of this function is rather straightforward, call the function with a datastorename or a datastore object.

Get-VmwOrphan -Datastore MyDS1

1	Get-VmwOrphan -Datastore MyDS1

$ds = Get-Datastore -Name MyDS1
Get-VmwOrphan -Datastore $ds

$ds = Get-Datastore -Name MyDS1

Get-VmwOrphan -Datastore $ds

The function also accepts the datastore objects via the pipeline. You can do

Get-Cluster -Name MyCluster | Get-Datastore | Get-VmwOrphan

1	Get-Cluster -Name MyCluster \| Get-Datastore \| Get-VmwOrphan

The objects that are returned by the function, depending on the type of file, contain information about the file. The following sample output shows how some properties are specific to certain file types.

If you want to create an orphaned files report for multiple datastores, it is handy to capture the results for each datastore in a separate worksheet in an Excel spreadsheet.

The following sample code uses Doug Finke‘s ImportExcel module to accomplish that.

$reportName = 'C:\orphan-report.xlsx'
foreach($ds in (Get-Cluster -Name MyCluster | Get-Datastore | Get-VmwOrphan | Group-Object -Property {$_.Folder.Split(']')[0].TrimStart('[')})){
$ds.Group | Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow
}

$reportName = 'C:\orphan-report.xlsx'

foreach($ds in (Get-Cluster -Name MyCluster | Get-Datastore | Get-VmwOrphan | Group-Object -Property {$_.Folder.Split(']')[0].TrimStart('[')})){

$ds.Group | Export-Excel -Path $reportName -WorkSheetname $ds.Name -AutoSize -AutoFilter -FreezeTopRow

}

The resulting spreadsheet has all the orphaned files, with a separate worksheet for each datastore that has orphaned files.

Enjoy!

92 Comments

Marco Hofmann

October 17, 2023 at 11:12

Hi! Really would like to use that script, but I only get zero results. Even with -Verbose.
This should work against VSAN 7, should it?

Orphaned Files Revisited

The Script

Annotations

Sample Usage

92 Comments

Shibin

Shibin

Shibin

Shibin

Shibin

Shibin

mmccar

Eric

Pablo

Rani

Rani

Rani

Rani

Ricardo Jose

Joy

Joy

Joy

Joy

Martin

Martin

Chris

Chris

Danny Z

Michael Anselmi

Johnn Wong

LB-ATL

LB-ATL

Nathan C

Otis Wheeler

Travis

Thomas

Thomas

Thomas

Prashant

Shaneb

Scott

Peter

Mariano

Rajug VCP

ANkoji

RegReg

Pete

Jon

JerH

Dennis Zimmer

Vladimir

Dennis Zimmer

Stu Green

Ritam9

Tony

Leave a Reply Cancel reply