yadr – A vdisk reporter

I know there are already numerous scripts to report on virtual hard disks and most of them without a doubt much better then what I came up with for this post.

The reason I started with this script was a question in the PowerCLI Community from Alan in his Thin Provisioned Disks post. He wanted to know if you could get the provisioned and the allocated disk size for a thin provisioned virtual disk.

From past experience I knew that the answer was to be found in the layoutEx property of the VirtualMachine object.

But while I was writing the script I saw opportunities to expand the script to make the produced report the “mother of all virtual disk reports”. Or at least an attempt to include much more information in the produced report 😉

The layoutEx property

The layoutEx property, that came with API 4.0, contains a wealth of information concerning the virtual disks connected to your guests.

Since you can find back the mapping to all individual files that constitute a virtual disk, it is the ideal property to show the actual storage used by each virtual disk and the snapshots.

The following graphic shows how the three major properties in the VirtualMachineFileLayoutEx object are organised. The values in the graphic are from a virtual machine, with one virtual hard disk, and two snapshots.

Handling of snapshots

I assume that most of you know what happens in the background with a virtual disk when you take a snapshot of a virtual machine. The following graphic is small aide-mémoire for understanding the basics.

If you look closely at the Snapshot table (first graphic) you will notice that an entry points to the files, or chain of files, before the snapshot was taken.

For example, Snapshot 2 results in the FileKey sequence 0, 1, 2 and 3. While the actual new files that are created for Snapshot 2, are those with FileKey 4 and 5.

Since I wanted to know how much datastore space Snapshot 2 was taking, I decided to let the script look for the files linked to Snapshot 2. In this case that would be the files with FileKey 4 and 5.

The script

There were a few problems to be solved:

Not all virtual machines have the same number of snapshots. If the data needs to be exported to a CSV file this poses a problem with the Export-Csv cmdlet. The solution I took:
- For all guests determine the number of snapshots, if any, they have and take the maximum of these numbers.
- Define a new object, I called it DiskInfo in the script, and define a sufficient number of snapshot-related properties (Name and Size) so that the guest with the highest number of snapshots can be represented.
- Define the new object in C# and use the Add-Type cmdlet to create the new type.
Like I mentioned earlier, I wanted to display the actual datastore space a specific snapshot takes. For that I had to find the FileKey values that are not in the Snapshot table. My reasoning for this was that I wanted to know how much datastore space would become free when a specific snapshot is removed.

$DiskInfoDef = @"
public struct DiskInfo {
public string VMname;
public string Label;
public string Path;
public bool Thin;
public bool RDM;
public string Used;
public string CapacityMB;
public string AllocatedMB;
"@
$vms = Get-View -ViewType VirtualMachine
$maxSnapshot = ($vms | %{$_.LayoutEx.Snapshot.Count} | Measure-Object -Maximum).Maximum
1..$maxSnapshot | %{
$DiskInfoDef += ("`n`tpublic string Snap" + $_ + "MB;")
$DiskInfoDef += ("`n`tpublic string Snap" + $_ + "Name;")
}
$DiskInfoDef += "`n}"
Add-Type -Language CsharpVersion3 -TypeDefinition $DiskInfoDef
$snapHash = @{}
filter Get-SnapHash{
$snapHash[$_.Snapshot.Value] = $_.Name
if($_.ChildSnapshotList){
$_.ChildSnapShotList | Get-SnapHash
}
}
$vms | %{
$vm = $_
if($vm.Snapshot){
$vm.Snapshot.RootSnapshotList | Get-SnapHash
}
$_.Config.Hardware.Device | where {$_.DeviceInfo.Label -like "Hard disk *"} | %{
$hd = $_
$vm.LayoutEx.Disk | where {$_.Key -eq $hd.Key} | %{
$diskFiles = $_.Chain | %{$_.FileKey} | %{$_}
}
$diskAllocated = 0
$vmdkName = $vm.LayoutEx.File[$diskFiles[0]].Name
$vm.LayoutEx.File | where {$diskFiles -contains $_.Key} | %{
$diskAllocated += $_.Size
}
$used = "na"
if($hd.Backing.ThinProvisioned){
$used = ("{0:P1}" -f ($diskAllocated / 1KB/$hd.CapacityInKB))
}
$diskInfo = New-Object DiskInfo
$diskInfo.VMname = $vm.Name
$diskInfo.Label = $hd.DeviceInfo.Label
$diskInfo.Path = $vmdkName
$diskInfo.Thin = $hd.Backing.ThinProvisioned
$diskInfo.RDM = $hd.Backing.GetType().Name.Contains("RawDisk")
$diskInfo.Used = $used
$diskInfo.CapacityMB = ("{0:N1}" -f ($hd.CapacityInKB/1KB))
$diskInfo.AllocatedMB = ("{0:N1}" -f ($diskAllocated / 1MB))
$snapNr = 1
if($vm.Snapshot){
$vm.LayoutEx.Snapshot | %{
$_.Disk | where {$_.Key -eq $hd.Key} | %{
$prevFiles = $_.Chain | %{$_.FileKey} | %{$_}
}
$vm.LayoutEx.Disk | where {$_.Key -eq $hd.Key} | %{
foreach($chain in $_.Chain){
if($prevFiles -notcontains $chain.FileKey[0] -and $prevFiles -notcontains $chain.FileKey[1]){
break
}
}
}
$snapFiles = $chain.FileKey | %{$_}
$snapSize = ""
$snapName = ""
if($snapFiles){
$snapAllocated = 0
$vm.LayoutEx.File | where {$snapFiles -contains $_.Key} | %{
$snapAllocated += $_.Size
}
$snapSize = ("{0:N1}" -f ($snapAllocated / 1MB))
$snapName = $snapHash[$_.Key.Value]
}
$diskInfo.("Snap" + $snapNr + "MB") = $snapSize
$diskInfo.("Snap" + $snapNr + "Name") = $snapName
$snapNr++
}
}
$diskInfo
}
} | Export-Csv "C:\Yadr-report.csv" -UseCulture -NoTypeInformation

$DiskInfoDef = @"

public struct DiskInfo {

public string VMname;

public string Label;

public string Path;

public bool Thin;

public bool RDM;

public string Used;

public string CapacityMB;

public string AllocatedMB;

$vms = Get-View -ViewType VirtualMachine

$maxSnapshot = ($vms | %{$_.LayoutEx.Snapshot.Count} | Measure-Object -Maximum).Maximum

1..$maxSnapshot | %{

$DiskInfoDef += ("`n`tpublic string Snap" + $_ + "MB;")

$DiskInfoDef += ("`n`tpublic string Snap" + $_ + "Name;")

}

$DiskInfoDef += "`n}"

Add-Type -Language CsharpVersion3 -TypeDefinition $DiskInfoDef

$snapHash = @{}

filter Get-SnapHash{

$snapHash[$_.Snapshot.Value] = $_.Name

if($_.ChildSnapshotList){

$_.ChildSnapShotList | Get-SnapHash

}

$vms | %{

$vm = $_

if($vm.Snapshot){

$vm.Snapshot.RootSnapshotList | Get-SnapHash

}

$_.Config.Hardware.Device | where {$_.DeviceInfo.Label -like "Hard disk *"} | %{

$hd = $_

$vm.LayoutEx.Disk | where {$_.Key -eq $hd.Key} | %{

$diskFiles = $_.Chain | %{$_.FileKey} | %{$_}

}

$diskAllocated = 0

$vmdkName = $vm.LayoutEx.File[$diskFiles[0]].Name

$vm.LayoutEx.File | where {$diskFiles -contains $_.Key} | %{

$diskAllocated += $_.Size

}

$used = "na"

if($hd.Backing.ThinProvisioned){

$used = ("{0:P1}" -f ($diskAllocated / 1KB/$hd.CapacityInKB))

}

$diskInfo = New-Object DiskInfo

$diskInfo.VMname = $vm.Name

$diskInfo.Label = $hd.DeviceInfo.Label

$diskInfo.Path = $vmdkName

$diskInfo.Thin = $hd.Backing.ThinProvisioned

$diskInfo.RDM = $hd.Backing.GetType().Name.Contains("RawDisk")

$diskInfo.Used = $used

$diskInfo.CapacityMB = ("{0:N1}" -f ($hd.CapacityInKB/1KB))

$diskInfo.AllocatedMB = ("{0:N1}" -f ($diskAllocated / 1MB))

$snapNr = 1

if($vm.Snapshot){

$vm.LayoutEx.Snapshot | %{

$_.Disk | where {$_.Key -eq $hd.Key} | %{

$prevFiles = $_.Chain | %{$_.FileKey} | %{$_}

}

$vm.LayoutEx.Disk | where {$_.Key -eq $hd.Key} | %{

foreach($chain in $_.Chain){

if($prevFiles -notcontains $chain.FileKey[0] -and $prevFiles -notcontains $chain.FileKey[1]){

break

}

$snapFiles = $chain.FileKey | %{$_}

$snapSize = ""

$snapName = ""

if($snapFiles){

$snapAllocated = 0

$vm.LayoutEx.File | where {$snapFiles -contains $_.Key} | %{

$snapAllocated += $_.Size

}

$snapSize = ("{0:N1}" -f ($snapAllocated / 1MB))

$snapName = $snapHash[$_.Key.Value]

}

$diskInfo.("Snap" + $snapNr + "MB") = $snapSize

$diskInfo.("Snap" + $snapNr + "Name") = $snapName

$snapNr++

}

$diskInfo

}

} | Export-Csv "C:\Yadr-report.csv" -UseCulture -NoTypeInformation

Annotations

Line 1-11: The “static” part of the C# code that defines the DiskInfo type.

Line 13: Get all the virtual machines. I used the Get-View cmdlet for speed and also because the script only needs properties from the VirtualMachine object.

Line 14: Determine the highest number of snapshot present on any guest.

Line 15-18: The “dynamic” part of the C# code. For each snapshot this adds two properties, SnapiName and SnapiMB,

Line 19: The “newline” character is not really. Just gives a better view when you dump the content of the $DiskInfoDef variable.

Line 21: The new type is created. Note that when you change anything in the definition, you will most probably have to restart your PS environment, otherwise you will get a message that the type already exists. There is, afaik, no way to get rid of a type definition in the .Net environment.

Line 23-30: This filter will recursively run through the snapshot tree and will create a hash table. The table will later be used to find the snapshot name.

Line 32-98: The main loop.

Line 35-37: Create a hash table from the snapshot tree.

Line 39: Loop through all the virtual disks connected to this virtual machine.

Line 41-43: Get all the FileKeys for this specific virtual disk

Line 46: Get the name of the first disk file.

Line 48-50: Summarise the size of each file that comprises the virtual hard disk.

Line 52-55: Check if the virtual hard disk is Thin Provisioned. If it is, calculate the percentage of storage that is actually allocated on the datastore.

Line 57: Create a new object based on the DiskInfo type.

Line 58-65: Start filling in the values for the properties of the DiskInfo object.

Line 66-94: Handle the snapshots (if present) for this virtual hard disk.

Line 67: Start by checking if there is at least one snapshot for this virtual hard disk.

Line 68: Loop through all the snapshots.

Line 69-71: Get the FileKey for all the files for this virtual hard disk before the snapshot was taken. Store these FileKey values in the $prevFiles array.

Line 72-78: Find the first pair of FileKeys in the chain that is not in the $prevFiles array. These are the files that constitute the snapshot.

Line 79: Store the pair of FileKeys in the array $snapFiles.

Line 82-89: Calculate the total size of the snapshot files and get the name of the snapshot from the hash table created in the beginning of the main loop.

Line 90-91: Store the values in the correct properties in the DiskInfo object.

Line 97: Export the array to a CSV file.

Result

The following screenshot shows a sample run of the script.

There are some interesting results in this sample run.

The maximum snapshots any guest has is two.
Server1 (row 2-5) has three virtual hard disks. The first one is thin provisioned and uses currently 47.7% of it’s capacity.
The other two hard disks of Server1 are RDM disks. Notice that the AllocatedMB column says “zero” because there is not really a point in talking about allocated space for RDM disks.
The guest in row 8 has two snapshots.

And last but not least, I suspect there is a bug in the VirtualMachineFileLayoutEx object, more specifically in the Size property of some entries in the Files array.

Have a look at row 20, this disk is not a thin provisioned disk but the AllocatedMB value is less then the CapacityMB value. I would have expected the same value in those two columns.

When I investigate from the console with a simple “ls -l” I see the correct value.

For one reason or another, there are some (very few) virtual disks that have an incorrect size values in the Files table. I couldn’t find out in which cases this seems to happen.

PS: yadr stands for yet another disk reporter

54 Comments

Will

September 9, 2020 at 20:22

Hello! For some reason, my comments are not being moderated.
If the snapshot was taken with virtual machine memory (Powered on), can the amount of memory (.vmem file size) be included in the snapshot report ?

yadr – A vdisk reporter

The layoutEx property

Handling of snapshots

The script

Annotations

Result

54 Comments

Will

Will

WILL

Will

Pragg

Ed Z

Westsider

Westsider

Shannon

SAL

SAL

Martin

Amit Patange

txolson

Robert

Simon

steve

Paulo Aguiar

Paulo Aguiar

MasterPi

MasterPi

Sathya

Sathya

Sathya

Sathya

Vuong Pham

Andrea

Brian

vmachine

Leave a Reply Cancel reply