The feature allows a vSphere administrator to control the placement of virtual machines in a vSphere cluster. There are the VM to VM affinity and anti-affinity rules, and the newer VM to VMHost rules. With the VM to VMHost rules, vSphere introduced the concept of VM and VMHost groups, and the ability to have rules that are a requirement (‘shall’) or a preference (‘should’).

In a recent VMTN PowerCLI community thread a PowerCLI user had a query about exporting and importing DRS rules and groups. At that point Matt Boren and myself developed the idea to provide a PowerShell module. The PowerShell module, which we named DRSRule, provides all the functions we deemed useful for working with DRS rules and groups. And yes, the module includes an export and an import cmdlet !

Matt Boren, famous for the high quality posts on his vNugglets blog, and myself, started working on the module, and after some late evenings and long weekends, we now have a first version we want to make public.

There are already many blogposts on the subject of DRS rules and groups. Including some excellent PowerCLI posts. The first one that springs to mind is Arnim‘s excellent Managing VMware DRS rules using PowerCLI.

And don’t forget to read Matt’s post on the DRSRule module as well.

To get you on your way with the DRSRule module, Matt and myself compiled the following introductory text.

Why the DRSRule module?

The core PowerCLI from VMware allows for many of the DRS rule operations that one might want to perform.  There have been questions over the years in the VMware Communities PowerCLI forum about how to export DRS rules and groups, and, on how to then import such items.

While there are several posts on the web about how to get/export such things, importing/recreating DRS rules/groups from such exported data has not received much focus.  This was the spark for this DRSRule module:  to allow for easy export/import of such rules/groups.  As part of making a module to handle these actions, other cmdlets were born, like the Get-/New-/Remove-/Set-* cmdlet sets for DRS VM groups and VMHost groups, along with sets for DRS VM to VM rules and VM to VMHost rules.

How to use the DRSRule module?

The DRSRule module comes with an about_DRSRule.
Once the module is installed, use that help topic to get a general introduction to the use of the DRSRule module:

Get-Help about_DRSRule

How to get the DRSRule module ?

The source for this PowerShell module is hosted on GitHub.com.
The repository is managed by the GitHub organization PowerCLIGoodies, and in there is the DRSRule repository itself.

There are a couple of ways to get the module, and use it in your PowerShell session:

Automated

Via a script

Use a PowerShell script to perform the download and the extract.
The script is included in the repository as file DownloadLatest.ps1.

Via PsGet

If you have the PsGet module loaded in your PowerShell session, then installing this DRSRule module is as simple as updating PsGet and then installing DRSRule:

Update-Module -Verbose -Module PsGet
Install-Module DRSRule

Manual

Grab the latest version from the GitHub repository.
Extract the content to a suitable folder.

You can go for the Modules folder linked to the Windows account you are using. To find this path, run:

Join-Path ${env:\userprofile} "Documents\WindowsPowerShell\Modules")

Or you can select any other folder.

Either way you will end up with a folder named [pathToModule]\DRSRule, in which the PowerShell files for the DRSRule module reside.

Then, load the DRSRule module into your session.

If you selected one of the folders in $env:PSModulePath (your PSModule path), you can do this with

Import-Module DRSRule

If you picked another folder, you should use the full path to the folder that contains the module’s files, like:

Import-Module [PathToModule]\DRSRule

Please also read the note below about using Unblock-File, since this module is not Authenticode signed.

Note on Unblock-File

Since there is no Authenticode certificate supplied for this module, and, assuming that one’s PowerShell ExecutionPolicy is at least RemoteSigned, below is a quick/easy way to unblock files.  But, this should only be done when you trust the code in the files that you are unblocking — inspecting the files is for your own good.

Get-ChildItem [pathToModules]\DRSRule | Unblock-File

The DRSRule module was set up as community resource, so we are hoping to see your comments, suggestions and code.

Enjoy !

And best of all, you don’t have to code the grouping yourself, it was all done for you by the PowerShell Team.

So make sure this cmdlet belongs to your basic PowerShell repertoire. It will prove invaluable when processing statistical data.

This post will show you several of the different options you have to group statistical data together. And I will illustrate each of these with a sample script.

Grouping using one property

Let’s start with something simple.

We want to produce a CSV file with the average CPU usage per core on a specific ESX host over the last full hour.

With the -Start and -Finish parameters of the Get-Stat cmdlet we can limit the returned data to the last full hour.

And we know that the Instance property has the core-Id or an empty string, to indicate the aggregate value (i.e. the average over all cores).

It is then the obvious choice to group the data on the Instance property.

$esxName = "esx41.test.local"
$esxImpl = Get-VMHost -Name $esxName
$prevHour = (Get-Date -Minute 0 -Second 0).AddSeconds(-20)
$report = @()

$stats = Get-Stat -Entity $esxImpl -Stat cpu.usage.average -Start $prevHour.AddHours(-1) -Finish $prevHour -IntervalMins 5
$groups = $stats | Group-Object -Property Instance
$groups | where {$_.Name -ne ""} | %{
	$report += New-Object PsObject -Property @{
		CoreId = $_.Name
		Average = [Math]::Round(($_.Group | Measure-Object -Property Value -Average).Average, 1)
	}
}
$report | Export-Csv "C:\Core-Avg-1-host-1-hour.csv" -NoTypeInformation -UseCulture

Annotations

Line 3: Gives a timestamp with the current hour minus some seconds so as not to get a value for the full hour.

Line 6: We use the -IntervalMins 5 parameter to make sure Get-Stat returns values for the complete hour. If you leave out this parameter, the Get-Stat cmdlet in some cases returns samples from the Realtime interval. But then you risk, depending how many minutes after the full hour you run the script, that you don’t have samples for the complete hour.

Line 7: The returned statistical data is grouped on the Instance property, which is the Core-Id.

Line 8: The Where-Object filters out the data for the aggregate instance (the average over all the cores).

Line 9-12: This uses a PowerShell v2 feature (-Properties) of the New-Object cmdlet.

Since a picture says more than a 1K words, a simple diagram.

• Get-Stat returns a number of samples
• The CPU core-Id is stored in the Instance property
• The Group-Object cmdlet creates a new array element for each value present in the Instance property seen over all the samples
• Such an array element contains:
  • a Name property that holds in this sample the core-Id
  • a Group property that holds all the samples for the specific code-Id in the Name property

If we have a look with the Variable Inspector from PowerShell Plus, we can see how the $groups variable is composed.

The Group-Object cmdlet creates a GroupInfo object for each “group” it detects.

In the Name property we get the value that was used for a specific group. For $groups[0] this is an empty string, in other words the group for the aggregate value. In $groups[1] we can see from the Name property that this group instance was for core-Id 0.

The resulting CSV file looks like this when the script is executed on single quad-core Nehalem processor with hyper-threading enabled.

Grouping using multiple properties

The previous script produced a CSV file for a specific host. But what if we want the same type of CSV file for multiple ESX hosts ?

In fact this is quite easy with the Group-Object cmdlet, just add the Entity property to the -Properties parameter.

$esxImpl = Get-VMHost
$prevHour = (Get-Date -Minute 0 -Second 0).AddSeconds(-20)
$report = @()

$stats = Get-Stat -Entity $esxImpl -Stat cpu.usage.average -Start $prevHour.AddHours(-1) -Finish $prevHour -IntervalMins 5
$groups = $stats | Group-Object -Property Entity,Instance
$groups | where {$_.Values[1] -ne ""} | %{
	$report += New-Object PsObject -Property @{
		Hostname = $_.Values[0]
		CoreId = $_.Values[1]
		Average = [Math]::Round(($_.Group | Measure-Object -Property Value -Average).Average, 1)
	}
}
$report | Export-Csv "C:\Core-Avg-All-hosts-1-hour.csv" -NoTypeInformation -UseCulture

Annotations

Line 6: The additional property Entity was added to include it in the grouping process.

Line 7: The values for both grouping conditions (Entity and Instance) are available in the Values property of the GroupInfo object. Note that the Name property in the GroupInfo object now contains the values for both grouping conditions.

Line 9: The first element in the Values array is the hostname. The order of the elements corresponds with order that was used on the -Property parameter of the Group-Object cmdlet.

Line 10: The second element is the core-Id.

Another look at your best friend the Variables Inspector makes all this perhaps a bit clearer.

Notice how the Values property contains an array with values for the conditions for a specific group.

And the resulting CSV file looks like this. This was produced for three ESX hosts each with a quad-core Nehalem processor with hyper-threading enabled.

Grouping with Boolean expressions

We take our first sample script one step further.

This time we only want to report the intervals where one or more of the CPU cores used more than 15% of their capacity. I know this 15% is not really a realistic report but consider it’s for demonstration purposes and I didn’t want to overload my test boxes

$esxName = "esx41.test.local"
$esxImpl = Get-VMHost -Name $esxName
$prevHour = (Get-Date -Minute 0 -Second 0).AddSeconds(-20)
$report = @()

$stats = Get-Stat -Entity $esxImpl -Stat cpu.usage.average -Start $prevHour.AddHours(-1) -Finish $prevHour -IntervalMins 5
$groups = $stats | Group-Object -Property {$_.Instance -ne "" -and $_.Value -gt 15}
$groups[$true].Group | %{
	$report += New-Object PsObject -Property @{
		CoreId = $_.Instance
		Time = $_.Timestamp
		Average = [Math]::Round($_.Value, 1)
	}
}
$report | Export-Csv "C:\Core-15%-1-host-1-hour.csv" -NoTypeInformation -UseCulture

Annotations

Line 7: The -Property parameter is now a script block that can return $true or $false. The test for the aggregate instance is now included in the Boolean expression.

Line 8: Another great concept by the PowerShell Team, the array of GroupInfo objects is in fact a hash table! Which means you get access to a specific group by using the required condition value as a key. Since we want all the conditions met, we use the $true key.

The view with the Variable Inspector.

And the resulting CSV file.

Nested groups

And for the “Pièce de résistance” we now do this for multiple ESX hosts and we want to produce separate CSV files for each host.

$esxImpl = Get-VMHost
$prevHour = (Get-Date -Minute 0 -Second 0).AddSeconds(-20)
$report = @()

$stats = Get-Stat -Entity $esxImpl -Stat cpu.usage.average -Start $prevHour.AddHours(-1) -Finish $prevHour -IntervalMins 5
$groups1 = $stats | Group-Object -Property {$_.Instance -ne "" -and $_.Value -gt 15}
$groups2 = $groups1[$true].Group | Group-Object -Property Entity
$groups2 | % {
	$_.Group | % {
		$report += New-Object PsObject -Property @{
			CoreId = $_.Instance
			Time = $_.Timestamp
			Average = [Math]::Round($_.Value, 1)
		}
	}
	$fileName = "C:\Core-15-percent-" + $_.Name + ".CSV"
	$report | Export-Csv $fileName -NoTypeInformation -UseCulture
}

Annotations

Line 6: The method for the first grouping is identical to that of the previous script.

Line 7: The second grouping is done with all the values where the previous Boolean expression returned $true. On those values we now group per Entity which allows us to create  a separate CSV file for each host.

Line 8: The script loops through all the groups (e.g. hosts).

Line 9: The script creates an entry for each statistical value.

Line 16-17: Finally, a CSV file is created for each host.

In my test environment there were two hosts that had cores where the average CPU usage was higher then 15%.

Conclusion

I hope this post made you see the usefulness of the Group-Object cmdlet.

Feel free to post any questions or remarks in the Comments section.