During our presentation at the Dutch VMUG Event 2011, Alan and myself showed how several entries of the Top 10 vSphere 5 Features session could be automated with the help of PowerCLI. In the session we showed several demos.
This post is the first in a series, that will publish and document most of the scripts we used for the demos.
On the first position in the Top 10 we have Storage DRS. This feature brings intelligent placement of VMs and storage load balancing based on space usage and IO metrics. See part 4, part 5 and part 6 in Cormac‘s excellent series on vSphere 5 Storage Features.
In the current PowerCLI version (5, build 435426) there is apparently only 1 cmdlet that works with Datastore Clusters, the Get-DatastoreCluster cmdlet. Unfortunately the cmdlet doesn’t seem to provide everything we came to expect from the regular PowerCLI objects. There is for example no ExtensionData property that would give us access to the server-side object for a Datastore Cluster.
The other basic actions (create, remove, configure) around Datastore Clusters are, for the time being, unfortunately missing from PowerCLI. And as you might have guessed, I felt the need to have some functions in my toolkit, that would allow me to do some basic stuff with Datastore Clusters.
The functions
Please note that the VMware vSphere API Reference refers to a Datastore Cluster as a StoragePod.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 |
function Set-DatastoreCluster{ <# .SYNOPSIS Configure a DatastoreCluster .DESCRIPTION The function is used to configure a DatastoreCluster. This includes the settings and the Datastores that are part of the cluster. .NOTES Author: Luc Dekens .PARAMETER Pod The StoragePod object. This can a name, an object returned by Get-DatastoreCluster or a StoragePod object. .PARAMETER SDRSEnabled A switch to define is SDRS shall be enabled or not in the DatastoreCluster .PARAMETER AutomationLevel Specifies the default SDRS behavior for virtual machines in the DatastoreCluster. .PARAMETER IOMetric A switch whether or not SDRS takes into account storage I/O workload while calculating the recommendations .PARAMETER SpaceUtilization When space utiixation exceeds this threshold, on one or more of the datastores in the DatastoreCluster, SDRS will generate migration recommendations. .PARAMETER SpaceUtilizationDifference SDRS will generate migration recommendations when the space utilization difference between any of the datastores in the DatastoreCluster exceeds this threshold .PARAMETER IOLatency When latency exceeds this threshold, on one or more of the datastores in the DatastoreCluster, SDRS will generate migration recommendations. .PARAMETER IOLatencyDifference SDRS will generate migration recommendations when the IO latency difference between any of the datastores in the DatastoreCluster exceeds this threshold .PARAMETER Interval The interval, in minutes, that SDRS runs to load balance among datastores within a DatastoreCluster .PARAMETER Datastore An array of datastores to add to the DatastoreCluster. The value can be a name or an object returned by Get-Datastore .EXAMPLE PS> Set-DatastoreCluster -Name DSC -SDRSEnabled:$false .EXAMPLE PS> Set-DatastoreCluster -Name DSC -IOLatency 25 .EXAMPLE PS> Set-DatastoreCluster -Name DSC -Datastore $ds #> param( [CmdletBinding()] [parameter(Position = 0, Mandatory = $true, ValueFromPipeline = $true)] [PSObject]$Pod, [switch]$SDRSEnabled, [ValidateSet("automated","manual")] [string]$AutomationLevel, [switch]$IOMetric, [ValidateRange(50,100)] [int]$SpaceUtilization, [ValidateRange(1,50)] [int]$SpaceUtilizationDifference, [ValidateRange(5,100)] [int]$IOLatency, [ValidateRange(1,100)] [int]$IOLatencyDifference, [ValidateRange(60,43200)] [int]$Interval, [PSObject]$Datastore ) begin{ $srm = Get-View StorageResourceManager } process{ if($Pod.GetType().Name -eq "string"){ $Pod = Get-StoragePod -Name $Pod } elseif($Pod.GetType().Name -eq "DatastoreClusterImpl"){ $Pod = Get-StoragePod -Name $Pod.Name } if(!($PSCmdlet.MyInvocation.BoundParameters.Count -le 2 -and $PSCmdlet.MyInvocation.BoundParameters.ContainsKey("Datastore"))){ $spec = New-Object VMware.Vim.StorageDrsConfigSpec $spec.podConfigSpec = New-Object VMware.Vim.StorageDrsPodConfigSpec $spec.podConfigSpec.DefaultIntraVmAffinity = $true if($AutomationLevel){ $spec.podConfigSpec.DefaultVmBehavior = $AutomationLevel } if($SDRSEnabled){ $spec.podConfigSpec.Enabled = $SDRSEnabled } if($IOLatency -or $IOLatencyDifference){ $spec.podConfigSpec.IoLoadBalanceConfig = New-Object VMware.Vim.StorageDrsIoLoadBalanceConfig if($IOLatency){ $spec.podConfigSpec.IoLoadBalanceConfig.IoLatencyThreshold = $IOLatency } if($IOLatencyDifference){ $spec.podConfigSpec.IoLoadBalanceConfig.IoLoadImbalanceThreshold = $IOLatencyDifference } } if($IOMetric){ $spec.podConfigSpec.IoLoadBalanceEnabled = $IOMetric } if($Interval){ $spec.podConfigSpec.loadBalanceInterval = $Interval } if($SpaceUtilizationDifference -or $SpaceUtilization){ $spec.podConfigSpec.spaceLoadBalanceConfig = New-Object VMware.Vim.StorageDrsSpaceLoadBalanceConfig if($SpaceUtilizationDifference){ $spec.podConfigSpec.spaceLoadBalanceConfig.MinSpaceUtilizationDifference = $SpaceUtilizationDifference } if($SpaceUtilization){ $spec.podConfigSpec.spaceLoadBalanceConfig.SpaceUtilizationThreshold = $SpaceUtilization } } $srm.ConfigureStorageDrsForPod($Pod.MoRef, $spec, $true) } if($Datastore){ $dsList = $Datastore | %{ if($_.GetType().Name -eq "string"){ (Get-Datastore -Name $_).ExtensionData.MoRef } else{$_.ExtensionData.MoRef} } $Pod.MoveIntoFolder($dsList) } } end{$Pod} } function New-DatastoreCluster{ <# .SYNOPSIS Create a DatastoreCluster .DESCRIPTION The function is used to configure a DatastoreCluster. This includes the settings and the Datastores that are part of the cluster. .NOTES Author: Luc Dekens .PARAMETER Name The name of the DatastoreCluster .PARAMETER Folder The Folder where the DatastoreCluster shall be created. .PARAMETER SDRSEnabled A switch to define is SDRS shall be enabled or not in the DatastoreCluster .PARAMETER AutomationLevel Specifies the default SDRS behavior for virtual machines in the DatastoreCluster. .PARAMETER IOMetric A switch whether or not SDRS takes into account storage I/O workload while calculating the recommendations .PARAMETER SpaceUtilization When space utiixation exceeds this threshold, on one or more of the datastores in the DatastoreCluster, SDRS will generate migration recommendations. .PARAMETER SpaceUtilizationDifference SDRS will generate migration recommendations when the space utilization difference between any of the datastores in the DatastoreCluster exceeds this threshold .PARAMETER IOLatency When latency exceeds this threshold, on one or more of the datastores in the DatastoreCluster, SDRS will generate migration recommendations. .PARAMETER IOLatencyDifference SDRS will generate migration recommendations when the IO latency difference between any of the datastores in the DatastoreCluster exceeds this threshold .PARAMETER Interval The interval, in minutes, that SDRS runs to load balance among datastores within a DatastoreCluster .PARAMETER Datastore An array of datastores to add to the DatastoreCluster. The value can be a name or an object returned by Get-Datastore .EXAMPLE PS> New-DatastoreCluster -Name DSC -SDRSEnabled:$false .EXAMPLE PS> New-DatastoreCluster -Name DSC -IOLatency 25 .EXAMPLE PS> New-DatastoreCluster -Name DSC -Datastore $ds #> param( [CmdletBinding()] [parameter(Position = 0, Mandatory = $true, ValueFromPipeline = $true)] [string]$Name, [parameter(Position = 1)] [PSObject]$Folder = "datastore", [switch]$SDRSEnabled = $true, [ValidateSet("automated","manual")] [string]$AutomationLevel = "automated", [switch]$IOMetric = $true, [ValidateRange(50,100)] [int]$SpaceUtilization = 80, [ValidateRange(1,50)] [int]$SpaceUtilizationDifference = 5, [ValidateRange(5,100)] [int]$IOLatency = 15, [ValidateRange(1,100)] [int]$IOLatencyDifference = 5, [ValidateRange(60,43200)] [int]$Interval = 480, [PSObject[]]$Datastore ) begin{ if($Folder.GetType().Name -eq "string"){ $Folder = Get-Folder -Name $Folder } } process{ $podMoRef = $Folder.ExtensionData.CreateStoragePod($Name) $Pod = Get-View -Id $podMoRef Set-DatastoreCluster -Pod $pod ` -SDRSEnabled:$SDRSEnabled -AutomationLevel $AutomationLevel ` -IOMetric:$IOMetric -SpaceUtilization $SpaceUtilization ` -SpaceUtilizationDifference $SpaceUtilizationDifference ` -IOLatency $IOLatency -IOLatencyDifference $IOLatencyDifference ` -Interval $Interval if($Datastore){ Set-DatastoreCluster -Pod $pod -Datastore $Datastore } } end{$Pod} } function Get-StoragePod{ <# .SYNOPSIS Find a DatastoreCluster .DESCRIPTION The function will return a StoragePod object. This is the server-side object used by vSphere for a DatastoreCluster. .NOTES Author: Luc Dekens .PARAMETER Name The name of the DatastoreCluster .EXAMPLE PS> Get-StoragePod .EXAMPLE PS> Get-StoragePod -Name "SDC*" #> param( [CmdletBinding()] [parameter(Position = 0, ValueFromPipeline = $true)] [string]$Name = "*" ) begin{ function Get-StoragePodInternal{ param( [CmdletBinding()] [parameter(Mandatory = $true,ValueFromPipeline = $true)] [VMware.Vim.Folder]$Folder ) $Folder.ChildEntity | %{ if($_.Type -eq "StoragePod"){ Get-View -Id $_ } elseif($_.Type -eq "Folder"){ Get-View -Id $_ | Get-StoragePodInternal } } } } process{ Get-StoragePodInternal -Folder (Get-Folder -Name datastore).ExtensionData | where {$_.Name -like $Name} } } function Remove-DatastoreCluster{ <# .SYNOPSIS Remove a DatastoreCluster .DESCRIPTION The function will remove a DatastoreCluster. .NOTES Author: Luc Dekens .PARAMETER Name The DatastoreCluster to be removed. The parameter accepts a string, an object returned by the Get-DatastoreCluster cmdlet or a StoragePod object, returned by the Get-StoragePod function .EXAMPLE PS> Get-DatastoreCluster -Name "SDC" | Remove-DatastoreCluster .EXAMPLE PS> Remove-DatastoreCluster -Name $pod #> param( [CmdletBinding()] [parameter(Position = 0, Mandatory = $true, ValueFromPipeline = $true)] [PSObject]$Pod ) process{ if($Pod.GetType().Name -eq "string"){ $Pod = Get-DatastoreCluster -Name $Pod } elseif($Pod.GetType().Name -eq "DatastoreClusterImpl"){ $Pod = Get-StoragePod -Name $Pod.Name } $pod.Destroy() } } |
Annotations
Line 73: To configure a DatastoreCluster we need the StorageResourceManager
Line 76-82: A kind of Object By Name logic. This will allow the function to access the Pod parameter as a Name in a string, or as an object returned by the Get-DatastoreCluster cmdlet or as a StoragePod object.
Line 83-84: The Set-DatastoreCluster function can be used to change settings of the DatastoreCluster but also to add Datastores to the DatastoreCluster. To avoid calling the ConfigureStorageDrsForPod method when no settings have to be changed, these lines check if only the Datastore parameter was passed.
Line 85-119: These lines set up the parameters for the call to the ConfigureStorageDrsForPod method. Since the script calls this method in ‘incremetal’ mode, we only set the properties in the StorageDrsConfigSpec object that are actually set. Hence all the tests in this part of the code.
Line 122-130: This part of the function adds the Datastores to the DatastoreCluster with the MoveIntoFolder method.
Line 133: Since we want to use the functions in pipeline constructs, the function places the DatastoreCluster in the pipeline as a StoragePod object.
Line 186-207: Note that the New-DatastoreCluster function uses all the defaults that are defined in the description of the StorageDrsConfigSpec object in the VMware vSphere API Reference
Line 209-213: The Object By Name functionality for the Folder parameter.
Line 216: The creation of the DatastoreCluster through the CreateStoragePod method.
Line 218-223: The function calls the Set-DatastoreCluster function.
Line 224-226: When the call passed Datastores, these are added to the DatastoreCluster through a 2nd call to the Set-DatastoreCluster function.
Line 229: Since we want to use the functions in pipeline constructs, the function places the DatastoreCluster in the pipeline as a StoragePod object.
Line 232-276: Normally you don’t need to use this function since there is already the Get-DatastoreCluster cmdlet in PowerCLI. This function returns a StoragePod object and is primarily for internal use. An exception could be when you want to report on properties in your DatastoreCluster that are not in the object returned by the Get-DatastoreCluster cmdlet.
Line 254-269: The function uses a local function called GetStoragePodInternal in order to recursively traverse the vSphere tree.
Line 274: When the function is called with a specific Name, this Where-clause will filter out the desired StoragePod objects.
Line 300-305: A kind of Object By Name logic. This will allow the function to access the Pod parameter as a Name in a string, or as an object returned by the Get-DatastoreCluster cmdlet or as a StoragePod object.
Sample use
The following samples will show you how you can use these functions to set up, configure and remove a DatastoreCluster.
In my testlab I have set up the following simple environment to demonstrate the functions. It consists of 3 VMFS5 datastores that I want to use to create a DatastoreCluster.
First we will create a DatastoreCluster, called DSC, with 2 datastores, DS2 and DS3. The code to do that looks as follows.
1 |
New-DatastoreCluster -Name DSC -Datastore DS2,DS3 |
The result looks like this
Since we didn’t specify any parameters, besides the datastores, for the DatastoreCluster, the DatastoreCluster took all the defaults. As a guide, this is a table showing the defaults as used in the New-DatastoreCluster function.
Folder | datastore |
AutomationLevel | automatic |
IOMetric | True |
SpaceUtilization | 80% |
SpaceUtilizationDifference | 5% |
IOLatency | 15 |
IOLatencyDifference | 5 |
Interval | 480 |
Now we want to add the Datastore DS4 to this DatastoreCluster.
1 2 |
Get-StoragePod -Name DSC | Set-DatastoreCluster -Datastore (Get-Datastore -Name DS4) |
The result.
As a guide to understand which parameter that corresponds with which property, I made the following screenshots.
The General option.
The SDRS Automation option
The SDRS Runtime options
The other settings are currently not implemented in the functions.
The supported parameters can all be changed with the Set-DatastoreCluster function. Something like this for example.
1 2 |
Get-StoragePod -Name "DSC" | Set-DatastoreCluster -Interval 240 -SpaceUtilization 70 |
This call changes the SDRS interval to 240 minutes or 4 hours (instead of the default 8 hours). And it changes the Utilized Space threshold to 70% (instead of the default 80%).
To remove a DatastoreCluster, you can do
1 |
Get-StoragePod -Name "DSC" | Remove-DatastoreCluster |
Enjoy the functions !
Matt
Luc,
Does it work in case, there are multiple datacenters in the VC-inventory ..?
LucD
@Matt, do you mean the Get-StoragePod function ? Yes, that should return multiple datastoreclusters.
Jonathan Marrott
Any way to do a Move-VM into a Datastore Cluster?
LucD
@Jonathan, the current PowerCLI build (5.0.1) only supports Datastore Clusters on the New-VM and the New-Harddisk cmdlets.
But I suspect that is a vSphere limitation, the RelocateVM_Task method also doesn’t seem to support a Storage Cluster.
Rick van Wijnen
Luc,
Thanks for the excellent presentation at VMUG 2011.
Another topic I’m very much interested in is your “find the vCenter server” script. Is that powershell script public available?
We had such event, next time I want to be prepared 🙂
Thanks again,
Rick.
LucD
@Rick, thanks.
The “find the vCenter” script has been posted on Alan’s site as VM Start-up script