Automate SIOC

With vSphere 4.1 came 150+ new features. One of these is called Storage IO Control or SIOC.And it has been a very popular subject in the last weeks. Just a small selection of blog posts on the subject:

The only thing missing was a way to automate everything surrounding SIOC. And so I decided to write a couple of functions to fill that gap.

The script

Annotations

Line 27-34: Since you have SIOC settings on a Datastore and on a VMHost, I use 2 parameter sets.

Line 37: Via the $PsCmdlet.ParameterSetName the script can find out with which parameter set the function was called.

Line 39: When the function was called with the VMHost parameter set, we need the ServiceInstance object to get to the StorageResourceManager.

Line 48-50: The StorageResourceManager object is not included in the PowerCLI 4.1 framework. The script uses the method, I also used in Script vSphere 4.1 AD Authentication to get to the StorageResourceManager object and use the QueryIORMConfigOption method. Thanks again to Yasen for providing us with this method.

Line 62: If the function is called with the Datastore parameter set, the SIOC properties can be found in the StorageIORMInfo object.

Line 102: Notice that the script uses a default threshold of 30 ms, which is apparently the default except for SSD storage.

Line 113-116: Again, to call the ConfigureDatastoreIORM_Task method, the script uses the method provided by Yasen.

Samples

The Get-SIOC function can be used with two parameter sets. First the VMHost parameter set will retrieve the SIOC settings on a specific host.

This will return the SIOC settings for that specific host. Your output will look something like this.

You can of course use the Get-SIOC function a pipeline as well.

This will give a nice tabular overview

With the Datastore parameter set you get the SIOC settings for a specific datastore

And again this can be used in a pipeline

To change your SIOC settings on a datastore you can do

Or in a pipeline

Notice that the Set-SIOC function doesn’t return anything !

27 Comments

    Hernan Rojas

    Usually, I don’t spend time writing reviews or comments but your post has been so useful that I take the time to say “thank you”.

    great piece of code and with examples, those examples are priceless.

    thanks again.

    Hernan
    Bogotá-Col

      LucD

      Thanks Hernan

    Paul

    @LucD

    @LucD

    That snippet of code has been incredibly useful in managing IOPS greedy VMs in our environment. So thanks very much.

    A question, how would it be possible to report on IOPS limits instead of reconfiguring?

    Regards

    Paul

    Oleg

    @LucD
    Luc,

    No the code was correct (I double check it with your book). However, I did not have PowerShell 2.0 Now everything is working.
    Thanks a lot!

    Oleg

    Luc,

    I know I ran this script some time ago (maybe one year ago) and it worked just fine. However, now I have errors when I run it in both functions “get-sioc” and “set-sioc”.

    When I’m running “Get-SIOC” function I have the following error:

    Parameter declarations are a comma-separated list of variable names with optional initializer expressions.
    At :line:6 char:34
    + HelpMessage = “Enter a host”)]

    and when I’m running “Set-SIOC” function I have the following error:

    Missing closing ‘)’ in expression.
    At :line:78 char:3
    + [ <<<< VMware.VimAutomation.ViCore.Impl.V1.DatastoreManagement.DatastoreImpl[]]

    I checked and rechecked everything, but could not find the issue.
    Could it be because I am running newer version of PowerCLI/PowerShell?

    Please help.

    Thank you

      LucD

      @Oleg, are you sure the copy/paste of the code worked correctly ?
      It looks as if there is a comma missing on the 2nd line of this part in the Get-Sioc function.

      [parameter(ParameterSetName = "VMHost",
      valuefrompipeline = $true,position = 0,
      HelpMessage = "Enter a host")]

    victor

    hello,

    The ReconfigVM_Task script works great; thanks 🙂

    do you know how to modify it to change the IOPS limit for multiple VMs, e.g. the vm names start with webservers (webservers01, webservers02, etc), a wildcard vm name webservers* throws an error ?

    thanks!

      LucD

      @Victor, I’m not sure if I understood your question correctly, but the activation of SIOC and the setting of the thresholds is done for Datastores, not VMs.
      And in the latest PowerCLI build, the Set-Datastore cmdlet offers this option as well.

    Stephen

    This has been added to the Set-Datastore cmdlet in PowerCLI 5.0.0. Thus, you can set the SIOC with the one liner below:

    Get-Datastore “DS*” | Set-Datastore -CongestionThresholdMillisecond 30 -StorageIOControlEnabled $true

      LucD

      @Stephen, that is correct (there often appear features in newer builds for which I wrote a function 😉 ).
      I will leave the post up because the Get-Sioc functon returns more than the Get-Datastore cmdlet, and because there are some interesting scripting techniques in the functions.

    Justin

    @LucD

    To get it working on vSphere 5, I assume Olef’s simple fix is to change:

    [VMWare.Vim.VIConvert]::ToVim41($si.Content.StorageResourceManager),
    [VMWare.Vim.VIConvert]::ToVim41($_.Extensiondata.MoRef),
    [VMWare.Vim.VIConvert]::ToVim41($spec))

    to:

    [VMWare.Vim.VIConvert]::ToVim50($si.Content.StorageResourceManager),
    [VMWare.Vim.VIConvert]::ToVim50($_.Extensiondata.MoRef),
    [VMWare.Vim.VIConvert]::ToVim50($spec))

      LucD

      @Justin, yes that should do the trick.

    Oleg

    @Oleg

    Luc,

    never mind. I fixed myself (was really easy)

    Thanks again for the script

      LucD

      @Oleg, would you mind sharing the solution ?
      Perhaps I can update the function to work for both vSphere versions.

    Oleg

    Luc,

    Your script work just fine for vSphere 4.1, but when I ran it against vSphere 5.0 I got the following error:

    Cannot convert argument “0”, with value: “VimApi_41.ManagedObjectReference”, for “ConfigureDatastoreIORM_Task” to type “VimApi_50.ManagedObjectReference”: “Cannot convert the “VimApi_41.ManagedObjectReference” value of type “VimApi_41.ManagedObjectReference” to type “VimApi_50.ManagedObjectReference”.”

    + $taskMoRef = $si.Client.VimService.ConfigureDatastoreIORM_Task <<<< (

    I updated VMware PowerCLI to v.5.01, but it did not help.

    Can you please tell me what should I change?

    Thanks a lot,
    Oleg

    Oleg

    Luc,

    Great script, thanks a lot!!!

    However, when I run:

    Get-VMHost | Get-Sioc | Where-Object {$_.SIOCSupported -and -not $_.SIOCEnabled}

    I got the following errors for each host I have on the VC:

    You cannot call a method on a null-valued expression.
    At line:27 char:60
    + $result = $si.Client.VimService.QueryIORMConfigOption <<<< (
    + CategoryInfo : InvalidOperation: (QueryIORMConfigOption:String) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

    The line 27 at my script is line 47 at yoru script.

    However, when I run Get-SIOC (Get-VMHost) I got similar error only for few hosts. Here is the error:

    Exception calling "QueryIORMConfigOption" with "2" argument(s): "The operation is not supported on the object.
    "
    At line:27 char:60
    + $result = $si.Client.VimService.QueryIORMConfigOption <<<< (
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

    What is wrong with my hosts?

    Thank you,
    Oleg

    David

    @LucD
    Hi Luc

    Ahh my misunderstanding of the concept and what was happening here. Appreciate the guidance.

    Cheers
    David

    David

    Hi Luc

    I was trying to see what the Get-Sioc command does in relation to your script, however when running against VMware vSphere PowerCLI 4.1 U1 build 332441, I get the following error “The term Get-SIOC is not recognised as the name of the cmdlet”. Other command work fine, and listing all the commands via GET-VICommand doesnt show the Get-SIOC option. Has it been removed from this version do you know? my host servers are vSphere 4.1 update 2.

    Thanks
    David

      LucD

      @David, I think there is a slight misunderstanding here. The PowerCLI tool never had a Get-Sioc cmdlet.
      That’s why I wrote this function.
      What I tried to say in the opening paragraph is that with vSphere 4.1 SIOC was introduced, but not that there was a Get-Sioc cmdlet in PowerCLI.

    Robert van den Nieuwendijk

    While trying to use the Set-Sioc function in my environment with three vCenter servers connected, I got some errors. Looking at the code I saw that it expects that you have only one vCenter server connected. So I made some small changes to both functions to solve this problem and let the functions work with any number of vCenter servers connected.

    In both functions I changed the line (lines 39 and 105):

    $si = Get-View ServiceInstance

    into:

    $sis = @(Get-View ServiceInstance)

    to make sure that it always returns an array.

    Before the lines containing “$si.Client” (lines 48 and 113) I inserted the following two lines:

    $vCenterServer = $_.Uid.Split(“:”)[0].Split(“@”)[1]
    $si = $sis | Where-Object {$_.Client.ServiceUrl -like “*$vCenterServer*”}

    This solved the problem. Both functions now work fine with any number of vCenter servers connected.

      LucD

      Great improvement. Thanks for sharing this.

    marc

    Hi Luc,

    Slightly off-topic but just curious of your experiences in the field with the “get-vm” command the very first time its invoked? What are typical execute durations based on # of VMs in the vCenter inventory? What is the best way to speed up execution time *specifically* on the first time the cmdlet is executed? More memory, more CPU? Does PowerCLI work well on a 2 vCPU VM compared to a 1? Is longer than 20 seconds typical for the get-vm on its first run?

    Finally, in your opinion, what’s the most optimum code for retrieving all VM names, # of CPU and amount of RAM each VM has in a given vCenter’s inventory?

    Regards
    marc

    Ali

    hello guys,

    is it possible to get host I/O data thought powercli ? any one got script ? thanks

    Br, Ali

    marc

    @LucD
    Thanks Luc.

    marc

    Hi Luc,

    How do I set the “Limit – IOPs” for the VM “vm48” on all the virtual disks it has attached to it using PowerCLI?

    (Manually this is done via vCenter > Select “vm48” > Edit Settings > Resources > Disk > Limit – IOPs)

    regards
    marc

      LucD

      @Marc, unfortunately there is currently no -DiskLimitIOPS parameter on the Set-VMResourceConfiguration cmdlet.
      But with the ReconfigVM_Task method we can set the limits.

      $vmName = "MyVM"
      $DiskLimitIOPS = 1150
      $vm = Get-VM -Name $vmName
      $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
      $vm | Get-HardDisk | %{
      $dev = New-Object VMware.Vim.VirtualDeviceConfigSpec
      $dev.Operation = "edit"
      $dev.device = $_.Extensiondata
      $dev.device.storageIOAllocation.limit = $DiskLimitIOPS
      $spec.deviceChange += $dev
      }
      $vm.ExtensionData.ReconfigVM($spec)

      Note that this will set the same IOPS limit on each harddisk of your guest.
      If you want to set different IOPS limits per harddisk, then you can use the Where-Object cmdlet to filter out the harddisks you want.

      To set the limit back to Unlimited you specify the limit as -1

      Final note, be careful when applying limits !

    Tomi Hakala

    Wow, just what I need. Thank you!

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.