Automate SIOC

With vSphere 4.1 came 150+ new features. One of these is called Storage IO Control or SIOC.And it has been a very popular subject in the last weeks. Just a small selection of blog posts on the subject:

The only thing missing was a way to automate everything surrounding SIOC. And so I decided to write a couple of functions to fill that gap.

The script

Annotations

Line 27-34: Since you have SIOC settings on a Datastore and on a VMHost, I use 2 parameter sets.

Line 37: Via the $PsCmdlet.ParameterSetName the script can find out with which parameter set the function was called.

Line 39: When the function was called with the VMHost parameter set, we need the ServiceInstance object to get to the StorageResourceManager.

Line 48-50: The StorageResourceManager object is not included in the PowerCLI 4.1 framework. The script uses the method, I also used in Script vSphere 4.1 AD Authentication to get to the StorageResourceManager object and use the QueryIORMConfigOption method. Thanks again to Yasen for providing us with this method.

Line 62: If the function is called with the Datastore parameter set, the SIOC properties can be found in the StorageIORMInfo object.

Line 102: Notice that the script uses a default threshold of 30 ms, which is apparently the default except for SSD storage.

Line 113-116: Again, to call the ConfigureDatastoreIORM_Task method, the script uses the method provided by Yasen.

Samples

The Get-SIOC function can be used with two parameter sets. First the VMHost parameter set will retrieve the SIOC settings on a specific host.

This will return the SIOC settings for that specific host. Your output will look something like this.

You can of course use the Get-SIOC function a pipeline as well.

This will give a nice tabular overview

With the Datastore parameter set you get the SIOC settings for a specific datastore

And again this can be used in a pipeline

To change your SIOC settings on a datastore you can do

Or in a pipeline

Notice that the Set-SIOC function doesn’t return anything !

31 Comments

    Oleg

    February 5, 2019 at 18:32

    Hi Luc,

    I was using your script just fine till I recently had to move to new laptop and looks like I am missing something now (either module or using a wrong version or something similar).

    Anyway, when I ‘m trying to run your script against ESXi 5.5 (and I know it worked before) I’m getting the following error:
    Method invocation failed because [VMware.Vim.VIConvert] does not contain a method named ‘ToVim55’.
    At line:57 char:8
    + $taskMoRef = $si.Client.VimService.ConfigureDatastoreIORM_Task(
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : MethodNotFound

    Get-View : Cannot validate argument on parameter ‘VIObject’. The argument is null or empty. Provide an argument that is not null or empty, and then try the command
    again.
    At line:61 char:25
    + $task = Get-View ([VMWare.Vim.VIConvert]::ToVim($taskMoRef))
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidData: (:) [Get-View], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIView

    And this is what I have in the code:

    process{
    $Datastore | %{
    $taskMoRef = $si.Client.VimService.ConfigureDatastoreIORM_Task(
    [VMWare.Vim.VIConvert]::ToVim55($si.Content.StorageResourceManager),
    [VMWare.Vim.VIConvert]::ToVim55($_.Extensiondata.MoRef),
    [VMWare.Vim.VIConvert]::ToVim55($spec))
    $task = Get-View ([VMWare.Vim.VIConvert]::ToVim($taskMoRef))
    while (“running”,”queued” -contains $task.Info.State){
    $task.UpdateViewData(“Info.State”)
    }
    }
    }

    Can you please point what am I missing now?

    Thanks a lot!

    Reply

      LucD

      February 5, 2019 at 20:06

      Hi Oleg,
      Which PowerCLI version are you using?
      Since vSphere went out of support last year, the 5.5 methods have been removed from PowerCLI.
      You could install a PowerCLI version pre-September 2018, or try with the following updated code, which doesn’t use that Convert anymore.


      function Set-Sioc{
      <# .SYNOPSIS Enables/disables Storage IO Control for a a datastore .DESCRIPTION The function enables or disables SIOC for a datastore. .NOTES Authors: Luc Dekens .PARAMETER Datastore On or more datastores .PARAMETER Enabled A switch that defines if SIOC will be enabled or disabled .PARAMETER Threshold Specify the threshold .EXAMPLE PS> Set-Sioc -Datastore (Get-Datastore -Name "DS1") -Enabled:$true
      .EXAMPLE
      PS> Get-Datastore | Set-SIOC -Enabled:$true -Threshold 30
      .EXAMPLE
      PS> Get-Datastore | Set-SIOC -Enabled:$false
      #>
      param(
      [parameter(
      valuefrompipeline = $true,position = 0,
      HelpMessage = "Enter a datastore")]
      [VMware.VimAutomation.ViCore.Impl.V1.DatastoreManagement.DatastoreImpl[]]$Datastore,
      [switch]$Enabled,
      [int]$Threshold = 30
      )
      begin{
      $si = Get-View ServiceInstance
      $spec = New-Object VMware.Vim.StorageIORMConfigSpec
      $spec.congestionThreshold = $Threshold
      $spec.enabled = $Enabled
      }

      process{
      $Datastore | %{
      $storMgr = Get-View -Id $si.Content.StorageResourceManager
      $storMgr.ConfigureDatastoreIORM($_.ExtensionData.MoRef,$spec)
      }
      }
      }

      function Get-Sioc{
      <# .SYNOPSIS Get the Storage IO Control settings for a host or a datastore .DESCRIPTION When called against a VMHost, the cmdlet will return the SIOC settings on the host When called against a Datastore, the cmdlet will return the SIOC settings for all hosts on which the datastore is accessible .NOTES Authors: Luc Dekens .PARAMETER VMHost On or more hosts .PARAMETER Datastore On or more datastores .EXAMPLE PS> Get-Sioc -VMHost (Get-VMHost)
      .EXAMPLE
      PS> Get-VMHost -Name "esx41" | Get-Sioc
      .EXAMPLE
      PS> Get-Sioc -Datastore (Get-Datastore)
      .EXAMPLE
      PS> Get-Datastore -Name "DS1" | Get-Sioc
      #>
      param(
      [parameter(ParameterSetName = "VMHost",
      valuefrompipeline = $true,position = 0,
      HelpMessage = "Enter a host")]
      [VMware.VimAutomation.ViCore.Impl.V1.Inventory.VMHostImpl[]]$VMHost,
      [parameter(ParameterSetName = "Datastore",
      valuefrompipeline = $true, position = 0,
      HelpMessage = "Enter a datastore")]
      [VMware.VimAutomation.ViCore.Impl.V1.DatastoreManagement.DatastoreImpl[]]$Datastore
      )
      begin{
      switch($PsCmdlet.ParameterSetName){
      "VMHost"{
      $si = Get-View ServiceInstance
      }
      "Datastore"{}
      }
      }
      process{
      switch($PsCmdlet.ParameterSetName){
      "VMHost"{
      $VMHost | %{
      $storMgr = Get-View -Id $si.Content.StorageResourceManager
      $result = $storMgr.QueryIORMConfigOption($_.ExtensionData.MoRef)
      $row = "" | Select Name,SIOCSupported,SIOCStateDefault,
      SIOCThresholdMinimum,SIOCThresholdMaximum,SIOCThresholdDefault
      $row.Name = $_.Name
      $row.SIOCSupported = $result.enabledOption.supported
      $row.SIOCStateDefault = $result.enabledOption.defaultValue
      $row.SIOCThresholdMinimum = $result.congestionThresholdOption.min
      $row.SIOCThresholdMaximum = $result.congestionThresholdOption.max
      $row.SIOCThresholdDefault = $result.congestionThresholdOption.defaultValue
      $row
      }
      }
      "Datastore"{
      $Datastore | %{
      $row = "" | Select Name,SIOCEnabled,SIOCThreshold
      $row.Name = $_.Name
      $row.SIOCEnabled = $_.Extensiondata.iormConfiguration.enabled
      $row.SIOCThreshold = $_.Extensiondata.iormConfiguration.congestionThreshold
      $row
      }
      }
      }
      }
      }

      Let me know if the new code works for you, then I can update the post.

      Reply

        Oleg

        February 6, 2019 at 16:10

        Luc,

        Thank you very much!!! Now it’s working like a champ!!!

        Thanks again for such a great script!

        All the best!

        (just get rid of space in the second line “< # .SYNOPSIS Enables/disables Storage IO Control" between "<" and "#", we know you just missed it)

        Reply

          LucD

          February 6, 2019 at 16:20

          Thanks for confirming, I’ll update the post.

          Reply

    Hernan Rojas

    September 13, 2018 at 18:04

    Usually, I don’t spend time writing reviews or comments but your post has been so useful that I take the time to say “thank you”.

    great piece of code and with examples, those examples are priceless.

    thanks again.

    Hernan
    Bogotá-Col

    Reply

      LucD

      September 20, 2018 at 14:26

      Thanks Hernan

      Reply

    Paul

    November 26, 2012 at 13:44

    @LucD

    @LucD

    That snippet of code has been incredibly useful in managing IOPS greedy VMs in our environment. So thanks very much.

    A question, how would it be possible to report on IOPS limits instead of reconfiguring?

    Regards

    Paul

    Reply

    Oleg

    June 7, 2012 at 16:22

    @LucD
    Luc,

    No the code was correct (I double check it with your book). However, I did not have PowerShell 2.0 Now everything is working.
    Thanks a lot!

    Reply

    Oleg

    June 6, 2012 at 22:22

    Luc,

    I know I ran this script some time ago (maybe one year ago) and it worked just fine. However, now I have errors when I run it in both functions “get-sioc” and “set-sioc”.

    When I’m running “Get-SIOC” function I have the following error:

    Parameter declarations are a comma-separated list of variable names with optional initializer expressions.
    At :line:6 char:34
    + HelpMessage = “Enter a host”)]

    and when I’m running “Set-SIOC” function I have the following error:

    Missing closing ‘)’ in expression.
    At :line:78 char:3
    + [ <<<< VMware.VimAutomation.ViCore.Impl.V1.DatastoreManagement.DatastoreImpl[]]

    I checked and rechecked everything, but could not find the issue.
    Could it be because I am running newer version of PowerCLI/PowerShell?

    Please help.

    Thank you

    Reply

      LucD

      June 6, 2012 at 22:31

      @Oleg, are you sure the copy/paste of the code worked correctly ?
      It looks as if there is a comma missing on the 2nd line of this part in the Get-Sioc function.

      [parameter(ParameterSetName = "VMHost",
      valuefrompipeline = $true,position = 0,
      HelpMessage = "Enter a host")]

      Reply

    victor

    May 27, 2012 at 17:53

    hello,

    The ReconfigVM_Task script works great; thanks 🙂

    do you know how to modify it to change the IOPS limit for multiple VMs, e.g. the vm names start with webservers (webservers01, webservers02, etc), a wildcard vm name webservers* throws an error ?

    thanks!

    Reply

      LucD

      May 27, 2012 at 19:38

      @Victor, I’m not sure if I understood your question correctly, but the activation of SIOC and the setting of the thresholds is done for Datastores, not VMs.
      And in the latest PowerCLI build, the Set-Datastore cmdlet offers this option as well.

      Reply

    Stephen

    May 20, 2012 at 23:53

    This has been added to the Set-Datastore cmdlet in PowerCLI 5.0.0. Thus, you can set the SIOC with the one liner below:

    Get-Datastore “DS*” | Set-Datastore -CongestionThresholdMillisecond 30 -StorageIOControlEnabled $true

    Reply

      LucD

      May 21, 2012 at 07:22

      @Stephen, that is correct (there often appear features in newer builds for which I wrote a function 😉 ).
      I will leave the post up because the Get-Sioc functon returns more than the Get-Datastore cmdlet, and because there are some interesting scripting techniques in the functions.

      Reply

    Justin

    April 2, 2012 at 17:56

    @LucD

    To get it working on vSphere 5, I assume Olef’s simple fix is to change:

    [VMWare.Vim.VIConvert]::ToVim41($si.Content.StorageResourceManager),
    [VMWare.Vim.VIConvert]::ToVim41($_.Extensiondata.MoRef),
    [VMWare.Vim.VIConvert]::ToVim41($spec))

    to:

    [VMWare.Vim.VIConvert]::ToVim50($si.Content.StorageResourceManager),
    [VMWare.Vim.VIConvert]::ToVim50($_.Extensiondata.MoRef),
    [VMWare.Vim.VIConvert]::ToVim50($spec))

    Reply

      LucD

      April 2, 2012 at 19:07

      @Justin, yes that should do the trick.

      Reply

    Oleg

    February 27, 2012 at 21:21

    @Oleg

    Luc,

    never mind. I fixed myself (was really easy)

    Thanks again for the script

    Reply

      LucD

      February 27, 2012 at 21:27

      @Oleg, would you mind sharing the solution ?
      Perhaps I can update the function to work for both vSphere versions.

      Reply

    Oleg

    February 27, 2012 at 17:37

    Luc,

    Your script work just fine for vSphere 4.1, but when I ran it against vSphere 5.0 I got the following error:

    Cannot convert argument “0”, with value: “VimApi_41.ManagedObjectReference”, for “ConfigureDatastoreIORM_Task” to type “VimApi_50.ManagedObjectReference”: “Cannot convert the “VimApi_41.ManagedObjectReference” value of type “VimApi_41.ManagedObjectReference” to type “VimApi_50.ManagedObjectReference”.”

    + $taskMoRef = $si.Client.VimService.ConfigureDatastoreIORM_Task <<<< (

    I updated VMware PowerCLI to v.5.01, but it did not help.

    Can you please tell me what should I change?

    Thanks a lot,
    Oleg

    Reply

    Oleg

    January 31, 2012 at 18:17

    Luc,

    Great script, thanks a lot!!!

    However, when I run:

    Get-VMHost | Get-Sioc | Where-Object {$_.SIOCSupported -and -not $_.SIOCEnabled}

    I got the following errors for each host I have on the VC:

    You cannot call a method on a null-valued expression.
    At line:27 char:60
    + $result = $si.Client.VimService.QueryIORMConfigOption <<<< (
    + CategoryInfo : InvalidOperation: (QueryIORMConfigOption:String) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

    The line 27 at my script is line 47 at yoru script.

    However, when I run Get-SIOC (Get-VMHost) I got similar error only for few hosts. Here is the error:

    Exception calling "QueryIORMConfigOption" with "2" argument(s): "The operation is not supported on the object.
    "
    At line:27 char:60
    + $result = $si.Client.VimService.QueryIORMConfigOption <<<< (
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

    What is wrong with my hosts?

    Thank you,
    Oleg

    Reply

    David

    November 9, 2011 at 20:29

    @LucD
    Hi Luc

    Ahh my misunderstanding of the concept and what was happening here. Appreciate the guidance.

    Cheers
    David

    Reply

    David

    November 9, 2011 at 11:45

    Hi Luc

    I was trying to see what the Get-Sioc command does in relation to your script, however when running against VMware vSphere PowerCLI 4.1 U1 build 332441, I get the following error “The term Get-SIOC is not recognised as the name of the cmdlet”. Other command work fine, and listing all the commands via GET-VICommand doesnt show the Get-SIOC option. Has it been removed from this version do you know? my host servers are vSphere 4.1 update 2.

    Thanks
    David

    Reply

      LucD

      November 9, 2011 at 13:08

      @David, I think there is a slight misunderstanding here. The PowerCLI tool never had a Get-Sioc cmdlet.
      That’s why I wrote this function.
      What I tried to say in the opening paragraph is that with vSphere 4.1 SIOC was introduced, but not that there was a Get-Sioc cmdlet in PowerCLI.

      Reply

    Robert van den Nieuwendijk

    June 28, 2011 at 19:46

    While trying to use the Set-Sioc function in my environment with three vCenter servers connected, I got some errors. Looking at the code I saw that it expects that you have only one vCenter server connected. So I made some small changes to both functions to solve this problem and let the functions work with any number of vCenter servers connected.

    In both functions I changed the line (lines 39 and 105):

    $si = Get-View ServiceInstance

    into:

    $sis = @(Get-View ServiceInstance)

    to make sure that it always returns an array.

    Before the lines containing “$si.Client” (lines 48 and 113) I inserted the following two lines:

    $vCenterServer = $_.Uid.Split(“:”)[0].Split(“@”)[1]
    $si = $sis | Where-Object {$_.Client.ServiceUrl -like “*$vCenterServer*”}

    This solved the problem. Both functions now work fine with any number of vCenter servers connected.

    Reply

      LucD

      June 28, 2011 at 20:11

      Great improvement. Thanks for sharing this.

      Reply

    marc

    April 17, 2011 at 12:07

    Hi Luc,

    Slightly off-topic but just curious of your experiences in the field with the “get-vm” command the very first time its invoked? What are typical execute durations based on # of VMs in the vCenter inventory? What is the best way to speed up execution time *specifically* on the first time the cmdlet is executed? More memory, more CPU? Does PowerCLI work well on a 2 vCPU VM compared to a 1? Is longer than 20 seconds typical for the get-vm on its first run?

    Finally, in your opinion, what’s the most optimum code for retrieving all VM names, # of CPU and amount of RAM each VM has in a given vCenter’s inventory?

    Regards
    marc

    Reply

    Ali

    February 21, 2011 at 09:49

    hello guys,

    is it possible to get host I/O data thought powercli ? any one got script ? thanks

    Br, Ali

    Reply

    marc

    February 14, 2011 at 06:34

    @LucD
    Thanks Luc.

    Reply

    marc

    February 13, 2011 at 06:30

    Hi Luc,

    How do I set the “Limit – IOPs” for the VM “vm48” on all the virtual disks it has attached to it using PowerCLI?

    (Manually this is done via vCenter > Select “vm48” > Edit Settings > Resources > Disk > Limit – IOPs)

    regards
    marc

    Reply

      LucD

      February 13, 2011 at 14:49

      @Marc, unfortunately there is currently no -DiskLimitIOPS parameter on the Set-VMResourceConfiguration cmdlet.
      But with the ReconfigVM_Task method we can set the limits.

      $vmName = "MyVM"
      $DiskLimitIOPS = 1150
      $vm = Get-VM -Name $vmName
      $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
      $vm | Get-HardDisk | %{
      $dev = New-Object VMware.Vim.VirtualDeviceConfigSpec
      $dev.Operation = "edit"
      $dev.device = $_.Extensiondata
      $dev.device.storageIOAllocation.limit = $DiskLimitIOPS
      $spec.deviceChange += $dev
      }
      $vm.ExtensionData.ReconfigVM($spec)

      Note that this will set the same IOPS limit on each harddisk of your guest.
      If you want to set different IOPS limits per harddisk, then you can use the Where-Object cmdlet to filter out the harddisks you want.

      To set the limit back to Unlimited you specify the limit as -1

      Final note, be careful when applying limits !

      Reply

    Tomi Hakala

    October 20, 2010 at 19:01

    Wow, just what I need. Thank you!

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.