VMFS Datastores – Expand and Extend

We all know, and love, PowerCLI‘s New-Datastore and Set-Datastore cmdlets to create and manipulate VMFS datastores. But when we look at the functionality available through the Web Client, there is one interesting feature for manipulating VMFS datastores that is missing from the PowerCLI cmdlets. The Increase button, which allows us to Expand or Extend an existing VMFS datastore*.

DS-Increase

Recently there were a couple of threads on this subject in the VMTN PowerCLI Community, so I decided to streamline my quick-and-dirty scripts into something more presentable, and create a PowerShell module to bundle the functions. I present the VMFSIncrease module!
The VMFSIncrease module will also be my first contribution to the PowerCLI Community Repository! More on that further on in this post.

 

 

* The expand and extend functions for a VMFS datastore depend on the availability of free space on the VMFS datastore extents and/or the availability of free LUNs

The Increase button

The Increase button in fact offers two different methods to increase the size of a datastore. The definitions of two methods according to the

  • Expand: increases the capacity of an existing VMFS datastore by expanding (increasing the size of) an existing extent of the datastore
  • Extend: Increases the capacity of an existing VMFS datastore by adding new extents to the datastore

Perhaps the following schematic makes it a bit more clear.

vmfs-2

The schematic shows a VMFS datastore with one extent, on LUN A. If we want to increase the Used capacity, we can do an Expand, on the same extent (LUN A). Or we can do an Extend on a new LUN, in this case LUN B.In the case of the Extend, the VMFS datastore will now have two extents, LUN A and LUN B.

Note that for the Expand and the Extent you do not need to use all of the free space. The following is perfectly possible.

vmfs-1

The Increase button presents the user with a series of candidate LUNs. The Expand and Extend candidate LUNs are intermixed in that list. The underlying API method makes sure that the LUNs are not already used by another datastore or by a RDM disk.

The Module

The module is available in my GitHub VMFSIncrease repository.

Sample runs

Reporting

The Get-VmfsDatastoreInfo displays basic information about the extents and the partitions that make up the datastore.

This produces something like this

vmfs-3

The Datastore lives on one extent, and on that extent there are three partitions. The partition marked with (2) is the part where the VMFS Datastore lives. Part (3) is the free space, and part (1) is for alignment.

To check how we can increase the capacity of the Datastore, we can run

This returns something like this

vmfs-4

The returned information shows that there are two options to increase the capacity of the Datastore. In (1) we see that we can Expand the extent by 80GB. In (2) we see that we can Extend the capacity by 100GB by adding a new extent to the Datastore.

Expand

The Help page for the New-VmfsDatastoreIncrease function already contains several examples on how to use the function. But to give you an idea what is possible, just a few examples.

We have the possibility to add up to 80GB. If we don’t specify a size, all the free space on the extent is used.

If we do specify how much we want to increase the capacity, only that specific amount of space from the free space will be taken.

If we have a Datastore that spans more than one extent, we can, through the Canonicalname parameter, indicate on which extent the increase shall be taken.

Extend

As we explained in the beginning of the post, with the Extend switch we indicate that we want to increase the capacity on a new extent. Again we can take the full extent.

Interesting to note, if you don’t specify a LUN, through the Canonicalname parameter, the function will order the available LUNs alphanumerically on their Canonicalname, and take the first one. So the following is perfectly possible, provided of course there are free LUNs available.

And, as with the Expand option, you do not need to use all of the available space with the Extend option. You can control how much space is taken through the IncreaseSizeGB parameter.

The option is also available without specifying a LUN. The function uses the same logic as was mentioned before, it will order the available LUNs alphanumerically on their Canonicalname, and take the first one.

PowerCLI Community Repository

Since a couple of days the long awaited PowerCLI Community Repository is live! More info can be found in the Updating the VMware PowerCLI Community Repository! post over on the PowerCLI Blog.

If you have any scripts, functions, modules… that might be useful for the community, please submit them there.

The VMFSIncrease module will be available on the PowerCLI Community Repository shortly.

 

Enjoy, and share!

 

14 Comments

    Josef Zach

    Hi, module did not work.

    I have single esxi host with hpe smart array controller (raid 1+0) expanded with 2 more drives. On this same raid was esxi installed and also VMFS 5 (esxi 6.7u1).

    I think there was issue with backup gpt partition which was not moved to the end of free space (just thought..)-

    Get-VmfsDatastoreInfo -Datastore TestDS did not return free space as expand/extend but NONE.

    expand/extend command return issue – there is no free space…

    Cody Hosterman workaround mentioned above works.

    LucD module helps me many time before..

      LucD

      Thanks, I’ll have a look if I can improve the code to deal with this

    Mike

    Hi,
    I have to automate the resizing of datastore in ruby.I performed a manual resizing through vcenter
    and in vcenter logs(vpxa.logs) I found the below apis are calling in sequence–
    vim.host.StorageSystem.retrieveDiskPartitionInfo
    vim.host.StorageSystem.computeDiskPartitionInfoForResize
    vim.host.DatastoreSystem.expandVmfsDatastore

    I assume that if these apis will be called in same sequnece, the resizing can be automated.
    Can I get some sample code regarding this.

    Regards,
    Mike

      LucD

      Hi Mike,
      Sorry, but I’m afraid I’m a noob concerning Ruby.

    james

    Hello LucD,

    Tested on my system, failed to increase, got following errors:
    PS C:\> Get-VmfsDatastoreIncrease -Datastore ‘Prod_Unity_DS03’
    PS C:\> New-VmfsDatastoreIncrease -Datastore ‘Prod_Unity_DS03’ -Expand -IncreaseSizeGB 200
    Exception calling “RetrieveDiskPartitionInfo” with “1” argument(s): ”
    Required parameter devicePath is missing
    while parsing call information for method RetrieveDiskPartitionInfo
    at line 1, column 171
    while parsing SOAP body
    at line 1, column 64
    while parsing SOAP envelope
    at line 1, column 0
    while parsing HTTP request for method retrieveDiskPartitionInfo
    on object of type vim.host.StorageSystem
    at line 1, column 0″
    At C:\Users\222222\Documents\WindowsPowerShell\Modules\VMFSIncrease-master\VMFSIncrease.psm1:183 char:6
    + … $partInfo = $hsSys.RetrieveDiskPartitionInfo($lun.DeviceN …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : VimException

    Cannot index into a null array.
    At C:\Users\222222\Documents\WindowsPowerShell\Modules\VMFSIncrease-master\VMFSIncrease.psm1:186 char:6
    + … $partUsed = ($partInfo[0].Layout.Partition | where{ $_.Ty …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

    Exception calling “ExpandVmfsDatastore” with “2” argument(s): “Cannot change the host configuration.”
    At C:\Users\222222\Documents\WindowsPowerShell\Modules\VMFSIncrease-master\VMFSIncrease.psm1:203 char:5
    + … $hdSys.ExpandVmfsDatastore($Datastore.ExtensionData.MoRef …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : VimException

      LucD

      Hi James,
      From the error message, it looks as if the method RetrieveDiskPartitionInfo has an issue finding partition information.

      What kind of LUNs are you working with?
      Are the FC LUNs?

      Also, do you see the Increase button in the Web Client?
      Select the datastore tab, then the datastore you are trying to increase and then the Configure tab.
      Under the General option you should see an Increase button in the righ panel.

        james

        Thank you very much for the quick reply, I am using EMC Unity FC LUNs, the
        Get-VmfsDatastoreInfo -Datastore Prod_Unity_DS03 working fine, I saw 3 partitions, which match exactly with what you have described here. I tested both PowerShell 5.1.18362.145 and 6.2.3. The vCenter version is 6.5.0.

        BTW, all the ESXi hosts and vCenter see the new LUN 5TB already, the vmfs datastore is 4TB now.

          LucD

          Thanks for the feedback.

          Do you see the Increase button in the Web Client?

            James

            I did see the see the Increase button in the Web Client. I checked the Device Backing, it is 5TB already

              LucD

              That could explain why the call to Get-VmfsDatastoreIncrease didn’t return anything.
              If that call doesn’t return anything, then the call to New-VmfsDatastoreIncrease will not work either (and error out).

              I’m not familiar with that type of HW.
              It could be something peculiar to the HW.

              What does the output from the call to Get-VmfsDatastoreInfo actually return?

                James

                Thanks a lot for quick reply!

                BTW, the increase is not working in Web client either. I managed to increased one of the same datastore from 4TB to 6TB use other work around by CODY HOSTERMAN. I want to test your script this time. Thanks.

                PS C:\> Get-VmfsDatastoreInfo -Datastore Prod_Unity_DS03

                Datastore : Prod_Unity_DS03
                CanonicalName : naa.6006016032d24400d65351594d57115e
                Model : DGC.VRAID.4401
                DiskSizeGB : 5120
                DiskBlocks : 5242880
                DiskBlockMB : 1
                PartitionFormat : gpt
                Partition :
                Used : False
                Type : none
                PartitionSizeGB : 0
                PartitionBlocks : 1
                PartitionBlockMB : 1
                Start : 0
                End : 0

                Datastore : Prod_Unity_DS03
                CanonicalName : naa.6006016032d24400d65351594d57115e
                Model : DGC.VRAID.4401
                DiskSizeGB : 5120
                DiskBlocks : 5242880
                DiskBlockMB : 1
                PartitionFormat : gpt
                Partition : 1
                Used : True
                Type : vmfs
                PartitionSizeGB : 4096
                PartitionBlocks : 4194303
                PartitionBlockMB : 1
                Start : 1
                End : 4194303

                Datastore : Prod_Unity_DS03
                CanonicalName : naa.6006016032d24400d65351594d57115e
                Model : DGC.VRAID.4401
                DiskSizeGB : 5120
                DiskBlocks : 5242880
                DiskBlockMB : 1
                PartitionFormat : gpt
                Partition :
                Used : False
                Type : none
                PartitionSizeGB : 1024
                PartitionBlocks : 1048576
                PartitionBlockMB : 1
                Start : 4194304
                End : 5242879

                  LucD

                  Do you have a link for Cody’s workaround?

                      LucD

                      I had a look, and basically it is doing more or less the same as my function with the Expand feature.
                      It’s not immediately clear to me why it fails in your environment.
                      I would need to do a debugging session to find out what exactly is going wrong.

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.