We all know, and love, PowerCLI‘s New-Datastore and Set-Datastore cmdlets to create and manipulate VMFS datastores. But when we look at the functionality available through the Web Client, there is one interesting feature for manipulating VMFS datastores that is missing from the PowerCLI cmdlets. The Increase button, which allows us to Expand or Extend an existing VMFS datastore*.
Recently there were a couple of threads on this subject in the VMTN PowerCLI Community, so I decided to streamline my quick-and-dirty scripts into something more presentable, and create a PowerShell module to bundle the functions. I present the VMFSIncrease module!
The VMFSIncrease module will also be my first contribution to the PowerCLI Community Repository! More on that further on in this post.
* The expand and extend functions for a VMFS datastore depend on the availability of free space on the VMFS datastore extents and/or the availability of free LUNs
The Increase button
The Increase button in fact offers two different methods to increase the size of a datastore. The definitions of two methods according to the
- Expand: increases the capacity of an existing VMFS datastore by expanding (increasing the size of) an existing extent of the datastore
- Extend: Increases the capacity of an existing VMFS datastore by adding new extents to the datastore
Perhaps the following schematic makes it a bit more clear.
The schematic shows a VMFS datastore with one extent, on LUN A. If we want to increase the Used capacity, we can do an Expand, on the same extent (LUN A). Or we can do an Extend on a new LUN, in this case LUN B.In the case of the Extend, the VMFS datastore will now have two extents, LUN A and LUN B.
Note that for the Expand and the Extent you do not need to use all of the free space. The following is perfectly possible.
The Increase button presents the user with a series of candidate LUNs. The Expand and Extend candidate LUNs are intermixed in that list. The underlying API method makes sure that the LUNs are not already used by another datastore or by a RDM disk.
The Module
The module is available in my GitHub VMFSIncrease repository.
Sample runs
Reporting
The Get-VmfsDatastoreInfo displays basic information about the extents and the partitions that make up the datastore.
1 |
Get-VmfsDatastoreInfo -Datastore TestDS |
This produces something like this
The Datastore lives on one extent, and on that extent there are three partitions. The partition marked with (2) is the part where the VMFS Datastore lives. Part (3) is the free space, and part (1) is for alignment.
To check how we can increase the capacity of the Datastore, we can run
1 |
Get-VmfsDatastoreIncrease -Datastore TestDS |
This returns something like this
The returned information shows that there are two options to increase the capacity of the Datastore. In (1) we see that we can Expand the extent by 80GB. In (2) we see that we can Extend the capacity by 100GB by adding a new extent to the Datastore.
Expand
The Help page for the New-VmfsDatastoreIncrease function already contains several examples on how to use the function. But to give you an idea what is possible, just a few examples.
We have the possibility to add up to 80GB. If we don’t specify a size, all the free space on the extent is used.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Expand |
If we do specify how much we want to increase the capacity, only that specific amount of space from the free space will be taken.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Expand -IncreaseSizeGB 25 |
If we have a Datastore that spans more than one extent, we can, through the Canonicalname parameter, indicate on which extent the increase shall be taken.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Expand -IncreaseSizeGB 15 -CanonicalName 'naa.60050123017775af1800000000000011' |
Extend
As we explained in the beginning of the post, with the Extend switch we indicate that we want to increase the capacity on a new extent. Again we can take the full extent.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Extend -CanonicalName 'naa.60050123017775af1800000000000012' |
Interesting to note, if you don’t specify a LUN, through the Canonicalname parameter, the function will order the available LUNs alphanumerically on their Canonicalname, and take the first one. So the following is perfectly possible, provided of course there are free LUNs available.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Extend |
And, as with the Expand option, you do not need to use all of the available space with the Extend option. You can control how much space is taken through the IncreaseSizeGB parameter.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Extend -IncreaseSizeGB 50 -CanonicalName 'naa.60050123017775af1800000000000012' |
The option is also available without specifying a LUN. The function uses the same logic as was mentioned before, it will order the available LUNs alphanumerically on their Canonicalname, and take the first one.
1 |
New-VmfsDatastoreIncrease -Datastore 'TestDS' -Extend -IncreaseSizeGB 35 |
PowerCLI Community Repository
Since a couple of days the long awaited PowerCLI Community Repository is live! More info can be found in the Updating the VMware PowerCLI Community Repository! post over on the PowerCLI Blog.
If you have any scripts, functions, modules… that might be useful for the community, please submit them there.
The VMFSIncrease module will be available on the PowerCLI Community Repository shortly.
Enjoy, and share!
Josef Zach
Hi, module did not work.
I have single esxi host with hpe smart array controller (raid 1+0) expanded with 2 more drives. On this same raid was esxi installed and also VMFS 5 (esxi 6.7u1).
I think there was issue with backup gpt partition which was not moved to the end of free space (just thought..)-
Get-VmfsDatastoreInfo -Datastore TestDS did not return free space as expand/extend but NONE.
expand/extend command return issue – there is no free space…
Cody Hosterman workaround mentioned above works.
LucD module helps me many time before..
LucD
Thanks, I’ll have a look if I can improve the code to deal with this
Mike
Hi,
I have to automate the resizing of datastore in ruby.I performed a manual resizing through vcenter
and in vcenter logs(vpxa.logs) I found the below apis are calling in sequence–
vim.host.StorageSystem.retrieveDiskPartitionInfo
vim.host.StorageSystem.computeDiskPartitionInfoForResize
vim.host.DatastoreSystem.expandVmfsDatastore
I assume that if these apis will be called in same sequnece, the resizing can be automated.
Can I get some sample code regarding this.
Regards,
Mike
LucD
Hi Mike,
Sorry, but I’m afraid I’m a noob concerning Ruby.
james
Hello LucD,
Tested on my system, failed to increase, got following errors:
PS C:\> Get-VmfsDatastoreIncrease -Datastore ‘Prod_Unity_DS03’
PS C:\> New-VmfsDatastoreIncrease -Datastore ‘Prod_Unity_DS03’ -Expand -IncreaseSizeGB 200
Exception calling “RetrieveDiskPartitionInfo” with “1” argument(s): ”
Required parameter devicePath is missing
while parsing call information for method RetrieveDiskPartitionInfo
at line 1, column 171
while parsing SOAP body
at line 1, column 64
while parsing SOAP envelope
at line 1, column 0
while parsing HTTP request for method retrieveDiskPartitionInfo
on object of type vim.host.StorageSystem
at line 1, column 0″
At C:\Users\222222\Documents\WindowsPowerShell\Modules\VMFSIncrease-master\VMFSIncrease.psm1:183 char:6
+ … $partInfo = $hsSys.RetrieveDiskPartitionInfo($lun.DeviceN …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : VimException
Cannot index into a null array.
At C:\Users\222222\Documents\WindowsPowerShell\Modules\VMFSIncrease-master\VMFSIncrease.psm1:186 char:6
+ … $partUsed = ($partInfo[0].Layout.Partition | where{ $_.Ty …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
Exception calling “ExpandVmfsDatastore” with “2” argument(s): “Cannot change the host configuration.”
At C:\Users\222222\Documents\WindowsPowerShell\Modules\VMFSIncrease-master\VMFSIncrease.psm1:203 char:5
+ … $hdSys.ExpandVmfsDatastore($Datastore.ExtensionData.MoRef …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : VimException
LucD
Hi James,
From the error message, it looks as if the method RetrieveDiskPartitionInfo has an issue finding partition information.
What kind of LUNs are you working with?
Are the FC LUNs?
Also, do you see the Increase button in the Web Client?
Select the datastore tab, then the datastore you are trying to increase and then the Configure tab.
Under the General option you should see an Increase button in the righ panel.
james
Thank you very much for the quick reply, I am using EMC Unity FC LUNs, the
Get-VmfsDatastoreInfo -Datastore Prod_Unity_DS03 working fine, I saw 3 partitions, which match exactly with what you have described here. I tested both PowerShell 5.1.18362.145 and 6.2.3. The vCenter version is 6.5.0.
BTW, all the ESXi hosts and vCenter see the new LUN 5TB already, the vmfs datastore is 4TB now.
LucD
Thanks for the feedback.
Do you see the Increase button in the Web Client?
James
I did see the see the Increase button in the Web Client. I checked the Device Backing, it is 5TB already
LucD
That could explain why the call to Get-VmfsDatastoreIncrease didn’t return anything.
If that call doesn’t return anything, then the call to New-VmfsDatastoreIncrease will not work either (and error out).
I’m not familiar with that type of HW.
It could be something peculiar to the HW.
What does the output from the call to Get-VmfsDatastoreInfo actually return?
James
Thanks a lot for quick reply!
BTW, the increase is not working in Web client either. I managed to increased one of the same datastore from 4TB to 6TB use other work around by CODY HOSTERMAN. I want to test your script this time. Thanks.
PS C:\> Get-VmfsDatastoreInfo -Datastore Prod_Unity_DS03
Datastore : Prod_Unity_DS03
CanonicalName : naa.6006016032d24400d65351594d57115e
Model : DGC.VRAID.4401
DiskSizeGB : 5120
DiskBlocks : 5242880
DiskBlockMB : 1
PartitionFormat : gpt
Partition :
Used : False
Type : none
PartitionSizeGB : 0
PartitionBlocks : 1
PartitionBlockMB : 1
Start : 0
End : 0
Datastore : Prod_Unity_DS03
CanonicalName : naa.6006016032d24400d65351594d57115e
Model : DGC.VRAID.4401
DiskSizeGB : 5120
DiskBlocks : 5242880
DiskBlockMB : 1
PartitionFormat : gpt
Partition : 1
Used : True
Type : vmfs
PartitionSizeGB : 4096
PartitionBlocks : 4194303
PartitionBlockMB : 1
Start : 1
End : 4194303
Datastore : Prod_Unity_DS03
CanonicalName : naa.6006016032d24400d65351594d57115e
Model : DGC.VRAID.4401
DiskSizeGB : 5120
DiskBlocks : 5242880
DiskBlockMB : 1
PartitionFormat : gpt
Partition :
Used : False
Type : none
PartitionSizeGB : 1024
PartitionBlocks : 1048576
PartitionBlockMB : 1
Start : 4194304
End : 5242879
LucD
Do you have a link for Cody’s workaround?
james
It is here.
https://www.codyhosterman.com/2018/09/growing-a-vmfs-datastore-with-powercli/
LucD
I had a look, and basically it is doing more or less the same as my function with the Expand feature.
It’s not immediately clear to me why it fails in your environment.
I would need to do a debugging session to find out what exactly is going wrong.