Create VMFS datastores on “free space” partitions

The New-Datastore cmdlet allows you to easily create VMFS datastores on a free LUN or local disk. But what if you want to create a VMFS datastore on that free partition you have left on a LUN or on a local disk ? These free GBs could come in handy and it’s a shame letting them go to waste. Unfortunately, the New-Datastore cmdlet doesn’t have an option (yet) to handle “free space” partitions.

Exactly such a question was raised by Alasdair in his thread new-datastore on ESX4i Installable local disk in the PowerCLI Community recently.

When you create a new datastore via the vSphere client and you select a LUN or local disk that has already partitions that are in use, you get two options.

You can select to take the complete LUN/local disk and this will remove the partitions that were already there.

Or you can select the option to take the “free space” partition.

The following functions will allow you to find the “free space” partitions and to create a VMFS datastore on such a partition.

The script

function Get-ScsiFreePartition{
param (
[parameter(ValueFromPipeline = $true,Position=1)]
[ValidateNotNullOrEmpty()]
[VMware.VimAutomation.Client20.VMHostImpl]
$VMHost
)
process{
$esx = $VMHost | Get-View
$storMgr = Get-View $esx.ConfigManager.DatastoreSystem
$storSys = Get-View $esx.ConfigManager.StorageSystem
$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)
foreach($lun in $lunExt){
$info = $storMgr.QueryVmfsDatastoreCreateOptions($lun.DevicePath)
foreach($dsOpt in $info){
$info2 = $storSys.ComputeDiskPartitionInfo($lun.DevicePath,$dsOpt.Info.Layout)
$info2.Layout.Partition | where {$_.Type -eq "vmfs"} | %{
New-Object PSObject -Property @{
DeviceName = $lun.DeviceName
DeviceSizeMB = $lun.Capacity.block * $lun.Capacity.BlockSize / 1MB
Partition = $_.Partition
PartitionSizeMB = ($_.End.block - $_.Start.Block) * $_.Start.BlockSize / 1MB
FullDisk = &{if($_.Partition -eq 1){$true}else{$false}}
}
}
}
}
}
}
function New-PartitionDatastore{
param (
[parameter(ValueFromPipeline = $true,Position=1)]
[ValidateNotNullOrEmpty()]
[VMware.VimAutomation.Client20.VMHostImpl]
$VMHost,
[parameter(Position=2)]
[ValidateNotNullOrEmpty()]
[PSObject]
$Partition,
[parameter(Position=3)]
[ValidateNotNullOrEmpty()]
[String]
$Name
)
process{
$esx = $VMHost | Get-View
$storMgr = Get-View $esx.ConfigManager.DatastoreSystem
$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)
$device = $lunExt | where {$_.DeviceName -eq $Partition.DeviceName}
$dsOpt = $storMgr.QueryVmfsDatastoreCreateOptions($Partition.DeviceName) | where {$_.Info.VmfsExtent.Partition -eq $Partition.Partition}
$spec = $dsOpt.Spec
$spec.Vmfs.VolumeName = $Name
$spec.Extent += $spec.Vmfs.Extent
$dsMoRef = $storMgr.CreateVmfsDatastore($spec)
Get-Datastore (Get-View $dsMoRef).Name
}
}

function Get-ScsiFreePartition{

param (

[parameter(ValueFromPipeline = $true,Position=1)]

[ValidateNotNullOrEmpty()]

[VMware.VimAutomation.Client20.VMHostImpl]

$VMHost

)

process{

$esx = $VMHost | Get-View

$storMgr = Get-View $esx.ConfigManager.DatastoreSystem

$storSys = Get-View $esx.ConfigManager.StorageSystem

$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)

foreach($lun in $lunExt){

$info = $storMgr.QueryVmfsDatastoreCreateOptions($lun.DevicePath)

foreach($dsOpt in $info){

$info2 = $storSys.ComputeDiskPartitionInfo($lun.DevicePath,$dsOpt.Info.Layout)

$info2.Layout.Partition | where {$_.Type -eq "vmfs"} | %{

New-Object PSObject -Property @{

DeviceName = $lun.DeviceName

DeviceSizeMB = $lun.Capacity.block * $lun.Capacity.BlockSize / 1MB

Partition = $_.Partition

PartitionSizeMB = ($_.End.block - $_.Start.Block) * $_.Start.BlockSize / 1MB

FullDisk = &{if($_.Partition -eq 1){$true}else{$false}}

}

function New-PartitionDatastore{

param (

[parameter(ValueFromPipeline = $true,Position=1)]

[ValidateNotNullOrEmpty()]

[VMware.VimAutomation.Client20.VMHostImpl]

$VMHost,

[parameter(Position=2)]

[ValidateNotNullOrEmpty()]

[PSObject]

$Partition,

[parameter(Position=3)]

[ValidateNotNullOrEmpty()]

[String]

$Name

)

process{

$esx = $VMHost | Get-View

$storMgr = Get-View $esx.ConfigManager.DatastoreSystem

$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)

$device = $lunExt | where {$_.DeviceName -eq $Partition.DeviceName}

$dsOpt = $storMgr.QueryVmfsDatastoreCreateOptions($Partition.DeviceName) | where {$_.Info.VmfsExtent.Partition -eq $Partition.Partition}

$spec = $dsOpt.Spec

$spec.Vmfs.VolumeName = $Name

$spec.Extent += $spec.Vmfs.Extent

$dsMoRef = $storMgr.CreateVmfsDatastore($spec)

Get-Datastore (Get-View $dsMoRef).Name

}

Annotations

Line 14,53: The QueryAvailableDisksForVmfs method will return all disks that can be used to create VMFS datastores.

Line 16,55: The QueryVmfsDatastoreCreateOptions method will return all possible options to create (or extend) a datastore on the specific disk.

Line 18: The ComputeDiskPartitionInfo method will determine the disk partition layout for the specified disk layout.

Line 20-25: Creates and populates the objects with the available partition configurations.

Line 57-59: Populate the VmfsDatastoreCreateSpec object.

Line 60: Create the new VMFS datastore with the CreateVmfsDatastore method.

Line 61: Return a VMware.VimAutomation.Client20.DatastoreImpl object to make the function consistent with the New-Datastore cmdlet.

Sample runs

With the Get-ScsiFreePartition you can find the “free space” partitions on LUNs or local disks connected to your ESX(i) server.

$esxName = "esx4i.test.local"
$esxImpl = Get-VMHost -Name $esxName
$esxImpl | Get-ScsiFreePartition | ft -AutoSize

$esxName = "esx4i.test.local"

$esxImpl = Get-VMHost -Name $esxName

$esxImpl | Get-ScsiFreePartition | ft -AutoSize

This will produce output similar to this.

All returned partitions that are partition one mean the complete LUN or local disk. And they will hence destroy the existing data. For your convenience the function adds the FullDisk property which indicates this fact again.

If we want to use the “free space” partition, we have to do some filtering. Something like this for example

$esxName = "esx4i.test.local"
$esxImpl = Get-VMHost -Name $esxName
$partition = $esxImpl | Get-ScsiFreePartition | where {!$_.FullDisk} | Select -First 1
$esxImpl | New-PartitionDatastore -Partition $partition -Name "MyDS"

$esxName = "esx4i.test.local"

$esxImpl = Get-VMHost -Name $esxName

$partition = $esxImpl | Get-ScsiFreePartition | where {!$_.FullDisk} | Select -First 1

$esxImpl | New-PartitionDatastore -Partition $partition -Name "MyDS"

will create a new VMFS datastore on the first “free space” partition and the datastore will be named MyDS.

The properties of the datastore show that it is located on partition 3, the previous “free space” partition.

Notice that the New-PartitionDatastore returns the newly created datastore, similar to what the New-Datastore cmdlet does.

15 Comments

Gary Mobey

January 17, 2013 at 12:40

Hi Luc,

is there a way to add an extent to a previous Datastore with Powercli?

Gary

Jörg Kälin

October 29, 2013 at 18:13

I would also be interested in that 🙂

Kiran

June 17, 2012 at 10:38

Hi Luc,

Getting the below error while running your script. Pls help me out here.

Cannot find an overload for “QueryVmfsDatastoreCreateOptions” and the argument
count: “1”.

Regards,
kiran

LucD

June 17, 2012 at 11:28

Hi Kiran, there are 2 problems with this somewhat older script.
1) The PowerCLI types have changed in PowerCLI v5.x
2) In vSphere 5 some methods have extra parameters.

The following should work with vSphere 5.x and PowerCLI 5.x
function Get-ScsiFreePartition{ param ( [parameter(ValueFromPipeline = $true,Position=1)] [ValidateNotNullOrEmpty()] [VMware.VimAutomation.ViCore.Impl.V1.Inventory.VMHostImpl] $VMHost )


	process{

$esx = $VMHost | Get-View

$storMgr = Get-View $esx.ConfigManager.DatastoreSystem

$storSys = Get-View $esx.ConfigManager.StorageSystem
		$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)

foreach($lun in $lunExt){

$info = $storMgr.QueryVmfsDatastoreCreateOptions($lun.DevicePath,$null)

foreach($dsOpt in $info){

$info2 = $storSys.ComputeDiskPartitionInfo($lun.DevicePath,$dsOpt.Info.Layout,$null)

$info2.Layout.Partition | where {$_.Type -eq "vmfs"} | %{

New-Object PSObject -Property @{

DeviceName = $lun.DeviceName

DeviceSizeMB = $lun.Capacity.block * $lun.Capacity.BlockSize / 1MB

Partition = $_.Partition

PartitionSizeMB = ($_.End.block - $_.Start.Block) * $_.Start.BlockSize / 1MB

FullDisk = &{if($_.Partition -eq 1){$true}else{$false}}

}

}

}

}

}

}
function New-PartitionDatastore{

param (

[parameter(ValueFromPipeline = $true,Position=1)]

[ValidateNotNullOrEmpty()]

[VMware.VimAutomation.ViCore.Impl.V1.Inventory.VMHostImpl]

$VMHost,

[parameter(Position=2)]

[ValidateNotNullOrEmpty()]

[PSObject]

$Partition,

[parameter(Position=3)]

[ValidateNotNullOrEmpty()]

[String]

$Name

)
	process{

$esx = $VMHost | Get-View

$storMgr = Get-View $esx.ConfigManager.DatastoreSystem
		$lunExt = $storMgr.QueryAvailableDisksForVmfs($null)

$device = $lunExt | where {$_.DeviceName -eq $Partition.DeviceName}

$dsOpt = $storMgr.QueryVmfsDatastoreCreateOptions($Partition.DeviceName,$null) | where {$_.Info.VmfsExtent.Partition -eq $Partition.Partition}

$spec = $dsOpt.Spec $spec.Vmfs.VolumeName = $Name $spec.Extent += $spec.Vmfs.Extent $dsMoRef = $storMgr.CreateVmfsDatastore($spec) Get-Datastore (Get-View $dsMoRef).Name } }

Let me know if it works for you ?
I will then update the code to be generic (vSphere 4 and 5)

Denis

November 26, 2011 at 18:13

Hi Luc,
I’m new in powercli and when I try to execute the line “$esxImpl | Get-ScsiFreePartition | ft -AutoSize”, I got the message :

“The term ‘Get-ScsiFreePartition’ is not recognized as a cmdlet, function, opera
ble program, or script file. Verify the term and try again.
At line:1 char:33
+ $esxImpl | Get-ScsiFreePartition <<<< | ft -AutoSize"

I think I have to register this new function in PowerCli but I don't know how.

Can you help me?

Thanks.
Denis

LucD

November 26, 2011 at 22:36

@Denis, sure.
The 2 functions that appear in the post can be used in several ways. The easiest method.
Copy both functions to a file and save that file as a .ps1 file.
In your editor or Gui, first dot-source this .ps1 file you created and then you can call any of the functiosn (just like in the sample run)
For example
. ./MyFile.ps1 $esxName = "esx4i.test.local" $esxImpl = Get-VMHost -Name $esxName $partition = $esxImpl | Get-ScsiFreePartition | where {!$_.FullDisk} | Select -First 1 $esxImpl | New-PartitionDatastore -Partition $partition -Name "MyDS"
To dot-source a .ps1 file you type a dot, followed by a space and then the path to the .ps1 file.
In the code above, the MyFile.ps1 file is called from the current directory.

I hope this will help you to use the functions.

Sagar

October 25, 2013 at 02:30

I am getting this error, when I am trying to use Get-SCSIFreePartition function –

Exception calling “ComputeDiskPartitionInfo” with “3” argument(s): “A specified p
arameter was not correct.
Vim.Host.DiskPartitionInfo.spec”
At line:18 char:43
+ $info2 = $storSys.ComputeDiskPartitionInfo <<<< ($lun.DevicePath,$dsOpt.Info.La
yout,$null)
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : DotNetMethodException

PowerCLI Version
—————-
VMware vSphere PowerCLI 5.1 Release 1 build 793510
—————
Snapin Versions
—————
VMWare AutoDeploy PowerCLI Component 5.1 build 768137
VMWare ImageBuilder PowerCLI Component 5.1 build 768137
VMware License PowerCLI Component 5.1 build 669840
VMware vSphere PowerCLI Component 5.1 build 793489

Alasdair Carnie

November 18, 2011 at 14:40

@LucD
Hi Luc,

Yes iSCSI Luns. I can get a list of luns easily enough with get-scsilun, but what they are trying to do is list the iSCSI LUNs by target IP address. I can retrieve a list of target IP addresses from the software initiator, but I can’t see any obvious way to list which Luns each target address is supplying.

Al…………..

Alasdair Carnie

November 17, 2011 at 17:32

Hi Luc,

I have a student who is using one target IP address per LUN on some of his systems and asked me if there was a way to extract a list of LUNS per target address. I’ve had a dig into get-scsilun and get-datastore, but cannot see any obvious property candidates.

Any ideas?

LucD

November 17, 2011 at 23:34

Just to make sure I understand this correctly, you are talking about iScsi LUNs ?

Alasdair Carnie

June 7, 2011 at 12:53

Hi Luc,

I’ve been trying to extract a list of free luns in my cluster. Using Get-View and the query methods I can extract the currently unformatted luns, but due to the way in which storage information is stored in different branches within, Config, ConfigManager, etc, I can’t seem to get a formatted list containing, CanonicalName, LUN number, Capacity.

Any Ideas?

Thanks,

Al……….

LucD

June 7, 2011 at 13:07

@Alasdair, you might try the script in my LUN report – datastore, RDM and node visibility post.

Alasdair Carnie

March 23, 2011 at 15:05

Hi Luc,

If I want to format a LUN with VMFS, I have the following script that will find the correct NAA based on LUN number and push it into a variable.

I explain how the script works to my students, but I keep being asked if there is an easier way to do this. Can you help???

Thanks,

Alasdair………..

#Put your ESX host name between the quotes.
$vmHost = “Enter_ESX_host_name”

#Put the lunnumber between the quotes.
$PrivateLun = “Enter_your_LUN_number”

$DSName = “PrivateDS” + $PrivateLun

$esx = Get-VMHost $vmHost | Get-View
$esx.Config.StorageDevice.ScsiLun | where {$_.LunType -eq “disk”} | %{
$ScsiLun = $_
$CanonicalName = $SCSIlun.CanonicalName
$esx.Config.StorageDevice.PlugStoreTopology.Device | where{$_.Lun -eq $ScsiLun.Key} | %{
$PSTdevice = $_
$esx.Config.StorageDevice.PlugStoreTopology.Path | where {$_.Device -eq $PSTdevice.Key} | %{
$PSTpath = $_
$lun = $PSTpath.LunNumber
if ($lun -eq $privateLun) {
#Add_datastore_below_this_line Like this New-DataStore -VMFS -VMHost $vmHost -Path $CanonicalName -Name $DSName
New-DataStore -VMFS -VMHost $vmHost -Path $CanonicalName -Name $DSName

Write-Host $CanonicalName $RuntimeName
}
}
}
}

LucD

March 23, 2011 at 20:13

@Alasdair, yes you can make that script a bit shorter and simpler.
Note that the following requires at least PowerCLI 4.1 U1.

#Put your ESX host name between the quotes.
$vmHost = “YourEsxHost”

#Put the lunnumber between the quotes.
$PrivateLun = “LunNumber”

$DSName = “PrivateDS” + $PrivateLun
$esx = Get-VMHost $vmHost
$lun = Get-ScsiLun -VmHost $esx -LunType disk | `
where {$_.RuntimeName.Split(‘:’)[3].TrimStart(‘L’) -eq $PrivateLun}
New-DataStore -VMFS -VMHost $vmHost -Path $lun.CanonicalName -Name $DSName

Kahashi

November 24, 2014 at 03:54

Hey, can you guys help me out here? The script is supposed to find “free” luns but when I run the script the second time, it tries to create a mew datastore on the same previous lun. Why is trying to format an already formatted lun?

Create VMFS datastores on “free space” partitions

The script

Annotations

Sample runs

15 Comments

Gary Mobey

Jörg Kälin

Denis

Sagar

Alasdair Carnie

Kahashi

Leave a Reply Cancel reply