RDM & vMotion: inaccessible direct-access LUN
When you try to migrate a guest, that is using one or more RDM disks, you might see this message.
The reason this is most probably because the LUN IDs are different on the source and the destination ESX server.
One solution is:
- stop the guest
- write down the Physical LUN ID
- remove the RDM disk(s)
- vMotion the guest
- add the RDM disk(s) to the guest based on the Physical LUN ID
- start the guest
But why do this the hard (manual) way when we have PowerCLI ?
Automating the above scenario doesn’t look too difficult, but there are some pitfalls on the way.
Gathering the information
In this step I collect extensive information about the guest and the RDM disk(s) it has connected.
As a safety-measure, the collected information is dumped to a CSV file. This should allow, in case of script-failure, to restore the RDM mappings quite easily.
The collected information should also include to which controller the RDM disk is connected and the RDM’s unit number. This is important because otherwise the guest’s OS might mix up the drive lettering.
Removing the RDM
For this step I use a filter, called Remove-HD. This makes it easy to combine this in a pipe with the Get-Harddisk cmdlet.
I could have used PowerCLI’s Remove-Harddisk but this cmdlet leaves the mapping files on the datastore and thus makes your guest consume unused disk space.
The filter has two steps, the first one removes the RDM from the guest’s configuration (ReconfigVM_Task) and the second step removes (DeleteDatastoreFile_Task) the mapping file(s) from the datastore.
Connecting the RDM
This function (New-RawHardDisk) already appeared in the PowerCLI Community thread Adding an existing hard disk. I just added some logic to be able to add physical and virtual mode RDMs.
Main function
This is driving part of the script. It controls the logic and the order in which the different functions and filters are called.
The script
# ESX server against which you want to execute the script
$tgtESX = "My-target-ESX-host"
# Attach a RDM disk to a VM on a specified controller
function New-RawHardDisk{
param($vm, $DeviceName, $DiskType, $controller, $unitnumber, $capacity)
$vmMo = $vm | Get-View
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$chg = New-Object VMware.Vim.VirtualDeviceConfigSpec
$chg.operation = "add"
$chg.fileoperation = "create"
$dev = New-Object VMware.Vim.VirtualDisk
$dev.CapacityInKB = $capacity
$dev.Key = - 100
$back = New-Object VMware.Vim.VirtualDiskRawDiskMappingVer1BackingInfo
$back.deviceName = $devicename
if($DiskType -eq "RawPhysical"){
$back.compatibilityMode = "physicalMode"
}
else{
$back.compatibilityMode = "virtualMode"
}
$back.FileName = ""
$dev.Backing = $back
$dev.ControllerKey = $controller
$dev.UnitNumber = $unitnumber
$chg.device = $dev
$spec.deviceChange += $chg
$taskMoRef = $vmMo.ReconfigVM_Task($spec)
$task = Get-View $taskMoRef
while("running","queued" -contains $task.Info.State){
$task.UpdateViewData("Info.State")
}
$task.UpdateViewData("Info.Result")
}
# Remove a RDM disk from a VM and optionally delete the backing file
filter remove-HD {
param($Delete)
$HDname = $_.Name
$vm = Get-View -Id $_.ParentId
# Get device key for the hard disk
foreach($dev in $vm.Config.Hardware.Device){
if ($dev.DeviceInfo.Label -eq $HDname){
continue
}
}
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.deviceChange = @()
$spec.deviceChange += New-Object VMware.Vim.VirtualDeviceConfigSpec
$spec.deviceChange[0].device = New-Object VMware.Vim.VirtualDevice
$spec.deviceChange[0].device.key = $dev.Key
$spec.deviceChange[0].device.unitnumber = $dev.UnitNumber
$spec.deviceChange[0].operation = "remove"
$taskMoRef = $vm.ReconfigVM_Task($spec)
$task = Get-View $taskMoRef
while("running","queued" -contains $task.Info.State){
$task.UpdateViewData("Info.State")
}
$task.UpdateViewData("Info.Result")
# Remove hard disk files (backing) from datastore
if ($Delete){
$fileMgr = Get-View (Get-View ServiceInstance).Content.fileManager
$datacenter = (Get-View (Get-VM $vm.name | Get-Datacenter).ID).get_MoRef()
foreach($disk in $vm.LayoutEx.Disk){
if($disk.Key -eq $dev.Key){
foreach($chain in $disk.Chain){
foreach($fileKey in $chain.FileKey){
$name = $vm.LayoutEx.File[$fileKey].Name
$taskMoRef = $fileMgr.DeleteDatastoreFile_Task($name, $datacenter)
$task = Get-View $taskMoRef
while("running","queued" -contains $task.Info.State){
$task.UpdateViewData("Info.State")
}
$task.UpdateViewData("Info.Result")
}
}
continue
}
}
}
}
# Main
$report = @()
# Report on all the VMs that have a RDM
$vms = Get-View -ViewType VirtualMachine
foreach($vm in $vms){
foreach($dev in $vm.Config.Hardware.Device){
if(($dev.gettype()).Name -eq "VirtualDisk"){
if(($dev.Backing.CompatibilityMode -eq "physicalMode") -or
($dev.Backing.CompatibilityMode -eq "virtualMode")){
$esx = Get-View $vm.Runtime.Host
$lun = $esx.Config.StorageDevice.ScsiLun | where {$_.Uuid -eq $dev.Backing.LunUuid}
$report += New-Object PSObject -Property @{
VMName = $vm.Name
VMHost = ($esx).Name
HDLabel = $dev.DeviceInfo.Label
HDDeviceName = $dev.Backing.DeviceName
HDFileName = $dev.Backing.FileName
HDMode = $dev.Backing.CompatibilityMode
HDSize = $dev.CapacityInKB
LunDisplayName = $lun.DisplayName
HDCtrlType = ($vm.Config.Hardware.Device | where {$_.Key -eq $dev.ControllerKey}).GetType().Name
HDController = $dev.ControllerKey
HDUnit = $dev.UnitNumber
HDDiskMode = $dev.Backing.DiskMode
LunCanonical = $lun.CanonicalName
LunDeviceName = $lun.DeviceName
}
}
}
}
}
$report | Export-Csv "C:\VM-with-RDM.csv" -NoTypeInformation -UseCulture
$report | Group-Object -Property VMName | % {
$vm = Get-VM $_.Name
# Stop the VM
Write-Host "Stop VM" $_.Name
$VM | Get-VMGuest | where {$_.State -eq "Running"} | Shutdown-VMGuest - Confirm:$false
# Remove the RDM disk(s)
$_.Group | % {
Write-Host "`tremove HD" $_.HDLabel
$vm | Get-HardDisk -DiskType ("Raw" + $_.HDMode.TrimEnd("Mode")) | Remove-HD - Delete:$true
}
# vMotion the VM
Write-Host "Moving VM"
$newvm = $vm | Move-VM -Destination (Get-VMHost -Name $tgtESX)
# Attach the RDM disk(s) back
$_.Group | % {
Write-Host "`tAdd HD" $_.HDLabel
New-RawHardDisk -Vm $newvm -DiskType ("Raw" + $_.HDMode.TrimEnd("Mode")) -DeviceName $_.LunDeviceName -Controller $_.HDController -UnitNumber $_.HDUnit -Capacity $_.HDSize
}
# Power on the VM
Write-Host "Start VM" $_.Name
$newvm | Start-VM - Confirm:$false
}
Annotation
Line 2: destination ESX server
Line 18-23: support for physical and virtual mode RDM
Line 118: dump the RDM info to a CSV file
Line 120-141: this is the part where the RDM disk(s) are removed, the VM vMotioned and the RDM disk(s) re-attached.
Line 136: because a guest can have more than one RDM I use the Group-Object cmdlet


Hello Luc,
Thank you for the great script. I’m trying to use it in two steps (two separate scripts) for our SRDF failover. First step (one script) to create the RDM report and the second step to import the CSV report and use that data to attach the RDM. The disk order is important as we have VMs with many RDMs,
The first part works great, export the report and remove the RDM and backing file, but I’m having difficulty importing the CSV and connecting the RDM back to its VM.
It goes through importing the CSV and I can see the variables populate with the data, but at the end I get I get errors about null argument and methods.
Can you help please?
@Michael, can you send me the scripts you are using, so I can have a look.
Send it to lucd (at) lucd (dot) info
Hi Luc,
Hope you had a nice trip back van SF.
As I told you in SF I am writing a PS script with a gui to move / clone VM’s with RDM”s connected. Thanks to your code I have come a far way.
The only thing I want to do with the above script is to clone VM-A (with VMDK & RDM) to antother location. But I ant to clone (the RDM is removed, the clone to VM-B is done, but the script does not wait untill the end of the clone operation to readd the RDM to VM-A. Is there a way to have the new-vm cmdlet wait until the clone is done before to move on with the next step in the script?
Thanks Gert, I did.
I’m afraid you have to look for another way to wait till the new VM-B is ready.
The New-VM cmdlet creates the new guest, creates the sysprep input files,copies the machine over and powers on VM-B. At that point the task is finished for New-VM.
On VM-B in the mean time the sysprep process is running which will require at least one reboot of VM-B.
If you can find a way that VM-B informs your script that the sysprep process is finished, you have a solution.
In a similar situation, I created a token-file, at the end of the sysprep process, on a share and let my script run in a loop till the token-file was present. But this requires network connectivity and membership.
Like I said at the beginning, this is not a New-VM problem but a sysprep problem. How do you find out, externally, that the sysprep process is finished.
Sorry I don’t have a clear solution for this.
updated my email address.
Hi Luc – thanks for doing that, it does make it a lot clearer – although i still cant seem to hack it for what I need.
Essentially, we are cloning VMs using the storage array and i’ve “written” (pinched) scripts to do that for normal vms (without RDM) – but we have some clusters that use RDMs and they do not clone well. So I need to 1) remove the RDMs from the cloned VMs (because these are invalid), 2) Readd the cloned RDM LUNs (which have different LUN ids to the originals).
Long shot – but do you know of any existing scripts (and i initially thought I could use this) – so I can pass it some parameters “VM Name”, “SCSI Controller ID”,”LUN ID” etc.. and it will add the RDM in physical compatibility mode to the VM?
Thanks again – ur a king in the community!
John, there are no stupid questions, just stupid answers
The script run against all the VMs on a specific ESX (which you specify in line 2).
I have added a few comment lines to the script and indented the lines where appropriate.
Let me know if that helps ?
Luc.
Hi – stupid question, but how do you actually run this. I cant see anywhere where the VM is specified. Pasting the above code into PS doesnt do anything.
THanks
So much typing !!
must have taken ages to type , instead why not do the following …much less effort!
Why not copy and paste your script
* stop the guest
* write down the Physical LUN ID
* remove the RDM disk(s)
* vMotion the guest
* add the RDM disk(s) to the guest based on the Physical LUN ID
* start the guest
Thanks for the tip.
While I agree this would surely be faster than me typing the script, the script has it’s advantage when you have to migrate substantial numbers of guests with RDMs.
And you avoid making “human errors”
Automation proves it’s worth when the task becomes repetitive and when you have to tackle bigger numbers.
Come on man ,couldn’t you have post this 3 Weeks ago ?
i wanted to write a script’ but because i didn’t have the time ‘; it did not happend.
nice one .
BTW : The Problem happens also if the LUN ID is the same in Both Host.
Sorry that my timing was not optimal
Thanks for the info on identical LUN ids, didn’t know that one.
Even if the LUN ID is the same under certain circumstances the error can still pop up – http://www.techopsguys.com/2009/08/18/its-not-a-bug-its-a-feature/
Worked fine in ESX 3.0, 3.02, 3.5 (when the LUN ID is the same). I call it a bug, they call it a feature.
Thanks for the feedback.
At least you can now automate the tedious process of re-mapping all the RDM volumes