Home > event, PowerCLI, PowerShell, vSphere > Events – Part 3 : Auditing VM device changes

Events – Part 3 : Auditing VM device changes

December 18th, 2009 Leave a comment Go to comments

In a comment on my Events, Dear Boy, Events – Part 2 post, Sham was wondering if it was possible to find out which device was actually changed on a guest.

Sham was able to find out who did a change, at what time and on which machine, but he also wanted to know what exactly was changed.

Since this is the kind of audit information that I would like to have as well I had a closer look. And luckily the VmReconfiguredEvent object had all the required information in the configSpec property.

The configSpec property contains a VirtualMachineConfigSpec object. This is the same object that you use when you call the CreateVM_Task or ReconfigVM_Task method.

That means that all the device information is stored under the deviceChange property as an array of VirtualDeviceConfigSpec objects.

Using my standard Task & Event script it was rather simple to generate a basic report on device changes.

Update May 20th 2010:

  • introduce task- and event-reader loops to handle high numbers of tasks and events in the requested time period
  • add code to monitor CPU and memory changes on guests
$hours = 36 # Number of hours back
$tasknumber = 999 # Windowsize for task collector
$eventnumber = 100 # Windowsize for event collector

$report = @()

$taskMgr = Get-View TaskManager
$eventMgr = Get-View eventManager

$tFilter = New-Object VMware.Vim.TaskFilterSpec
$tFilter.Time = New-Object VMware.Vim.TaskFilterSpecByTime
$tFilter.Time.beginTime = (Get-Date).AddHours(-$hours)
$tFilter.Time.timeType = "startedTime"

$tCollector = Get-View ($taskMgr.CreateCollectorForTasks($tFilter))

$dummy = $tCollector.RewindCollector
$tasks = $tCollector.ReadNextTasks($tasknumber)

while($tasks){
	$tasks | where {$_.Name -eq "ReconfigVM_Task"} | % {
		$task = $_
		$eFilter = New-Object VMware.Vim.EventFilterSpec
		$eFilter.eventChainId = $task.EventChainId

		$eCollector = Get-View ($eventMgr.CreateCollectorForEvents($eFilter))
		$events = $eCollector.ReadNextEvents($eventnumber)
		while($events){
			$events | % {
				$event = $_
				switch($event.GetType().Name){
					"VmReconfiguredEvent" {
						$event.ConfigSpec.DeviceChange | % {
							if($_.Device -ne $null){
								$report += New-Object PSObject -Property @{
									VMname = $task.EntityName
									Start = $task.StartTime
									Finish = $task.CompleteTime
									Result = $task.State
									User = $task.Reason.UserName
									Device = $_.Device.GetType().Name
									Operation = $_.Operation
								}
							}
						}
					}
					Default {}
				}
			}
			$events = $eCollector.ReadNextEvents($eventnumber)
		}
		$ecollection = $eCollector.ReadNextEvents($eventnumber)
# By default 32 event collectors are allowed. Destroy this event collector.
		$eCollector.DestroyCollector()
	}
	$tasks = $tCollector.ReadNextTasks($tasknumber)
}

# By default 32 task collectors are allowed. Destroy this task collector.
$tCollector.DestroyCollector()

$report | Sort-Object -Property Start | Export-Csv "C:\VM-device-operations.csv" -NoTypeInformation -UseCulture

Annotation

Line 1: define how far back in time you want the report to go
Line 2-3: these values limit the windows of the Task and the Event collectors. For me these values produce the best result, but depending on the size of your environment, you might have to change them to get a better performance from the script
Line 10-13: configure the filter for the Task collector
Line 21: we’re only interested in ReconfigVM_Task tasks
Line 24: find all events related to the task
Line 33: report on all devices under the DeviceChange property
Line 54 & 60: there is a limit of 32 Task and 32 Event collectors per session. Clean up the collectors otherwise your next run of the script might fail

Result

The CSV file that is produced by the script looks like this.

VM-device-changes

This version of the script only looked superficially at the device changes, but since we have access to the complete VirtualMachineConfigSpec object, the script can go a lot farther.

For each device it can report the actual settings (depending of course on the device type).

And besides the devices, we can also produce reports on subjects like

  • changes in CPU masks
  • hot add and remove of CPUs and memory
  • changes in network traffic shaping
  • changes in the VMware Tools configuration
  • addition or removal of the guest to a vApp

Let me know if you have any other suggestions for reporting on changes made to the guests.

CPU and memory changes

Skywalker (see comments) wanted to know how to monitor for memory changes.

This is in fact quite straightforward since the information is available in the ConfigSpec property of the VmReconfiguredEvent object. The only “trick” is that you need to know how the system indicates what is changed. For the NumCPUs and MemoryMB properties, if they are not equal to zero the value has been changed.

Unfortunately the task nor event keeps track of what the original value was.

To adapt the script to report on number of CPUs and memory size changes, replace lines 33-45 with the following code.

						$event.ConfigSpec | % {
							if($_.NumCPUs -ne 0 -or $_.MemoryMB -ne 0){
								$report += New-Object PSObject -Property @{
									VMname = $task.EntityName
									Start = $task.StartTime
									Finish = $task.CompleteTime
									Result = $task.State
									User = $task.Reason.UserName
									Memory = &{if($_.MemoryMB -ne 0){$_.MemoryMB}else{""}}
									NumCPU = &{if($_.NumCPUs -ne 0){$_.NumCPUs}else{""}}
								}
							}
						}

This produces a CSV file like this:

  1. jkb5054
    May 4th, 2012 at 19:26 | #1

    Is there a way to tell whether the resource was added or subtracted from the VM?

    I want to be able to track who is adding and who is deleting how much of CPU and memory.

    any ideas?

    • May 4th, 2012 at 19:46 | #2

      Hi jkb5054, sort of :-)
      The event only has the new value, not the original value.
      So you’ll be able to see who changed CPU and/or memory, but you will not be able to tell what the original value was.
      Try this

      $hours = 24 # Number of hours back
      $tasknumber = 999 # Windowsize for task collector
      $eventnumber = 100 # Windowsize for event collector

      $report = @()

      $taskMgr = Get-View TaskManager
      $eventMgr = Get-View eventManager

      $tFilter = New-Object VMware.Vim.TaskFilterSpec
      $tFilter.Time = New-Object VMware.Vim.TaskFilterSpecByTime
      $tFilter.Time.beginTime = (Get-Date).AddHours(-$hours)
      $tFilter.Time.timeType = "startedTime"

      $tCollector = Get-View ($taskMgr.CreateCollectorForTasks($tFilter))

      $dummy = $tCollector.RewindCollector
      $tasks = $tCollector.ReadNextTasks($tasknumber)

      while($tasks){
      $tasks | where {$_.Name -eq "ReconfigVM_Task"} | % {
      $task = $_
      $eFilter = New-Object VMware.Vim.EventFilterSpec
      $eFilter.eventChainId = $task.EventChainId

      $eCollector = Get-View ($eventMgr.CreateCollectorForEvents($eFilter))
      $events = $eCollector.ReadNextEvents($eventnumber)
      while($events){
      $events | % {
      $event = $_
      switch($event.GetType().Name){
      "VmReconfiguredEvent" {
      $report += New-Object PSObject -Property @{
      VMname = $task.EntityName
      Start = $task.StartTime
      Finish = $task.CompleteTime
      Result = $task.State
      User = $task.Reason.UserName
      NumCPU = $event.ConfigSpec.NumCPUs
      NumCoresPerSocket = $event.ConfigSpec.NumCoresPerSocket
      MemoryMB = $event.ConfigSpec.MemoryMB
      }
      }
      Default {}
      }
      }
      $events = $eCollector.ReadNextEvents($eventnumber)
      }
      # By default 32 event collectors are allowed. Destroy this event collector.
      $eCollector.DestroyCollector()
      }
      $tasks = $tCollector.ReadNextTasks($tasknumber)
      }

      # By default 32 task collectors are allowed. Destroy this task collector.
      $tCollector.DestroyCollector()

      $report | Sort-Object -Property Start

  2. Suresh
    July 1st, 2011 at 07:39 | #3

    How do use this script against one VM alone? i.e. want to see all the “reconfigure” events for one VM alone.

  3. Kayser Soze
    June 24th, 2011 at 02:03 | #4

    Hi Luc,

    Is it possible to also detect hard disk changes? i.e. increasing the existing disk size, or adding another disk to a VM?

    thanks.

  4. Pcli
    June 29th, 2010 at 06:30 | #5

    Hi LucD,

    when i run the first script i get a 0 Kb output file. i did make a few changes to cpu and memory before i ran the script

  5. Skywalker
    May 21st, 2010 at 04:50 | #6

    @LucD
    Luc,
    It seems that the column “Numcpu” is empty when I made 2 changes (more than 1 change) at a time. For instance, removed virtual floppy, and added 1 vCPU to the VM.

    • May 21st, 2010 at 12:28 | #7

      Hi Skywalker, just did a similar test myself and it looks indeed as if the event generated by such an action is missing the memory change.
      This looks like a bug. I’ll open an SR.
      Luc.

  6. Skywalker
    May 20th, 2010 at 09:11 | #8

    @Skywalker
    brilliant!!

  7. Skywalker
    May 20th, 2010 at 05:39 | #9

    Hi Luc,
    This script can’t report on memory change to guest OS. How to modify the script to get this done? Thanks in advance.

  8. December 22nd, 2009 at 11:49 | #11

    Luc,

    Just wanted to say a big thanks for posting this.

    My script was called via a virtualcenter alarm when a vmreconfigured event was initiated and so I was relying on the environment variables that the alarm passed back to provide this information.

    The code youve written clearly demontrates its possible to get the information out if you have the vmreconfigured event object so I guess Im going to have to authenticate back into VC to get the relevant event object to identify the change.

    Hopefully this will allow for the reporting of changes in real time.

    Ill let you know how I get on…

  9. Ian
    December 20th, 2009 at 11:52 | #12

    Is there a way to report on who started a VM? Currently the GUI doesn’t match who did the power on of a VM.

  1. December 18th, 2009 at 20:14 | #1