Dutch VMUG: The Statistics Reporting Session

I had the pleasure to present a session on “PowerCLI and vSPhere Performance and Capacity reporting” during the Dutch VMUG meeting of February 8th 2013. Although the meeting took place in the “Karnaval” weekend, there was a great turnout and, as always, a very attentive and interested audience. Thanks for attending the session guys !

karnaval

In the session I tried to show how easy it is to produce decent performance and capacity reports about your vSphere environment with PowerCLI. During the session I did some demos to show some aspects of PowerCLI and statistics. This post contains the code, and some annotations, I used during these demos.

I started off by showing what is available in PowerCLI for working with vSphere statistics.

Annotations

Line 10: In PowerShell v3 you can have your help displayed in a separate window, very handy to have the window open while writing your code.

Line 13-39: The Get-StatType cmdlet is used to find which counters are available for which VIObject in which time interval. See my PowerCLI & vSphere statistics – Part 1 – The basics post for additional info on counters.

Line 21-39: Depending on the interval you select, and the Statistics Level that is set in your vCenter, you will see a different number of available counters.

Line 44-51: Several counters return metrics for all the available instances and an aggregate counter. That aggregate metric is in fact the average over all the available instances. On the PerformanceManager pages that list the counters, you can see if a specific counter reports on instances or not. This also the first sample use of the Group-Object cmdlet, a cmdlet you must master when you are working with vSphere metrics.

VMUG-instance

Line 53-56: In the current PowerCLI release some counters can not be retrieved with the Get-Stat cmdlet. In that case you will have to fall back on the QueryPerf method available on the PerformanceManager object. Or you can use my Get-Stat2 function where I encapsulated that method in a PowerShell function.

Line 55: This line shows dot-sourcing at work. The code for the Get-Stat2 function is stored in the file Get-Stat2.ps1 and with the dot-sourcing we tell the PowerShell engine to read the file and thus “know” the definition of the function. After this dot-sourcing of the file, we can use the Get-Stat2 as a normal cmdlet in our scripts. This avoids that we will have to include this function code in all our scripts.

In the following script I present a quick method to create your own Counter Reference page.

Annotations

Line 8: The Exports-Xlsx function is available in Export-Xlsx, the sequel, and ordered data, or on Gilbert’s blog ITPilgrims A personal sandbox.

The resulting worksheet looks something like this.

VMUG-XRef

The Level column shows the Statistics Level at which this counter is available.

The StatType column shows the type of statistical measurement that is used for a counter. See PerfStatsType.

The Key column shows the internal key that vCenter uses for the counter. This value is unique for each environment, it depends on the features and components that are installed in your vCenter. This Key is used when you create for example an Alarm that is triggered on a performance counter exceeding a threshold. See for example Alarm expressions – Part 1 : Metric alarms.

In the following demo I tried to show how you can improve the performance of your statistic data gathering scripts. The first script shows the non-optimal method of gathering the data. A Get-Stat call for each counter and for each VIObject.

By limiting this to just one Get-Stat call your script will be considerably faster.

The use of the Group-Object cmdlet is key when retrieving statistical data. It allows your script to execute only one Get-Stat call and then separate the retrieved metrics based on a number of properties of the returned objects.

Annotations

Line 5: We don’t want the aggregate metric for this report, so we filter it out with a Where-clause

Line 6: The returned metrics are grouped by the name of the entity and the instance.

Line 10-13: Another cmdlet that is indispensable for working with statistical data is the Measure-Object cmdlet, it allows you to perform simple statistical functions on the data.

Line 14-20: In the results we want to see over which time period the data was collected, so we use the Timestamp on the first and the last returned metric to get these DateTime values.

The final demos showed some ways to add graphics to your reports.

The first one uses the Chart feature of Excel. With the Export-Xlsx function you easily add a chart to the worksheet or on a separate worksheet.

This way you have access to nearly all the charting features that are available in Excel. The result looks something like this.

VMUG-chart1

You can also use a commercial product like for example PowerGadgets. They offer a number of cmdlets to produce and annotate graphs.

Annotations

Line 17: In the $values variable the script collects the names of the Instances, which are the property names in the objects that were created before. These names are used to define which columns contain the data that needs to be added to the produced graph.

Line 19-24: There are numerous parameters on the Out-Chart cmdlet that can be used to annotate the produced graph.

Line 24: Since we collected the metrics in the Realtime interval we only want to see the time part of the Timestamp property. Notice how you easily use a .Net function in your PowerShell scripts.

The result looks something like this.

PowerGadgets-Demo3b

This post can of course not replace a “live” session, but I hope the demos that are in this post will help you to do your own discovery tour of PowerCLI, vSphere statistics and reporting.

Enjoy !

13 Comments

    mike

    Hi Luc:

    I’ve come with this script seems to run little faster. The idea is that I can this script every hour collect realtime data, I can do the aggregation outside. Can I do anything else to make this faster and complete within a reasonable time?

    ####Script
    # Add in the VI Toolkit
    if ( (Get-PSSnapin -Name VMware.VimAutomation.Core -ErrorAction SilentlyContinue) -eq $null )
    {
    Add-PSsnapin VMware.VimAutomation.Core
    }

    $Server=”dc1_vcenter.example.com”
    Connect-VIServer -server $Server -User -Password
    $allvms = @()
    $allhosts = @()
    $hosts = Get-VMHost
    $metrics=”cpu.usage.average”,”mem.usage.average”

    ForEach ($i in $hosts)
    {

    $Stats = Get-Stat -Realtime -Entity $i.Name -Stat $metrics

    $Stats | Select-Object Entity, Timestamp, MetricID, Value| ConvertTo-Csv -NoTypeInformation |Out-File -Append -Encoding ascii -FilePath “c:\output\dc1.csv”
    }
    Disconnect-VIServer -Server $Server -Confirm:$false

      LucD

      Hi Mike,
      You could consider getting the metrics in one Get-Stat call, instead of calling Get-Stat for each individual host.
      You can use a Sort-Object if you want the results in the CSV to be grouped by ESXi host.

    Sunny

    Great script. Is there not a way to pull the stats from vCenter for day, week, month, etc..? I like the performance report I’m able to pull by right clicking on the virtual machine, but it is not ideal when I have to pull reports for a 100 + virtual machines. It would be nice if we could use powershell to pull the report that is available in vCenter for a list of virtual machines; the graphs are really nice with the report option in vCenter. It would have been awesome if VMware allowed us to select multiple virtual servers to run the report instead of having to do it individually.

    thanks.

      LucD

      Hi Sunny, thanks.

      Afaik, there is no public API to the component that creates those Performance data and graphs.
      But like I showed in the post it is quite easy to create reports for multiple VMs and produce similar graphs to the ones you get in the vSphere Client.

    Sunil

    You are a true genius taking Powercli and its application to next levels. As always wonderful post and is helpful.

      LucD

      Thanks Sunil, much appreciated

    Sri

    Hello Lucd,

    First of all , your Scripts helped Stabilize my Environment a Lot.. It means lot to me .. I am really great full to all your postings in Vmware community and here .

    Your Data store Report helped me to keep a Check on the Data stores !

    Need a little help from you in consolidating a Monthly report we generate that takes hours for us to generate , that report mainly comprises of information out of all three virtual centers we have .

    1. Esx Clusters
    2.No of hosts in Each ESX Cluster
    3. VM Count in Each ESX Cluster
    4.Powered on machines in each cluster
    4.Total CPU and Memory in Each Cluster
    5.No of Datastores in Each Cluster
    6.Total Capacity in Cluster
    7 Free Space in Each Cluster.
    6 VLAN ID’s(Port Group ) Number of VMS’s in Each VLAN. (we have about three virtual centers and we are using the same vlan on three virtual centers.

    And once i get this i put it in excel sheet and sum of each count and i prepare a Graph.
    we have view servers also so we count the Information of each pool and then we create a Pie chart for that so we can figure out how many number of machines for each pool and also sum them up , we have dedicate pool(Full cloned Machines).

    Anything thing that you guide me on this would be a Great help to my Life !

    Eric Shanks

    Thanks Luc. Very useful info.

      LucD

      Thanks, great to hear it is of use to people.

    Francois-Xavier Cat

    Thanks for sharing LucD, Awesome script!!!! ;-D

      LucD

      Thanks, glad you like the script snippets.

    Chris Wahl

    Excellent details here, Luc. Thanks for sharing your code – will definitely find a use for it in my scripts repository, and will share with others.

      LucD

      Thanks Chris, much appreciated.

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*

This site uses Akismet to reduce spam. Learn how your comment data is processed.