PowerCLI & vSphere statistics – Part 5 – Rollup Types

In my previous posts in this series, I showed how you can use metrics to produce statistical reports.

One aspect of the metrics that we haven’t touched till now is the roll up types for metrics. If we look in the PerfCounterInfo object under the rollupType property, we discover that there are several of roll up types available. The enumeration lists the following: average, latest, minimum, maximum, none and summation.

What do all these types mean, and more importantly how do we handle these in our scripts ?

That is the subject of this episode in the statistics series.

Average

This roll up type is rather straightforward. The metric will contain the actual value collected over the interval or the average of the values collected over the interval. Some well-know metrics of this type: cpu.usage.average and mem.active.average.

When we retrieve values from the Realtime interval, we will get the average value of all the values that the ESX(i) server collected in the 20 seconds interval.

An example

Annotations

Line 1: Specify the metrics we want to retrieve

Line 2: Get the entities for which we want to retrieve the statistics

Line 3: Specify the start of the period for which we want to retrieve the data. Since we will not use the Finish parameter, we will get the data from 2 minutes ago till now.

Line 5: We want statistical data from the Realtime interval.

Line 6: Since we want to combine 2 statistical values in 1 row we group the returned data by entity and by timestamp. Each group will thus have 2 entries, one for the CPU metric and one for the memory metric.

Line 10,11: We filter the returned group elements to get respectively the CPU and the memory value.

Nothing fancy about this script. It produces a CSV file that looks something like this.

If we go for one of the Historical intervals instead of the Realtime interval, we again get an average value but with a difference. It’s the vCenter aggregation jobs, that calculated the average.

For example, the aggregation job that creates the statistical values for Historical Interval 1, which uses a 300 seconds interval, will take all the Realtime values, which uses a 20 seconds interval, that fall inside the 300 seconds interval and calculate the average. The aggregation job will take 15 Realtime values and calculate 1 Historical Interval 1 value from these.

It’s obvious that this will flatten your values, in other words extreme values, valleys and peaks, will be flattened.

And your data graph will become flatter and flatter, the farther in the Historical Interval series you fetch your data. An example, we collected Realtime values for 1 hour from an ESX(i) server and then we simulate what the aggregation job for Historical Interval 1 is doing.  If we graph these 2 data series it becomes obvious what I tried to explain in the previous section.

The following script will retrieve the realtime average write rate for a specific datastore for the last 30 minutes.

Annotations

Line 6: The datastore.write.average metric uses instances. The instance is the Uuid of the datastore. Note that this way of retrieving the Uuid only works for VMFS datastores.

The result is a nice CSV file with the average write rates over 20 seconds intervals.

Now let’s simulate what the aggregation job for Historical Interval 1 doing.  And display both series as a graph. For this I used the amazing Out-Chart cmdlet from the PowerGadgets snapin.

Annotations

Line 3-8: This loop simulates what the aggregation job is doing, it takes 15 twenty seconds intervals and calculates the average. This is the new average write value for the five minutes interval. To get this in the data array, the script adds a new property, called WriteAvgHI1.

Line 10-15: The Out-Chart cmdlet produces a graph of both series.

The script produces the following graph.

You can see immediately that a lot of the detail in the WriteAvg curve is gone when switching to the WriteAvgHI1 area.

Conclusion: depending on what you want to do with the data, make sure you select the correct interval.

Latest

This roll up type is described as “The most recent value of the performance counter over the summarization period. “.

Let’s have a look at a metric that has this roll up type. As an example I will take the sys.uptime.latest metric, that returns, in seconds, the time since the last system startup. It’s clear that we only want to most recent value for this metric.

Annotations

Line 4: Seen the the type of value this metric returns, we definitely want the value form the Realtime interval and we only need 1 value, hence the MaxSamples parameter.

Line 7: The value that is returned is in seconds. Since this unit isn’t very meaningful when dealing with large numbers, the script uses the New-Timespan cmdlet to get a more readable display of the returned value.

The resulting CSV file looks something like this

The Uptime value is expressed in a dd.hh:mm:ss format through the use of the New-Timespan cmdlet.

A handy metric to include in your regular vSphere reports.

Minimum/Maximum

I handle these roll up types in the same section, since they are essentially the same. And on top of that they are practically the same as the Average roll up type.

Two points to watch out for when you want to work with these roll up types:

• In the Historical Intervals the Minimum and Maximum roll up type metrics are only available when you select Statistics Level 4.
• The aggregation jobs use the average of the minimums and the maximums to calculate the value for the next Historical Interval

Summation

Metrics with this roll up type return the sum of all values of the metric over the interval.

The practical problem here can be that you want to have values over a standardized time unit. For example instead of reporting how many packets were transmitted over a specific vnic during a 20-second interval, you would most probably want to report the value as packets per second.

Not a big deal when using PowerShell as the following short sample will show.

Annotations

Line 9: Notice how the script uses the IntervalSecs that is present in the returned statistical data. This avoids hard-coding these interval values in your scripts.

The result looks something like this

None

According to the manual this one is simple, no roll up is done on the metric.

But are there any metrics of this type ? To be honest, I did a search but till now couldn’t find any metric with this roll up type.

If someone knows such a metric, please let me know and I will update this post.