Quite a while ago we added Histograms into our monitoring portal, and ever since then I’ve always found them to be incredibly useful!
I was recently talking to a prospect (who didn’t have as much of a passion for stats as we do at SC Towers) about the value of Histograms and they were questioning in what situation they should be using them. We had been talking about longer term analysis of performance data, specifically identifying when and to what degree their page performance varied from the average.
Quite naturally, they had been using the Daily Average report, showing mean, min and max values. This report is useful but the one thing it doesn’t allow you to investigate is the spread of the test results which had strayed beyond the mean.
Let me give you an example.
Using the Daily Average Speed report to look at performance data over a 7 day period gives you the following:
This gives a good indication of the mean variation throughout the week, and shows us just how bad (by bad I mean how far from the average) some of the results have been, but it doesn’t tell us how many of the results were on or near to the max.
By using a Histogram we can determine exactly what the spread of results is. The histogram below was generated for the same page monitor over the same time period.
Here we can see that although the majority of test results have been placed in the 8-9 seconds ‘silo’, which is in line with the average reported above, almost 40% of the total number of results returned are outside of this average (either because the page returned quicker than expected or because it was much slower to respond with a full page load).
The gamma distribution on the histogram below is much more severe, but could quite easily be produced using the same or similar data from the Average Speed report at the top of this blog.
This shows us that averages can hide a lot of sins!
So, why are histograms cool? Well, in my mind anything that gives me some extra information is cool. So the fact that these histograms have shown me the variance in these test results is a good thing.
They’re also cool because they can show things like this. The Histogram below is definitely what you don’t want to see being produced by your platform, as it is highlighting massive performance variations, in this case at different times of the day. During the quiet periods the site is relatively stable, but then as things get busier the site starts to slow down resulting in the ‘rock horns’ effect.
|Some Rock Horns!
And anything that makes the rock horns is cool by me!