Pingdom and response times

I personally love Pingdom, I think they provide a good service for uptime monitoring, also because they verify if your site is actually down or not, by checking with multiple nodes.

One thing I don’t like with Pingdom, is their ‘response time report’ that you can see for each test you have, because these tests can not be compared to other sites if you want to do so. You can only use it as a reference for your site, but never compare it, because you compare 2 sets of data, which didn’t have same source.

When using Pingdom, you have a a set of servers across Europe and United States. Because Pingdom limits the amount of locations for each site to 9-10 probes, it means that they have different probe groups. These groups is made up of servers from both US and EU to make a balance, they try to make it around 50/50 for each part of the world.

This means they have multiple groups, and when creating a test, one of these groups will be selected for your test. This means that each test you have, might have a different probe group, which affects the performance quite a lot.

To confirm this, I made 2 tests in Pingdom, each test was testing the same site on same server, same interval (1 minute) and protocol. – So everything was exactly the same, except for the probe group I was in.

Below you see two images, these images shows data from the last 24 hours:

test-1 test-2

So the data above, the ‘Overall Average’ is based on all tests performed in one day, the slowest average is for every ‘rotation’ in the probe server, and was the slowest run (This is usually the last group of samples in Pingdom, since all data isn’t in yet), and then the fastest.

As you see there’s quite some difference in the speed, to be exact the first result is on average 40.58% slower, just by using another probe group at Pingdom.

So a tip, if you want to compare performance between two sites, pingdom is not the right tool to use, because each test is getting data from different sources (probes).

To get a realistic result, always use same settings, and same server location to run tests from, else it will not be realistic, and stuff can look slower, but might be same speed if you was running on same probe group, or might even be faster.