Why WP-Cron sucks

In WordPress you have a small file called wp-cron.php, which simulates normal cron-jobs. except this file isn’t executed every x amount of minutes. Instead WP-Cron will execute when someone visit the website.

The reason why the wp-cron was build, was because some of the actions it does, can take longer than we want the user to wait, so it will run as a separate process in the background. This is good because it means that it won’t affect the user, in most cases.

So here’s why WP-Cron sucks.

If you have a high traffic site, and the WP-Cron is executed on every visit, this might cause multiple processes that is getting started, to run the cron. WordPress is smart enough, to make a lock on the cron itself, so multiple crons won’t loop over the same events, which really helps a lot.

But there is quite a few plugins, that hooks into the WP-Cron, this results in that those plugins will do their thing for every visit. This is usually fast, it takes maybe 100-500ms to execute the full cron, and doesn’t really matter. But lets say you’re having a high traffic site, this means you can get a lot of visits, which means it will spawn multiple crons, in the beginning it seems fine, but the more visits you get, the more processes will get started, this takes CPU time, and memory of the server. The more scripts you put in the ‘queue’ for the CPU, to do the calculations, will result in that the crons start taking longer and longer time. And might end up taking multiple seconds.

A small example was a site, getting around 200 visits within a minute, this started around 90 wp-crons. It started to fill up in the queue for the CPU, which resulted in slower processing of the wp-cron.php. Not only did it become slower, but it also resulted in 150 PHP processes running in the background for that specific site. This took some CPU, but especially memory was used, because each process was taking 18 megabyte of memory, due to some hooks in the WP-Cron. So that minute, the memory usage for that specific site increased 2700 megabyte.

There was a PHP memory limit of 256 megabyte, but this memory limit is for each php process running, and since the WP-Cron starts another process, this means that it doesn’t hit the limit at all.

Since the server had a lot of memory it didn’t really matter at all, but let’s say a few sites on a server is running with this amount of memory, it will eat the memory pretty fast. There’s systems like CloudLinux that fixes this problem, because you allocate x amount of memory to each user, and when the user reaches this limit, it won’t use more. But this means if it’s executing something, it will become slower for the end user, because it’s not able to use more memory, so it will save the server indeed. But might slow down the site a lot.

Another way to fix this, is to disable the cron in the wp-config.php

define('DISABLE_WP_CRON', true);

The problem with disabling the cron is, that some features of WordPress will stop working, things like scheduled posts doesn’t work, because this is depending on the cron itself, and  also other plugins that is hooked into the wp-cron will stop working.

So a way to fix this for a high traffic site, is to disable the cron with the code above, and set up a cron manually in your control panel. This can be done like this:

*/5 * * * * wget -q -O - "http://mydomain.com/wp-cron.php" > /dev/null 2>&1

#Sometimes it might be required to run PHP directly:
*/5 * * * * php /home/$USER/public_html/wp-cron.php

#You can also do it using curl:
*/5 * * * * curl -vs -o /dev/null http://mydomain.com/wp-cron.php > /dev/null 2>&1

The code above will run the cron for every 5 minutes, and redirect all output to /dev/null.
Running it every 5 minutes means it will run 288 times every day. Which should be fairly enough. You can even set it to every 10, 15, or 30 minutes. Based on what you prefer.

So setting the cron to a run every x minutes, will first of all lower the load on the server, and maybe even make your site load faster if you had problems, with slow loading times, when you have a lot of visitors coming in.

Multisite

If you have a wordpress multisite, and want to do the above, you have multiple ways of doing it.

Lets say you have a wordpress multisite with following domains:
1: http://domain1.com/
2: http://domain2.com/
3: http://domain2.com/blog/

Solution 1

What you can do, is using the method above, by disabling the wp-cron just the way you do for a normal wordpress site.
Then you can set up 3 cron jobs – all 3 using the command above, but replacing the domain. This means you will have 3 cron jobs running doing their own wp-cron.php

Solution 2

This sucks if you have, lets say 100 sites, so there is another solution for this:

Creata a file in / called wp-cron-multisite.php:

<?php
require('./wp-load.php');
global $wpdb;
$sql = $wpdb->prepare("SELECT domain, path FROM $wpdb->blogs WHERE archived='0' AND deleted ='0' LIMIT 0,300", '');

$blogs = $wpdb->get_results($sql);

foreach($blogs as $blog) {
    $command = "http://" . $blog->domain . ($blog->path ? $blog->path : '/') . 'wp-cron.php';
    $ch = curl_init($command);
    $rc = curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE);
    $rc = curl_exec($ch);
    curl_close($ch);
}
?>

What you then can do, is to use the command:

*/5 * * * * wget -q -O - "http://mydomain.com/wp-cron-multisite.php" > /dev/null 2>&1

Whats nice about solution 1 is that you’ll be able to set a frequency for each site you have in your network. So if you have a site that only needs to be updated every hour, but another needs to be updated every 5 minutes. Then you should go for solution 1.

But if everything just should update at same time, and same frequency, and that you have a lot of sites, then solution 2 should be the easiest one.

Update 1:

I found out that there was a bug in the cron when using ‘date +%s’ – this isn’t supported in cron jobs, so the % needs to be escaped  like this:  ‘date +\%s’. The code blocks is updated.

Update 2:

Thanks to Johann for finding errors in the multi-site script! Code above is corrected 🙂