Magento 2: CronJob bug? MySQL is always running at 30% usage and many php processes are running

Question

I am running magento 2 (2.2.1) on nginx and enable the cronjob by default.

php bin/magento cron:install 

I don't customize any configuration regarding schedule of cronjobs.

after running the machine a few days, i saw that mysql service is always nearly 30-40% usage and machine is running a lot of "php7.1" processes

i reboot the machine and the error is the same, right now there are only a "few" php7.1 processes but it continue to raise

I already checked the logs, no errors there or exceptions.

I only found an issue on m2 backend

this indexer is always not "up to date"

can you please give me a hint how i can fix this problem?

attached a screenshot from the crontab

Answer 1

My best guess,

See this at stackexchange and this at github. The issue somehow appears as some cronjobs aren't indicated as completed. It freaked me out for quite some time.

The quick diagnosis to see if you have the issue is to run below query and if it comes back with more than a 1200-1500 or so, you likely have the issue:

select count(*) from cron_schedule;

The quick fix is to delete most of the stuff that have status running in cron_schedule:

delete from cron_schedule where scheduled_at < date_sub(now(), interval 1 hour);

To avoid recurrence, establish a cron-job that periodically deletes the stuck jobs. Below job can be added in cron-tab. However, if you have it on a shared server, the MySQL password and username will be visible by other users running top -c and similar and it is not advisable, read more on guidelines for password security.

* */8 * * * <path_to_mysql_binary_dir>/mysql -u<sql_user> -p'<sql_user_pass>' <database_name> -e "delete from cron_schedule where scheduled_at < date_sub(now(), interval 1 hour)";

Adjust the * * * * * to fit the frequency, I don't see the need to run it very often, as I still haven't faced the same issue again after months of doing the quick fix.

Answer 2

There's a module to fix this issue:

https://github.com/magemojo/m2-ce-cron