Score:0

Cronjob Files Not Closing and 100% CPU usage

et flag

I'm running a php file every 10 seconds with a cronjob. While the system was running fine, I noticed last night that php was using 100% of the CPU and crashing the system.

When I checked with ps aux, I saw that my cron file had dozens of cron files in process. There wasn't even a reason for that when I checked the database. So my php page was supposed to return empty and finish instantly.

I have a few question:

1- Why can this malfunction in Cron be caused?

2- When I create a queuing system for eg codigniter instead of a Cron process, how is it different from cron?

3- Will I be free from such problems when I make PHP a system running in the background?

thank you advence

Score:0
vn flag
  1. Why can this malfunction in Cron be caused?

This is not a cron issue.

Cron is a very simple scheduler that simply starts a new batch at the time/interval you specify.

When, for whatever reason, your batch job can't complete in time before the next iteration starts you end with two concurrent processes. Cron does nothing to prevent that.

It really depends on your assumptions and code if that immediately causes an issue, or not.

It might be that immediately the second iteration creates some form of deadlock and stalls indefinitely, 10 seconds later the third iteration stalls on the deadlock caused by the second iteration and 10 seconds later the next iterations also stalls immediately. Etc. Etc. That can quickly snowball.

Or it might be a much slower process, where for each individual batch the runtime slowly increases and rather than stalling more and more longer running batches end up running concurrently. That (eventually) increases the load even more, starts resource contention and results in batch jobs that take even longer to complete and more simultaneous/concurrent batches and resource starvation.

  1. When I create a queuing system for eg codigniter instead of a Cron process, how is it different from cron?

As you experienced here, cron is not a queuing system. It is only a very simple scheduler.

If you want to add more complex job control you need to create that. See for example this answer here on how to simply prevent multiple cron batches from running concurrently in your batch specification.

Otherwise, indeed running something more advanced than simple cron may be a solution.

  1. Will I be free from such problems when I make PHP a system running in the background?

If the root cause is the fact that your code breaks whenever two batches run concurrently then measures designed to make your code run faster will neither eliminate that flaw, nor prevent that from ever happening again.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.