PHP execute in Background won´t work good without delay

Question

I need to ping a few servers every 5 minutes and this takes more than 30 sec overall. So I thought I use the exec command to do this in background:

function execInBackground($cmd) { 
    if (substr(php_uname(), 0, 7) == "Windows"){ 
        pclose(popen("start /B ". $cmd, "r"));  
    } 
    else { 
        exec($cmd . " > /dev/null &");   
    } 
} 

$result  = mysql_query("SELECT `id` FROM `servers` ORDER BY `id`");

while ($server_data = mysql_fetch_array($result, MYSQL_ASSOC)) {

    sleep(1);
    execInBackground("php server_updater.php " . $server_data['id']);

}

As you see I added a sleep of one second. Which I don´t want and doesn´t work as I have to check more than 30 servers. Well if I don´t set the 1 sec sleep, the update script fails for a few servers which are online:

The update script gets the id of the server which it should update from the script from above and than runs a query to check the servers status. If it succeeds it enters the data into the database if it fails it updates the server entry in the database to offline.

According to mysql_query, this only works with internal data, no possibility for the user to modify anything.

What do you think, why do some of the server checks fail if there is no sleep. Which optimazations or better ways could you recommend ?

Answer 1

Perhaps your (shared?) server runs out of resources (memory, number of processes, etc.) when you try to execute all 30+ scripts at the same time.

I would use something like a queue and a scheduler (cron in Linux) instead, see for example this answer. In summary you "check out" a batch of records to be updated and process them sequentially.

It would depend on the resources you have at your disposal, but you could run multiple jobs at the same time, each processing a part of your queue one after another (no need for async exec() calls).

You should also look into table locking to make sure no records get "checked out" by multiple versions of your script.

Answer 2

While not answering the backgrounding problem, this answer might be helpful anyway:

For storing time series data, RRDtool ist a great solution. It saves large amounts of data over time and convert this data into suchlike charts:

Additionally, I'd propose to use a specialist monitoring solution. One solution with a good visual presentation of system states ist CACTI. From the CACTI website: