Schedule API calls to 1/second limit

Question

I'm building web app that uses Blekko API( web search API ). Application is multi-user.

I need to limit calls to API to 1[call/second]. This limit should apply to all activities by all users i.e. there should be some schedule for using API.

I need some sugesstions how to do that?

Answer 1

It sounds like responsiveness to the API calls isn't too important since you are talking about queueing. If that's the case, I would dump the API request URL into a database table. Then with a background worker process, I would do something to this effect:

set_time_limit(0);

$api_requests = array();
while (TRUE)
{
    if (count($api_requests) == 0)
    {
        // get multiple records from DB to limit requests and add
        // to the $api_requests array.

        // if DB returns no results, maybe sleep a few extra seconds
        // to avoid "slamming" the database.
    }

    // get the next API request from the array
    $request = array_shift($api_requests);

    // send API request to Blekko

    // process API results

    // sleep 1 sec
    sleep(1);
}

This is a bit of a "busy" loop, but it will ensure that you never run more than one request per second and also guarantees that a queued request won't wait too long to be processed.

Note: This method does require that your server won't kill the process itself, regardless of the set_time_limit() call. Long running processes are oftentimes killed on shared servers.

Answer 2

A simple way to do this is to use usleep()

usleep(1000000); will pause the script for 1.0 seconds