Iron.io Push Queue and Laravel 4 - preventing the queue request from running multiple times

Question

I have a push queue set up to dial quite a few phone numbers and play back a recording - a blast announcement system powered by Twilio. It takes time to iterate through each number and place the call, so I am hoping to use a push queue to speed up the navigation of the app.

I attempted to use Iron.io push queues in the past with Laravel 4 and it seems that any task that takes a while to run, or if the HTTP request was slow at first, the code within the fire() method runs multiple times, even with $job->delete()

Here is an example of my queue handler -

    class callLotsOfPeople {
    public function fire($job, $data) {
        // Do stuff with data, like calling lots of people.. takes time
        $job->delete();
        // For some reason this method can be called multiple times after a single queue push, 
        // resulting in multiple phone calls and angry clients
    }
}

Answer 1

I've yet to leverage Push Queues fully, but a quick glance at IronMQ docs revealed the following:

Long Running Processes - aka 202’s

If you’d like to take some time to process a message, more than the 60 second timeout, you must respond with HTTP status code 202. Be sure to set the “timeout” value when posting your message to the maximum amount of time you’d like your processing to take. If you do not explicitly delete the message before the “timeout” has passed, the message will be retried. To delete the message, check the “Iron-Subscriber-Message-Url” header and send a DELETE request to that URL.

via: http://dev.iron.io/mq/reference/push_queues/#long_running_processes__aka_202s

Now, the timeout isn't something that Laravel seems to support at the moment since the payload is created behind the scenes with no easy access. You can create a Pull Request on the 4.1 branch to implement this functionality specifically for Iron push queues (tip: you'd need to edit both QueueInterface and all Queue drivers' push() function).

As a work-around, maybe you can just $job->delete() from the get-go (rather than after the time-consuming task) and just Queue::push() it again (or some chunk of it) if there are errors? Something like:

    class callLotsOfPeople {
        public function fire($job, $data) {
            $job->delete();

            // Do stuff with data, like calling lots of people.. takes time

            if ($error) {
                    Queue::push(...);
            }
        }
    }

Let me know how it goes, I may have a similar situation in the future and would like to know how you solve it!

Answer 2

This might be too late, but I had the same issue. I found a fix and have submitted a pull request to have it included in Laravel 4.1.

Basically, the code changes allow you to pass an $options array like this

Queue::push('MyJob', $message, $queue, array('timeout' => 300));

The functionality is already there in the IronMQ.class.php, but I couldn't find an easy way to pass them in from Laravel. Hopefully they include this, makes the multiple job submission issue go away. :-)

https://github.com/laravel/framework/pull/3555

EDITED: Changed Queue::pull to Queue::push, small typo.