Jobs processing in background from web application

Question

I want to schedule and run a lot of jobs in the background during a web application execution.
The web app is built on top of Symfony 2 and Doctrine 2.

I know the job-processing can be done with libraries like Resque or Sidekiq. However, these libraries and my application are written in different languages, so I am wondering how I can run Sidekiq jobs written in Ruby which should integrate with my app written in PHP.

What I'm asking myself is if the only way to do this is rewriting a large amount of code to query the database from PHP to ruby, to be able to execute the job in Sidekiq/Resque.

Answer 1

Have you taken a look at Gearman? It lets you run background jobs just like Sidekiq, but it's language agnostic. For example, you can use PHP for everything, or you can queue up jobs in PHP and have the actual workers written in Ruby.

Answer 2

I have made use of Resque in several projects using https://github.com/chrisboulton/php-resque and https://github.com/chrisboulton/php-resque-scheduler

Its been working really well, I even made a Symfony bundle to make working with it really easy. https://github.com/mcfedr/resque-bundle - supports background jobs and scheduled jobs. Much more powerful than using cron.

The main reason for choosing Resque over other options is that it works on Redis, which is easy to deploy and scale. On AWS I use Elasticache managed instances for a completely hassle free setup.