Separating back end and front end servers in Rails application?

Question

I have a Rails application that does a lot of media encoding. I am handling it via background processes, but I see that CPU gets overloaded and the front end loading times are definitely slower than they should be (or were before the back end part became bigger).

So the problem: Rails app with media encoding features experiences CPU loads and front-end slowed down. Goal - decouple front-end and back-end (media encoding) part.

Question - what is the best approach to split an existing application into two parts (front-end part and back-end part)?

1) Is it a good idea to run two copies of app on two servers and make calls between then POSTing/PUTting information via HTTP (or connecting to a remote db)?

2) Is it a good idea to keep the CPU-intesive part wrapped up in Rails code, or should I aim to strip it off the Rails functionality?

If someone can point to a good guide on running multi-server Rails application, that would be great (search returns questions on multi-server Capistrano deployment, but I need some less specific recipes).

Answer 1

A common approach that works well is to use a work queue like Resque.

Managing the code
For ease of management, keep the processing code and the app "in the app". Deploy two app-servers but run resque workers on the processing server.

State changes
If the processing jobs relate to ActiveRecord persisted objects you can poll state from the front-end and periodically update it from the backend during the encoding process.

You might find it useful to use state machine.

Your problems have moved
Now you're cloud-scale™ :D More processing hosts running workers can be added if your queue gets too long. If your front-end is the only web-accessible host you can set up a rack middleware or run rainbows to proxy the processed results through the frontend to the client.

Sounds like an interesting project. Good luck!