Issues with EventMachine (and looking into Sinatra Async)

Question

I've been trying to find a good way of dealing with asynchronous requests and organizing jobs that need to be repeated, and eventmachine seemed a good way to go, but I found some posts trying to discourage users from eventmachine (for example https://github.com/kyledrake/sinatra-synchrony). I was wondering what the issues they are referring to are? (and if someone would be nice enough, what the alternatives are?)

Answer 1

Considering you're basically searching for a job queue, take a look at Background Jobs at Ruby Toolbox and you'll find a plethora of good options. Manageability vs Speed goes something like this,

1. Delayed Job
2. Sidekiq/Resque
3. Beanstalkd

with DJ being slowest and most manageable and beanstalkd being fastest and least manageable. Your best bet is probably sidekiq or resque, they both depend on redis for managing their queue.

I'd discourage you to use EventMachine because:

1. It's hard to reason about the reactor pattern.
2. Fibers detangle reactor pattern's callback pyramid of doom into synchronous looking code but fiber support in third party apps tend to bite you.
3. You're limited to a very limited eco system when it comes to net-related code.
4. It's hard not to block the reactor and it's often not easy to catch it when you do.
5. There are finished solutions for background processing, you don't need to code your own.
6. It's not really maintained any more, just take a look at last commits and issue list on github.
7. There's celluloid and celluloid-io and dcell.

Actually, the Sinatra Synchrony people sum it up good:

This gem should not be considered for a new application. It is better to use threads with Ruby, rather than EventMachine. It also tends to break when new releases of ruby come out, and EM itself is not maintained very well and has some pretty fundamental problems.

I will not be maintaining this gem anymore. If anyone is interested in maintaining it, feel free to inquire, but I recommend not using EventMachine or sinatra-synchrony anymore.

Answer 2

Use EM if it fits your workflow. Callbacks can be fine to work with as long as you don't get too crazy. We built a lot of software on top of EM at my last job.

There is pretty good support for third party protocols, just take a look at the protocol implementations page.

As to blocking the reactor, you just need to make sure you don't do work on the main thread, and if you do, make sure it's work you do fast. There are some things you can do to determine if this is working. The simplest is just to add a latency check into your code. It's as simple as adding a periodic timer for every x seconds and logging a message (in development). Printing out the time between the calls will tell you how lagged the reactor has become. The greater this time is then your x value the more work you're doing on the main thread.

So, I'd say, try it for yourself. Try celluloid, try straight up threads, try EM with EM-Synchrony and fibers.

It really comes down to personal preference.