Resque losing connection to redis on heroku

Question

I'm running an app on heroku. The web worker is developed in the Grape framework.

As Grape doesn't have anything like config/initializers, I'm running a code like this before each access to Resque:

HEROKU_REDIS_URL = "redis://redistogo:XXXXX@squawfish.redistogo.com:9990/"
uri = URI.parse(HEROKU_REDIS_URL)
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :thread_safe => false)

After a while the Resque worker stops picking up jobs from Redis, even though some are queued there. When running locally everything works fine.

Any idea what am I doing wrong? Or where should I put the Resque's Redis initialization?

Answer 1

So after quite a few hours of debugging I made it work. Let me share what parts were wrong in my setup:

• :thread_safe => true: Redis instance should be created as thread safe. Not sure what it means in this context, haven't found anything in the Redis source code, but that's what they suggest in: http://redistogo.com/documentation/resque

• Make sure redis is assigned to Resque only once both for web server and the worker process: I ended up with a code like this:

    require 'resque'
    if ENV["REDIS_SERVICE"]
      begin
        Resque.redis.get('abc')
      rescue Redis::CannotConnectError
        HEROKU_REDIS_URL = "redis://redistogo:XXXXX@squawfish.redistogo.com:9990/"
  
        uri = URI.parse(HEROKU_REDIS_URL)
        Resque.redis = Redis.new(
          :host => uri.host,
          :port => uri.port,
          :password => uri.password)
  
      end
    end
  

  I placed it in app/resque_redis_init.rb and required both in the web app and the worker Rakefile in the resque:setup task

• Make sure nothing is using the same Redis server: I found out that a worker in another heroku app has this redis set up for resque and keeps eating the jobs out of the queue. The worker was misconfigured and the jobs were just failing without me noting it
• Make sure there's no rubbish in your Redis: I had to purge the whole contents of Redis to make sure there are no wrong workers registered.

• Make sure you're queuing the jobs in the right queues and that these have the right priority For me, it was the following setup in the Procfile

  resque: TERM_CHILD=1 VERBOSE=1 QUEUES=activity,polling bundle exec rake jobs:work
  

With this setup it works beautifully. Hope it helps somebody, as there are not much docs out there for non-rails apps.