Using redis in jruby threaded setup

Question

So the redis gem is supposed to be thread-safe, that's good. But I am wondering how I need to set it up. I am using jRuby and Celluloid (using default fibers).

The way I'm using it now is I have a global method defined:

def redis
  @_redis ||= Redis.new(...)
end

I have the timeout parameter for the Redis client set to 30. I don't think there are any thread-safety issues coming up for me (I do initialize it before spawning threads), however sometimes I start getting Redis::TimeoutError errors, so I am wondering if those threads might be keeping open connections forever and eventually taking up all available ones?

Basically the problem is that sometimes I will get Redis::TimeoutError. Is there a way I could check which connections are open at some moment? Or any idea what the issue could be?

Thanks

Answer 1

The problem here is, that the redis client locks exclusively (you can verify that by measuring the time that it takes for a command for 1,2,3 etc.. threads, it will most probably be linear).

There are two solutions here:

1. create a new client for every thread (be careful it will leave you with a lot of open files and for a long running process, it will kill your machine)
2. implement a connection pool
3. use this gem as your connection pool (I haven't tested it yet)