Use multiple Redis servers in Sidekiq

Question

I am using Sidekiq to process background jobs into one of our Rails project. We want to use a different Redis server located at different location to sepearate out ReportDB with other background processing job.

According to Sidekiq config wiki we can configure like

config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
   config.redis = { :url => 'redis://redis.example.com:7372/12', :namespace => 'mynamespace' }
end

Sidekiq.configure_client do |config|
   config.redis = { :url => 'redis://redis.example.com:7372/12', :namespace => 'mynamespace' }
end

But how can I initialise connection to multiple redis server?

Answer 1

Sidekiq 2 doesn't support multiple Redis servers, upgrade to sidekiq 3 which released just today and adds the new client sharding feature that you need.

From: Sidekiq 3 Release Note

Client Sharding

Sidekiq 2.x has a scalability limit: one Redis server. In practice the limit is greater than 5000 jobs per second on good hardware so to most it wasn’t a big deal but some of the more intense Sidekiq users were hitting that limit. An updated (Sidekiq 3) Sidekiq::Client API allows you to specify a Redis connection pool to use, rather than assuming the Sidekiq.redis global pool previously, allowing you to target different types of jobs to different Redis servers. Now you can scale Sidekiq to infinity and beyond!

The API is almost entirely backwards compatible: the only breaking change is in client-side middleware. See the upgrade notes for more detail.

===

Also take a look at the answer for - Work with two separate redis instances with sidekiq?