Sidekiq jobs stuck in enqueue

Question

Sidekiq has been working in development mode just perfectly. Now that I am trying to use it in production, all the jobs are just sitting in enqueue and aren't ever being run. Could anyone point me in the right direction as to how to solve this issue?

Answer 1

Please check if sidekiq process is actually running:

ps aux | grep sidekiq

If it is not, try to run sidekiq in foreground first and check the output.

bundle exec sidekiq -e production

Answer 2

In many cases for me it's because I haven't properly declared the queue for this specific service in config/sidekiq.yml.

Answer 3

This answer looks relevant: Sidekiq not processing queue If Sidekiq isn't told about the config file (which may require a different incantation in production) then it may not be using the right queue.

Answer 4

Just a heads up to all as I ended up this page trying to figure out why my Rails Sidkiq jobs were not moving off 'enqueued'.

1. Check your sidekiq console in your terminal
2. My issue which messed me up for almost an hour... duh... I had a 'byebug' breakpoint in the code that is silly to put in a background job. This will totally foul your job up. Obviously... I didn't mean to put that breakpoint there. I swear :)

Answer 5

My solution:

1.) Look at the Sidekiq Web UI for your app (this is where you can see the fact that jobs are getting into an enqueued state).

2.) Manually retry a failed job

3.) Inspect logs on the server running Sidekiq:

(generally this is where the log will be located, if say you are doing this on staging)

tail -f /var/www/yourappname/current/log/staging.log

This is generally where you will see a more detailed error message of why Sidekiq cannot process enqueued jobs. In our case, there was an environment variable pointing to an incorrect endpoint specific to our deployment configuration.

Answer 6

Make sure your app sidekiq process run

$ ps aux | grep sidekiq
server   26813  0.0  0.0  14228  1084 pts/11   S+   11:27   0:00 grep --color=auto sidekiq
server   27936  0.3  0.6 1178904 153240 ?      Sl   Apr17  46:02 sidekiq 5.2.7 app1 [0 of 10 busy]

The example show that there is only one sidekiq process running (app1). If your app not in the list, then you have to start sidekiq process for your app, in this example app2 . gem capistrano-sidekiq make your life easier:

$ cap production sidekiq:start
00:00 sidekiq:start
      01 $HOME/.rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /var/www/app2/shared/tmp/pids/sidekiq-0.pid --environment produ…
    ✔ 01 server@103.n.nnn.nnn 0.400s

and the app2 process will be activated. Now, check again

$ ps aux | grep sidekiq
server   25857  8.6  7.6 3073532 1870244 ?     Sl   11:15   1:02 sidekiq 5.2.8 app2 [0 of 10 busy]
server   26813  0.0  0.0  14228  1084 pts/11   S+   11:27   0:00 grep --color=auto sidekiq
server   27936  0.3  0.6 1178904 153240 ?      Sl   Apr17  46:02 sidekiq 5.2.7 app1 [0 of 10 busy]

the queue should have been running normally

Answer 7

Using capistrano-sidekiq gem solved our problem