delayed_job process silently quits

Question

I wish I had more information to put here, but i'm kind of just casting out the nets and hoping someone has some ideas on what I can try or what direction to look. Basically I have a rails app that uses delayed jobs. It offloads a process that takes about 10 or 15 minutes to a background task. It was working fine up until yesterday. Now every time I log onto the server, I find that there are no delayed job processes running. I've restarted, stopped and started, etc. a dozen times and am getting nowhere. The second it tries to process the first item in the queue, the process gets killed, and nothing gets logged to the log file.

I tried running it like this:

RAILS_ENV=production script/delayed_job run

Instead of the normal daemon:

RAILS_ENV=production script/delayed_job start

and that did not give me anymore info. Here is the output:

delayed_job: process with pid 4880 started.
Killed

It runs for probably 10 seconds before it just kills. I have no idea where to start on this. I've tried a number of things like downgrading daemon gem to 1.0.10 like suggested in other posts.

Any ideas would be amazing.

Answer 1

The solution to this in case anyone else comes across is, was that it was simply running out of memory and the OS was killing it.

I ran the jobs a couple times, and watched top while waiting. I saw the memory usage for that pid slowly climb till the process was killed and all memory released.

Hope that might help someone.