I went ahead and forked the PDF creation tasks and tested it out. The forked processes do indeed release the memory back to the system after they exit, and our Delayed Job processes no longer balloon out of control. I have attached the code in case anyone is facing a similar problem. Interestingly enough, running Ruby 1.9.3 + Rails 3.2.x, we don't need to reconnect to the database in either the forked processes or the parent process. There was much debate about this in other Stackoverflow questions.
def run_in_fork
read, write = IO.pipe
pid = fork do
error = nil
read.close
begin
yield
rescue => e
error = e
end
Marshal.dump(error, write)
exit!(0) # skips exit handlers.
end
write.close
result = read.read
Process.wait(pid)
raise "Child process failed" if result.empty?
if exception = Marshal.load(result)
raise exception
end
return true
end
def my_method
object = MyObject.find.first
run_in_fork do
object.long_running_process
end
object.reload
end