Dalayed Job execution expired on sending email

Question

I'm using Delayed Job gem to dispatch emails using Amazon SES. However it runs occasionally into execution expired error. Probably some connection issues to SES. I'm wondering how can I catch this exception and re-schedule the email manually? Due to internal reasons Delayed Job is configured to single max attempt

| execution expired
/usr/local/lib/ruby/2.0.0/net/smtp.rb:540:in `initialize'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:540:in `open'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:540:in `tcp_socket'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:550:in `block in do_start'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:549:in `do_start'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:519:in `start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/network/delivery_methods/smtp.rb:112:in `deliver!'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/message.rb:2129:in `do_delivery'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/message.rb:232:in `block in deliver'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/actionmailer-4.0.0/lib/action_mailer/base.rb:456:in `block in deliver_mail'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications.rb:159:in `block in instrument'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications.rb:159:in `instrument'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/actionmailer-4.0.0/lib/action_mailer/base.rb:454:in `deliver_mail'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/message.rb:232:in `deliver'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/performable_mailer.rb:6:in `perform'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:105:in `block in invoke_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:102:in `invoke_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206:in `block (2 levels) in run'
/usr/local/lib/ruby/2.0.0/timeout.rb:66:in `timeout'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206:in `block in run'
/usr/local/lib/ruby/2.0.0/benchmark.rb:296:in `realtime'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:205:in `run'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267:in `block in reserve_and_run_one_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267:in `reserve_and_run_one_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:189:in `block in work_off'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188:in `times'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188:in `work_off'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:153:in `block (4 levels) in start'
/usr/local/lib/ruby/2.0.0/benchmark.rb:296:in `realtime'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:152:in `block (3 levels) in start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:151:in `block (2 levels) in start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150:in `loop'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150:in `block in start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7:in `block (2 levels) in <class:ClearLocks>'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `block (2 levels) in add'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `block in add'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:149:in `start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:104:in `run'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:92:in `block in run_process'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:255:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:255:in `block in start_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/daemonize.rb:82:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/daemonize.rb:82:in `call_as_daemon'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:259:in `start_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:296:in `start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:159:in `block (2 levels) in start_all'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:158:in `fork'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:158:in `block in start_all'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:157:in `each'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:157:in `start_all'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/controller.rb:80:in `run'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons.rb:197:in `block in run_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `catch_exceptions'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons.rb:196:in `run_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:90:in `run_process'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:83:in `block in daemonize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:81:in `times'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:81:in `daemonize'
/home/ubuntu/www/tf/current/bin/delayed_job:5:in `<main>'

Answer 1

delayed_job provides some hooks, you can use the error hook to rescheduled the task.

def error(job, exception)
  # if exception is the one occurring right now for expired execution
  # reschedule the job
  # be careful to avoid infinite rescheduling
end