Found the workaround (set :delayed_job_command, "bin/delayed_job") and hope this helps someone else!
deploy.rb:
require 'delayed/recipes'
set :delayed_job_command, "bin/delayed_job"
after "deploy:start", "delayed_job:start"
after "deploy:stop", "delayed_job:stop"
after "deploy:restart", "delayed_job:stop","delayed_job:start"