So this is how I got it to work:
I installed the Dokku Shoreman plugin and then created a Procfile
in the apps root folder like this:
web: bundle exec rails server -p $PORT
worker: bundle exec rake jobs:work
Now everything works like it should.