I figured this out. Although the docs say that the flying-sphinx gem does all the configuration for you after the inital configuration. You still have to run these two commands:
$ heroku run bundle exec flying-sphinx index
$ heroku run bundle exec flying-sphinx start