Sounds like your path is wrong. You certainly know that Heroku doesn't offer persistent storage? OpenShift does though.
On Heroku I'd go with the hosted GrapheneDB offering. Then you can use neography or neo4j.rb (in server mode) to connect to your neo4j db.
See also http://neo4j.org/develop/ruby