I finally found the answer to this in a google-groups discussion about another ORCA related question, it just happened to include the database.yml config that I needed.
To have the database external to the OCRA compiled exe, such that the database file resides in the same directory as the .exe, set the database:
parameter in database.yml to this;
<%= File.expand_path('../production.sqlite3', ENV["OCRA_EXECUTABLE"]) %>
My compiled app runs in production mode so this is now how that part of my database.yml looks;
production:
adapter: sqlite3
database: <%= File.expand_path('../production.sqlite3', ENV["OCRA_EXECUTABLE"]) %>
pool: 5
timeout: 5000
This works perfectly now, I can run the app.exe make a new entry in the DB and then terminate app.exe. When I run app.exe again the changes to the data are still there.