The Rakefile is used to set up dependencies for running Rake within the context of your project. You have correctly described the purpose of the Gemfile.
Rake is an independent program from JRuby. It will be included in the bin folder of your JRuby install. By the time you reach your Rakefile, you are already in the context of the wrong version of Rake (the one included with the regular Ruby install).
I think what you are looking for is just a modification to your path. Include your JRuby bin folder in your path, and that will allow you to specify which version of Rake you want to use.
For the JRuby version:
jruby -S rake your:task
Or just the regular Ruby version:
rake your:task