It seems as if your version of rvm might be using a local install of ruby vs. the OSX system install (which xcode uses).
Type
env
...into terminal and post your output. If you have to type
sudo gem install __
versus
gem install __
that's a good indicator also.