Not that I think this is a good solution, but what I did (and what finally seems to bring me back to rails 3.2) is the following:
- Removing all gems with
gem list | cut -d" " -f1 | xargs gem uninstall aIx
(some had to be removed manually) aptitude purge ruby
aptitude install ruby
gem install rails