This might not be the best way, but it could get you close to removing rails 4.
If it rails 4 was installed by apt, you could get apt to remove it.
First comfirm rails 4 was installed by the package manager:
Run which rails
and see where the rails command lives.
Then run dpkg -L rails
which will list all the files installed with apt for the package rails.
You could of course do this in one line
dpkg -L rails | grep `which rails`
Then if your result from which matches, then remove this using apt.
apt-get --purge remove rails
Hope this helps,
Will