How do I completely remove Gitosis and then run Gitolite?

Question

I removed the gitosis-admin folder, but naturally something is still screwed up when I run gitolite. I ran apt-get install gitolite and then gl-setup on the new .pub key I created and used to set up the passwordless access. The set up seemed to run fine, but now I'm prompted for a password (of course I am!) when running the git clone gitolite@server:pathgoeshere command.

I'm so tired. I have wrestled with gitosis for 2 weeks. PLEASE don't tell me installing Gitolite will be as painful. Should I remove all the stuff Gitosis polluted my server with to ensure Gitolite is working correctly? Where would it be. Consultation of the interwebs revealed multiple possible hiding places for Gitosis things...like keys. I removed the Gitosis folder in home.... so far that's all.