"Seeing all repos" means you are somehow bypassing gitolite.
An ssh connection to a gitolite server must always be done with the account which manages gitolite repos: git or gitolite (I doubt it is gitosis, as in your question), you would never connect as user1.