Suggestion: Don't bother with switching. Remove it from source control (SVN) and check it in anew (git). Or, if removing it by Visual Studio tools doesn't work, here's how you do it manually:
- Close Visual Studio;
- Delete the .svn folder
- Open the .SLN file in notepad, find the lines which specify the SCC provider, and delete them.
- Open the solution in Visual Studio again - it should not be under source control anymore and you should be able to check it in git.
Edit: Manually removing this from the .sln file did the trick:
GlobalSection(SubversionScc) = preSolution
Svn-Managed = True
Manager = AnkhSVN - Subversion Support for Visual Studio
EndGlobalSection