The git.exe of "GithHub for Windows" is located at:
C:\Users\<user>\AppData\Local\GitHub\PortableGit_{hash}\bin\git.exe
It seems like the hash in the path is different on different systems.
By default Github for Windows clones github repos over HTTPS. Just for your demonstration purpose, in the VCS root setting of Teamcity build, you can use https url of your Github repo and choose "Password" as the "Authentication Method" in the "Authentication settings" section.
If you use "agent-side checkout", using "Password" as authentication method is supported only if you have Teamcity 7.1.2+ and git version 1.7.3+. See Git VCS settings Teamcity and Teamcity issue TW-18711.