The answer is quite simple although it took me hours to figure it out. The Heroku site is not being added to your known hosts.
In Eclipse, go to Window -> Show view -> Other... -> Git -> Git repositories. From there, clone the repository. Once you clone it, import it from the file menu. Do this since when you clone the repository, it does not add it to the explorer view. Now you should have the Git repository and the explorer view.