My Home (~/) dir somehow got added as part of my git repo, now i can't remove it

Question

I noticed that my git status suddenly includes my ~/ dir. When I did a git add . -A It included some files that were recently included into the directory. I don't know how they got there in the first place, but when i call git rm --cached ~/ it refuses saying:

fatal: /Users/me/: '/Users/me' is outside repository

How can i get rid of this from the repository?

UPDATE: on request here is my git status output:

On branch temp
Your branch is up-to-date with 'origin/temp'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    modified:   gradle.properties
    modified:   gradle/plugins/idea.gradle
    modified:   gradle/plugins/info.gradle
    modified:   gradle/plugins/stage.gradle
    modified:   myproject.gradle
    new file:   ~/NewDir/my.ear
    new file:   ~/NewDir/build.gradle
    new file:   ~/NewDir/deploy-grenade
    new file:   ~/NewDir/deployment/install_template.py
    new file:   ~/NewDir/deployment/uninstall_template.py
    new file:   ~/NewDir/gradle.properties
    new file:   ~/NewDir/gradle/wrapper/gradle-wrapper.properties
    new file:   ~/NewDir/gradlew
    new file:   ~/NewDir/gradlew.bat
    new file:   ~/NewDir/settings.gradle
    new file:   ~/NewDir/timestamp.txt

Answer 1

Turns out the directory '~' is actually a Dir under my git repository and not the actual shell-resolved home directory. Thanks @Paul Hicks for the insight.