Purging file from Git repo failed, unable to create new backup

Question

I tried to remove a file from my remote repo by running:

git filter-branch --index-filter 'git rm --cached --ignore-unmatch Rakefile' HEAD

But Git complains that

Cannot create new backup. A previous backup already exists in refs/original/
Force overwriting the backup with -f
rm: cannot remove /.git-rewrite/backup-refs : Permission denied
rm: cannot remove directory /.git-rewrite : Directory not empty

This was after I already deleted the .git-rewrite directory on Windows.

How can I remove that file? It's a 29Mb file sitting on my repo, so I quite need to remove the file.

I tried to delete the commit in git rebase -i, but apparently because the commit touched a lot of different files, Git complains of conflicts and I aborted to be safe.