Do your user accounts match in both username and password on the local and remote machine? If I understand your question, that may be the issue.
If you're on a domain, use the same user account on both machines. If you're not, create identically named and passworded accounts on both machines, and use those for the local VS user, and the remote debugger user.