You have to grant needed permissions on specific SharePoint sites related to projects too.
I recommend you to create AD security groups for each project, and grant permissions for every group respectively. That should solve your issues – users will then be able to access Shared documents in TFS projects.