As far as i know this is not possible from Team Foundation Build. As the common scenario is that the build server is on a different box it uses the public URL.
you could try adding it to the hosts file. \Windows\system32\dricers\etc contains a files called hosts. If you add an entry for the server name to the loopback address (172.0.0.1) then you might be able to get around the problem temporarily.
p.s. Why does the firewall come into play on your internal network?