The TeamCity Symbol Server plugin page showed the url to use was
http://ourserver/app/symbols/
The fix was to drop the trailing slash
http://ourserver/app/symbols
Hitting that URL gives a 404 but that is the format that works for Visual Studio 2012 and when trying to verify the symbols via symchk.exe
eg:
symchk /r mylibrary.dll /s SRV*http://outserver.acp.net/app/symbols
Using that URL then renders for a basic auth logon prompt and symbols load correctly