The issue was I had used a Named Instanced on SQL Server which by default uses dynamic ports. Hence each connection attempt used a different Port, and each time I was asking the server administrator to allow additional ports. The successful log ins where simply because the dynamic port chosen just so happened to be one previously allowed.
The answer was to use SQL Server Configuration Manager to remove the dynamic port setting and specify a single port to use for all connection attempts, and ensure firewalls etc had an exception for that particular port.