I have found the cause of the problem. I commented out all references to ClientDependency in the Web.config and now my site runs as expected. It seems as though that module is attempting to compress content but failing. This forum post helped locate the issue: http://our.umbraco.org/forum/ourumb-dev-forum/bugs/13525-Client-Dependency-Module-playing-havoc-with-gzip-content-compression
This causes some problems with the core product and is therefore not a fix, it does however give us a starting point for investigations into how the problem can be resolved.
I have updated the Umbraco issue tracker with this information. For those who are interested see here: http://issues.umbraco.org/issue/U4-2203