I eventually found out that this was caused by the application sending GZipped contents without the header that indicated to the browser that is was in fact GZipped, so it just dumped the binary content out. I got round this by adding the following to my Global.asax
Sub Application_Error()
Response.Filter = Nothing
End Sub
This will strip the filter out of the response when an error occurs, and my error message appears.
All credit to Rick Strahl with this blog post for the workaround. It's not ideal but it works for me and that's all I am really bothered about right now.