How to get http response Body when the status code is not valid?

Question

I'm trying to get a html page content from a private service. The page returns status code 0, which is not valid. But the page do render when browsing through a browser.

When I try use WebResponse.GetResponseStream() it simply returns empty stream. And when I try use HttpClient.GetStringAsync(url).Result, it throws AggregateException as follow:

System.AggregateException was unhandled
  HResult=-2146233088
  Message=One or more errors occurred.
  Source=mscorlib
  StackTrace:
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at DiagnosticInfoCrawler.Program.Main(String[] args) in c:\TFS\MSNMetro\Tools\Verticals\Sports\DiagnosticInfoCrawler\Program.cs:line 47
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
  InnerException: System.Net.Http.HttpRequestException
   HResult=-2146233088
   Message=Response status code does not indicate success: 0 ().
   InnerException: 

Two types of code used as follow:

WebResponse response = GetWebResponse(url);
responseBody = (new StreamReader(response.GetResponseStream())).ReadToEnd();

HttpClient httpClient = new HttpClient();
httpClient.BaseAddress = new Uri(url);
pageSource = httpClient.GetStringAsync(url).Result;

Anyone can advise how can I get the response body? (given that I have no control to fix the service to return the correct status code.)

Thank you, DD

Answer 1

Didn't resolve this issue directly, but instead went to the approach to use TcpClient to read the response.

Answer 2

Catch WebException to get the response. This is a poor design decision by Microsoft since non 200 codes can be part of the normal case.

try
{
    WebResponse response = GetWebResponse(url);
    responseBody = (new StreamReader(response.GetResponseStream())).ReadToEnd();

    HttpClient httpClient = new HttpClient();
    httpClient.BaseAddress = new Uri(url);
    pageSource = httpClient.GetStringAsync(url).Result;
}
catch (WebException exception)
{
    var response = (HttpWebResponse)exception.GetResponse();
    //the response is here..
}