The following code is what I ended up with (the buffer size will change in production)
This gets a file from a url, and starts streaming it chunk by chunk to a client through my server. The part that took me a bit to figure out was flushing the Response after each chunk was written.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(digitalAsset.FullFilePath);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Response.ClearHeaders();
Response.ClearContent();
Response.Clear();
Response.ContentType = "application/octet-stream";
Response.AddHeader("Content-Disposition", "attachment;filename=" + digitalAsset.FileName);
Stream inputStream = response.GetResponseStream();
byte[] buffer = new byte[512];
int read;
while ((read = inputStream.Read(buffer, 0, buffer.Length)) > 0)
{
Response.OutputStream.Write(buffer, 0, read);
Response.Flush();
}
Response.End();
}
This can stream any size file through my server without waiting for my server to fist store it into memory or onto disk. The real advantage is that it uses very little memory as only the buffered chunk is stored. To the client the downloads begins instantly. (this is probably obvious to most but was very cool for this novice to see) so we are simultaneously downloading the file from one location and uploading it to another using the server as a sort of proxy.