Download file in chunks (Windows Phone)

Question

In my application I can download some media files from web. Normally I used WebClient.OpenReadCompleted method to download, decrypt and save the file to IsolatedStorage. It worked well and looked like that:

 private void downloadedSong_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e, SomeOtherValues someOtherValues) // delegate, uses additional values
        {
            // Some preparations

                try
                {
                   if (e.Result != null)
                   {
                    using (isolatedStorageFile = IsolatedStorageFile.GetUserStoreForApplication())
                    {
                        // working with the gained stream, decryption
                        // saving the decrypted file to isolatedStorage
                        isolatedStorageFileStream = new IsolatedStorageFileStream("SomeFileNameHere", FileMode.OpenOrCreate, isolatedStorageFile);
                        // and use it for MediaElement
                        mediaElement.SetSource(isolatedStorageFileStream);
                        mediaElement.Position = new TimeSpan(0);
                        mediaElement.MediaOpened += new RoutedEventHandler(mediaFile_MediaOpened);

                        // and some other work
                     }
                    }
                 }
                 catch(Exception ex) 
                 {
                  // try/catch stuff
                 }
           }

But after some investigation I found out that with large files(for me it's more than 100 MB) I'm getting OutOfMemory exception during downloading this file. I suppose that's because WebClient.OpenReadCompleted loads the whole stream into RAM and chokes... And I will need more memory to decrypt this stream.

After another investigation, I've found how to divide large file into chunks after OpenReadCompleted event at saving this file to IsolatedStorage(or decryption and then saving in my ocasion), but this would help with only a part of problem... The primary problem is how to prevent phone chokes during download process. Is there a way to download large file in chunks? Then I could use the found solution to pass through decryption process. (and still I'd need to find a way to load such big file into mediaElement, but that would be another question)

---

Answer:

 private WebHeaderCollection headers;
 private int iterator = 0;
 private int delta = 1048576;
 private string savedFile = "testFile.mp3";

 // some preparations
 // Start downloading first piece


using (IsolatedStorageFile isolatedStorageFile = IsolatedStorageFile.GetUserStoreForApplication())
                    {
                        if (isolatedStorageFile.FileExists(savedFile))
                            isolatedStorageFile.DeleteFile(savedFile);
                    }

                    headers = new WebHeaderCollection();
                    headers[HttpRequestHeader.Range] = "bytes=" + iterator.ToString() + '-' + (iterator + delta).ToString();
                    webClientReadCompleted = new WebClient();
                    webClientReadCompleted.Headers = headers;
                    webClientReadCompleted.OpenReadCompleted += downloadedSong_OpenReadCompleted;
                    webClientReadCompleted.OpenReadAsync(new Uri(song.Link));
                    // song.Link was given earlier

private void downloadedSong_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
        {
            try
            {
                if (e.Cancelled == false)
                {
                    if (e.Result != null)
                    {
                        ((WebClient)sender).OpenReadCompleted -= downloadedSong_OpenReadCompleted;

                        using (IsolatedStorageFile myIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
                        {
                            using (IsolatedStorageFileStream fileStream = new IsolatedStorageFileStream(savedFile, FileMode.Append, FileAccess.Write, myIsolatedStorage))
                            {
                                int mediaFileLength = (int)e.Result.Length;
                                byte[] byteFile = new byte[mediaFileLength];
                                e.Result.Read(byteFile, 0, byteFile.Length);
                                fileStream.Write(byteFile, 0, byteFile.Length); 

                                // If there's something left, download it recursively
                                if (byteFile.Length > delta)
                                {
                                    iterator = iterator + delta + 1;

                                    headers = new WebHeaderCollection();
                                    headers[HttpRequestHeader.Range] = "bytes=" + iterator.ToString() + '-' + (iterator + delta).ToString();
                                    webClientReadCompleted.Headers = headers;
                                    webClientReadCompleted.OpenReadCompleted += downloadedSong_OpenReadCompleted;
                                    webClientReadCompleted.OpenReadAsync(new Uri(song.Link));
                                }
                            }
                        }
                    }
                }
            }

Answer 1

To download a file in chunks you'll need to make multiple requests. One for each chunk.
Unfortunately it's not possible to say "get me this file and return it in chunks of size X";

Assuming that the server supports it, you can use the HTTP Range header to specify which bytes of a file the server should return in response to a request.
You then make multiple requests to get the file in pieces and then put it all back together on the device. You'll probably find it simplest to make sequential calls and start the next one once you've got and verified the previous chunk.

This approach makes it simple to resume a download when the user returns to the app. You just look at how much was downloaded previously and then get the next chunk.

I've written an app which downloads movies (up to 2.6GB) in 64K chunks and then played them back from IsolatedStorage with the MediaPlayerLauncher. Playing via the MediaElement should work too but I haven't verified. You can test this by loading a large file directly into IsolatedStorage (via Isolated Storage Explorer, or similar) and check the memory implications of playing that way.

Answer 2

Confirmed: You can use BackgroundTransferRequest to download multi-GB files but you must set TransferPreferences to None to force the download to happen while connected to an external power supply and while connected to wi-fi, else the BackgroundTransferRequest will fail.

---

I wonder if it's possible to use a BackgroundTransferRequest to download large files easily and let the phone worry about the implementation details? The documentation seems to suggest that file downloads over 100 MB are possible, and the "Range" verb is reserved for it's own use, so it probably uses this automatically if it can behind the scenes.

From the documentation regarding files over 100 MB:

For files larger than 100 MB, you must set the TransferPreferences property of the transfer to None or the transfer will fail. If you do not know the size of a transfer and it is possible that it could exceed this limit, you should set the value to None, meaning that the transfer will only proceed when the phone is connected to external power and has a Wi-Fi connection.

From the documentation regarding use of the "Range" verb:

The Headers property of the BackgroundTransferRequest object is used to set the HTTP headers for a transfer request. The following headers are reserved for use by the system and cannot be used by calling applications. Adding one of the following headers to the Headers collection will cause a NotSupportedException to be thrown when the Add(BackgroundTransferRequest) method is used to queue the transfer request:

• If-Modified-Since
• If-None-Match
• If-Range
• Range
• Unless-Modified-Since

Here's the documentation: http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh202955(v=vs.105).aspx