Encode zip file to Base64

Question

I want decode zip file to base64 string. File can be up 100 MB and get OutOfMemory exception and restart Visual Studio. My Encode code:

public static string EncodeToBase64(string zipPath)
{
    using (FileStream fs = new FileStream(zipPath, FileMode.Open, FileAccess.Read))
    {
       byte[] filebytes = new byte[fs.Length];
       fs.Read(filebytes, 0, Convert.ToInt32(fs.Length));
       return Convert.ToBase64String(filebytes);
    }
}

What can I do to resolve it?

Answer 1

Don't try to do the whole thing in one chunk.

Loop through bytes from your FileStream. Grab multiples of three bytes and encode those. Be sure to use a StringBuilder to build your output.

That will greatly decrease memory usage.

Answer 2

[NOTE: This answer assumes that you can process the final base 64 string in chunks, for example by writing each block in turn to a stream of some kind.]

This is simplified if you write a helper method to read a file in byte[] blocks of a maximum size, e.g.:

public static IEnumerable<byte[]> ReadFileInBlocks(string filename, int blockSize)
{
    byte[] buffer = new byte[blockSize];

    using (var file = File.OpenRead(filename))
    {
        while (true)
        {
            int n = file.Read(buffer, 0, buffer.Length);

            if (n == buffer.Length)
            {
                yield return buffer;
            }
            else if (n > 0) // Must be the last block in the file (because n != buffer.Length)
            {
                Array.Resize(ref buffer, n);
                yield return buffer;         // Just this last block to return,
                break;                       // and then we're done.
            }
            else // Exactly read to end of file in previous read, so we're already done.
            {
                break;
            }
        }
    }
}

Then you can write a simple method to return a sequence of base 64 strings converted from each block of bytes from the file in turn:

public static IEnumerable<string> ToBase64Strings(string filename, int blockSize)
{
    return ReadFileInBlocks(filename, blockSize).Select(Convert.ToBase64String);
}

Then assuming you have a way of processing each converted base-64 string block, do something like this:

foreach (var base64String in ToBase64Strings(myFilename, 1024))
    process(base64String);

Alternatively, you could skip the intermediary ReadFileInBlocks() method and fold the conversion to base 64 string into it like so:

public IEnumerable<string> ConvertToBase64StringInBlocks(string filename, int blockSize)
{
    byte[] buffer = new byte[blockSize];

    using (var file = File.OpenRead(filename))
    {
        while (true)
        {
            int n = file.Read(buffer, 0, buffer.Length);

            if (n == 0) // Exactly read to end of file in previous read, so we're already done.
            {
                break;
            }
            else
            {
                yield return Convert.ToBase64String(buffer, 0, n);
            }
        }
    }
}