How to reduce the memory usage when encoding base64 string in c#?

Question

I am using the following code to retrieve a bitmap from a file located on my computer system:

Bitmap shellThumb = sf.Thumbnail.ExtraLargeBitmap;

where sf is a ShellFile.

I then use the following to convert the bitmap to base64:

public static string ToBase64String(this Bitmap bmp, ImageFormat imageFormat)
    {
        string base64String = string.Empty;


        MemoryStream memoryStream = new MemoryStream();
        bmp.Save(memoryStream, imageFormat);

        memoryStream.Position = 0;
        byte[] byteBuffer = memoryStream.ToArray();

        memoryStream.Close();

        base64String = Convert.ToBase64String(byteBuffer);
        byteBuffer = null;

        return base64String;
    }

I then call this method using the following:

string base64ImageAndTag = shellThumb.ToBase64String(ImageFormat.Png);

I then use JSON.Net to write this string as well as other properties of the file to a JSON file to be used in another project - there is no alternative to JSON, it has to be used.

string json = JsonConvert.SerializeObject(files.ToArray(), Formatting.Indented, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });

where files is a List.

I have to read in a thumbnail for each file in a specified folder.

From reading in the files, I get an OutOfMemory Exception and the conversion is the cause of this while writing to the JSON file.

Is there a way to better encode the bitmap so that the exception is avoided? I have searched for solutions for this problem, but am not sure how/where to apply any possible solutions to make this work.

Many thanks!!

Answer 1

You would want to compress it. If you want to preserve the bitmap format you can compress image with gzip (or any other better suited compressor). This will also be lossless compression, you don't lose any quality.

If you can convert it to jpeg, that will probably give you higher compression rate. Be careful though, this compression results in quality loss.

Answer 2

One way to reduce the memory footprint is to serialize a single object at a time and stream them into the output rather than trying to generate all of the objects (and their Base64 strings) in-memory upfront. You would use the JsonSerializer and JsonTextWriter classes in Json.Net to stream the image data one at a time. Below is a short example comparing the two ways of serializing. You would probably replace the StringWriter with a StreamWriter pointed at an output file. Inside GenerateImageFile is where you would call your ToBase64String method.

class ImageFile
{
    public string Name { get; set; }
    public string Base64Image { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        var files = new[] { "test.bmp", "test2.bmp" };

        // generate all the objects and then serialize.
        var imageFiles = files.Select(GenerateImageFile);
        var serialized = JsonConvert.SerializeObject(imageFiles.ToArray(), Formatting.Indented, new JsonSerializerSettings() { NullValueHandling = NullValueHandling.Ignore });
        Console.WriteLine(serialized);

        // generate objects one at a time.
        // use JsonSerialzer/JsonTextWriter to "stream" the objects
        var s = JsonSerializer.Create(new JsonSerializerSettings() { NullValueHandling = NullValueHandling.Ignore });
        var strWriter = new StringWriter();
        using (var writer = new JsonTextWriter(strWriter) { Formatting = Formatting.Indented })
        {
            writer.WriteStartArray();
            foreach(var file in files)
            {
                var imageFile = GenerateImageFile(file);
                s.Serialize(writer, imageFile);
            }
            writer.WriteEndArray();
        }
        Console.WriteLine(strWriter.GetStringBuilder());
    }

    private static ImageFile GenerateImageFile(string fileName)
    {
        return new ImageFile() { Name = fileName, Base64Image = fileName + fileName };
    }
}