SharpZipLib to compress a string

Question

I need to compress a string to reduce the size of a web service response. I see the unit tests in the SharpZipLib samples, but not an example of exactly what I need.

In the following code, the constructor for ZipOutputStream returns the exception: "No open entry"

        byte[] buffer = Encoding.UTF8.GetBytes(SomeLargeString);
        Debug.WriteLine(string.Format("Original byes of string: {0}", buffer.Length));

        MemoryStream ms = new MemoryStream();
        using (ZipOutputStream zipStream = new ZipOutputStream(ms))
        {
            zipStream.Write(buffer, 0, buffer.Length);
            Debug.WriteLine(string.Format("Compressed byes: {0}", ms.Length));
        }

        ms.Position = 0;
        MemoryStream outStream = new MemoryStream();

        byte[] compressed = new byte[ms.Length];
        ms.Read(compressed, 0, compressed.Length);

        byte[] gzBuffer = new byte[compressed.Length + 4];
        System.Buffer.BlockCopy(compressed, 0, gzBuffer, 4, compressed.Length);
        System.Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, gzBuffer, 0, 4);
        string compressedString = Convert.ToBase64String (gzBuffer);

Where did I get off track? Am I making this more complex than it should be?

Answer 1

Are you sure that the data will be that much smaller after you convert it to Base 64? That will bloat the binary data (zip) significantly. Can't you solve the issue at the transport level using HTTP compression?

Here's a post with full source that shows how to do the round-trip zip/unzip.

http://paultechguy.blogspot.com/2008/09/zip-xml-in-memory-for-web-service.html

Answer 2

For web service communication compression data from Silverlight I use this snippet:

private byte[] zipText(string text)
{
    if (text == null)
        return null;

    using(Stream memOutput = new MemoryStream())
    {
        using (GZipOutputStream zipOut = new GZipOutputStream(memOutput))
        {
            using (StreamWriter writer = new StreamWriter(zipOut))
            {
                writer.Write(text);

                writer.Flush();
                zipOut.Finish();

                byte[] bytes = new byte[memOutput.Length];
                memOutput.Seek(0, SeekOrigin.Begin);
                memOutput.Read(bytes, 0, bytes.Length);

                return bytes;
            }
        }
    }
}

private string unzipText(byte[] bytes)
{
    if (bytes == null)
        return null;

    using(Stream memInput = new MemoryStream(bytes))
    using(GZipInputStream zipInput = new GZipInputStream(memInput))
    using(StreamReader reader = new StreamReader(zipInput))
    {
        string text = reader.ReadToEnd();

        return text;
    }
}

1. I used GZip instead of Zip compression
2. Expected that text will be read/write from the similar environment, so I didn't do any extra encoding/decoding.

My case was the compression of the json data. From my observation in some cases the text data of about 95Kb was compressed to 1.5Kb. So even the data will be serialized into base 64, anyway are good traffic saving.

Posted my answer that may be for somebody some time saving.

Answer 3

A few issues with your code :

1. Always flush data when you work with streams.

2. To read the data from a MemoryStream, just use :

   byte[] data = ms.ToArray();

3. Zip files are containers that may contains multiple entries (files), comments... you may need to call the PutNextEntry() to add a new entry before starting to write data to it.

4. If you only need to compress a single data stream (which is your case), your best choice would be to simply us the deflate (or gzip) compression which is meant to compress a single data stream (actually zip format uses gzip internally to compress its entries...) .Net offers 2 very handy classes for data compression : GZipStream and DeflateStream. A good sample can be found here

Answer 4

You need to call PutNextEntry to add the header before writing the data.

Answer copied from: http://community.sharpdevelop.net/forums/p/5910/16947.aspx

Answer 5

The most simplified answer I have found is to deal with bytes when unzipping/zipping data and use a set size buffer to copy the data to a Stream object that can be used however you like:

    /// <summary>
    /// Unzips (inflates) zipped data.
    /// </summary>
    /// <param name="zippedData">The zipped data.</param>
    /// <returns>The inflated data.</returns>
    public Byte[] GUnzip(Byte[] zippedData)
    {
        using (MemoryStream unzippedData = new MemoryStream())
        {
            using (GZipInputStream zippedDataStream = new GZipInputStream(new MemoryStream(zippedData)))
            {
                CopyStream(zippedDataStream, unzippedData);
            }

            return unzippedData.ToArray();
        }
    }

    /// <summary>
    /// zips data.
    /// </summary>
    /// <param name="unzippedData">The unzipped data.</param>
    /// <returns>The zipped data.</returns>
    public Byte[] GZip(Byte[] unzippedData)
    {
        using (MemoryStream zippedData = new MemoryStream())
        {
            using (GZipOutputStream unzippedDataStream = new GZipOutputStream(new MemoryStream(unzippedData)))
            {
                CopyStream(unzippedDataStream, zippedData);
            }

            return zippedData.ToArray();
        }
    }

    /// <summary>
    /// Accepts an inStream, writes it to a buffer and goes out the outStream
    /// </summary>
    /// <param name="inStream">The input Stream</param>
    /// <param name="outStream">The output Stream</param>
    private static void CopyStream(Stream inStream, Stream outStream)
    {
        int nRead = 0;
        // Using a 2k buffer
        Byte[] theBuffer = new Byte[2048];

        while ((nRead = inStream.Read(theBuffer, 0, theBuffer.Length)) > 0)
        {
            outStream.Write(theBuffer, 0, nRead);
        }
    }