Can't inflate with C# using DeflateStream

Question

I'm using mono to inflate/deflate bytes. Here is the code:

public static byte[] Inflate(byte[] data)
    {
        using (MemoryStream inStream = new MemoryStream(data))
        {
            using (MemoryStream outStream = new MemoryStream())
            {
                using (DeflateStream decompressStream = new DeflateStream(inStream, CompressionMode.Decompress))
                {
                    decompressStream.CopyTo(outStream);
                }
                return outStream.ToArray();
            }
        }
    }

The input data is : <789c3dca b9110020 0c04b196 bc9c3f7a 73f11030 281652d1 88b04195 1e742987 2f86258f acdec63d 6dcf0184 560cde> 47bytes. The algorithm is DEFLATE.

I've successfully inflate the same data on other platform, but through the code above, it throws the following exception:

System.IO.IOException: Corrupted data ReadInternal
  at System.IO.Compression.DeflateStreamNative.CheckResult (Int32 result, System.String where) [0x00000] in <filename unknown>:0 
  at System.IO.Compression.DeflateStreamNative.ReadZStream (IntPtr buffer, Int32 length) [0x00000] in <filename unknown>:0 
  at System.IO.Compression.DeflateStream.ReadInternal (System.Byte[] array, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 
  at System.IO.Compression.DeflateStream.Read (System.Byte[] dest, Int32 dest_offset, Int32 count) [0x00000] in <filename unknown>:0 
  at System.IO.Stream.CopyTo (System.IO.Stream destination, Int32 bufferSize) [0x00000] in <filename unknown>:0 
  at System.IO.Stream.CopyTo (System.IO.Stream destination) [0x00000] in <filename unknown>:0 

Answer 1

Finally I used DotNetZip: http://dotnetzip.codeplex.com to solve the problem.

public static byte[] Inflate(byte[] data)
    {
        int outputSize = 1024;
        byte[] output = new Byte[ outputSize ];
        bool expectRfc1950Header = true;
        using ( MemoryStream ms = new MemoryStream())
        {
            ZlibCodec compressor = new ZlibCodec();
            compressor.InitializeInflate(expectRfc1950Header);

            compressor.InputBuffer = data;
            compressor.AvailableBytesIn = data.Length;
            compressor.NextIn = 0;
            compressor.OutputBuffer = output;

            foreach (var f in new FlushType[] { FlushType.None, FlushType.Finish } )
            {
                int bytesToWrite = 0;
                do
                {
                    compressor.AvailableBytesOut = outputSize;
                    compressor.NextOut = 0;
                    compressor.Inflate(f);

                    bytesToWrite = outputSize - compressor.AvailableBytesOut ;
                    if (bytesToWrite > 0)
                        ms.Write(output, 0, bytesToWrite);
                }
                while (( f == FlushType.None && (compressor.AvailableBytesIn != 0 || compressor.AvailableBytesOut == 0)) ||
                    ( f == FlushType.Finish && bytesToWrite != 0));
            }

            compressor.EndInflate();

            return ms.ToArray();
        }
    }

    public static byte[] Deflate(byte[] data)
    {
        int outputSize = 1024;
        byte[] output = new Byte[ outputSize ];
        int lengthToCompress = data.Length;

        // If you want a ZLIB stream, set this to true.  If you want
        // a bare DEFLATE stream, set this to false.
        bool wantRfc1950Header = true;

        using ( MemoryStream ms = new MemoryStream())
        {
            ZlibCodec compressor = new ZlibCodec();
            compressor.InitializeDeflate(CompressionLevel.BestCompression, wantRfc1950Header);  

            compressor.InputBuffer = data;
            compressor.AvailableBytesIn = lengthToCompress;
            compressor.NextIn = 0;
            compressor.OutputBuffer = output;

            foreach (var f in new FlushType[] { FlushType.None, FlushType.Finish } )
            {
                int bytesToWrite = 0;
                do
                {
                    compressor.AvailableBytesOut = outputSize;
                    compressor.NextOut = 0;
                    compressor.Deflate(f);

                    bytesToWrite = outputSize - compressor.AvailableBytesOut ;
                    if (bytesToWrite > 0)
                        ms.Write(output, 0, bytesToWrite);
                }
                while (( f == FlushType.None && (compressor.AvailableBytesIn != 0 || compressor.AvailableBytesOut == 0)) ||
                    ( f == FlushType.Finish && bytesToWrite != 0));
            }

            compressor.EndDeflate();

            ms.Flush();
            return ms.ToArray();
        }
    }

Answer 2

If you read my comment you will see that I encountered this problem 18 hours ago and although the answer to the problem is here in your answer it is not directly apparent. In your answer there is the variable set wantRfc1950Header = true and in your input stream the first two bytes are the RFC 1950 magic bytes 78 9c. The System.IO.Compression.DeflateStream expects a raw RFC 1951 stream that has these two bytes omitted. I imagine you should be able to use your initial example if you chop off these first two bytes before feeding it to the inflator.

On the downside it has taken me over 18 hours to find out that I need to remove two bytes of data. On the upside I am much more familiar with the internals of zlib and Huffman coding.

Answer 3

Yeah you can just +2 bytes but I'm not sure if that lets corrupted data through or works in all cases?

   // Note: Caller must Dispose/Close.
    public DataReader ReadCompressedData()
    {
        // TODO: Optimize when using MemoryStream to use GetBuffer?
        var uncompressedSize = ReadInt32();
        var compressedSize = ReadInt32();

        // Consuming 2 bytes for the 78 9C (Sometimes other like 78 DA)
        // Unity / .Net Deflate Stream expects the data to not have this header.
        // I could use the SharpZlib project to get around this or the DotNetZip.
        // https://stackoverflow.com/questions/762614/how-do-you-use-a-deflatestream-on-part-of-a-file
        // http://www.faqs.org/rfcs/rfc1950.html
        // https://stackoverflow.com/questions/20850703/cant-inflate-with-c-sharp-using-deflatestream
        //stream.Position += 2;
        byte[] magic = ReadBytes(2);
        compressedSize -= 2;
        // I wonder if I should check these?

        var compressedData = ReadBytes(compressedSize);
        if (compressedData.Length != compressedSize)
        {
            Debug.LogError("Data read from underlying stream does not match specified size.");
        }

        // Decompress the data in the stream leaving it open.
        // Note: Not sure how to stop DeflateStream gobbling up all data in the stream.
        // using (var ds = new DeflateStream(BaseStream, CompressionMode.Decompress, true))
        // {
        //
        // }

        // Note: We are trusting that the decompressed data will fit completely into uncompressedSize.
        var os = new MemoryStream(uncompressedSize);
        using (var inputStream = new MemoryStream(compressedData))
        {
            using (var ds = new DeflateStream(inputStream, CompressionMode.Decompress))
            {
                ds.CopyTo(os);
            }
        }

        // Reset the stream to the beginning for reading.
        os.Position = 0;
        return new DataReader(os);
    }