Error compress/decompress spdy name/value block with zlib+dictionary

Question

I was trying to write a spdy proxy server, but have problems compress/decompress spdy name/value blocks.

I was using python 3.3 zlib library to compress/decompress with dictionary.

When receiving spdy frames from chrome 31, frames can be parsed most of the time, but some name/value blocks can not be decompressed correctly.

I have 3 test cases:

import zlib

dictionary = (
b"optionsgetheadpostputdeletetraceacceptaccept-charsetaccept-encodingaccept-"
b"languageauthorizationexpectfromhostif-modified-sinceif-matchif-none-matchi"
b"f-rangeif-unmodifiedsincemax-forwardsproxy-authorizationrangerefererteuser"
b"-agent10010120020120220320420520630030130230330430530630740040140240340440"
b"5406407408409410411412413414415416417500501502503504505accept-rangesageeta"
b"glocationproxy-authenticatepublicretry-afterservervarywarningwww-authentic"
b"ateallowcontent-basecontent-encodingcache-controlconnectiondatetrailertran"
b"sfer-encodingupgradeviawarningcontent-languagecontent-lengthcontent-locati"
b"oncontent-md5content-rangecontent-typeetagexpireslast-modifiedset-cookieMo"
b"ndayTuesdayWednesdayThursdayFridaySaturdaySundayJanFebMarAprMayJunJulAugSe"
b"pOctNovDecchunkedtext/htmlimage/pngimage/jpgimage/gifapplication/xmlapplic"
b"ation/xhtmltext/plainpublicmax-agecharset=iso-8859-1utf-8gzipdeflateHTTP/1"
b".1statusversionurl\0")


def decompress(buf):
    decompressor = zlib.decompressobj(zdict=dictionary)
    return decompressor.decompress(buf)

def compress(buf):
    compressor = zlib.compressobj(zdict=dictionary)
    return compressor.compress(buf)

if __name__ == '__main__':
    # Test 1: buf -(compress)-> cb -(decompress)->buf2, buf2 become ''
    buf = b'\x00\x01\x00\x06status\x00\x1a200 Connection established'
    print(buf)
    cb = compress(buf)
    print(cb)  # b'x\xbb\xdf\xa2Q\xb2'
    buf = decompress(cb)
    print(buf) # b''

    # Test 2: This name/value block data was sent by chrome, which decompressed correctly
    print(decompress(b'8\xea\xdf\xa2Q\xb2b`e`\x01\xe5\x12\x06\x9e4`\xc6K\x02\x06\x83^r~.\x03[.0o\xe6\xa70\xb0;\xfb\xfb\xf9\xb9:\x8700\x83\x14\x0b\x00\x04PZbrjR~~\xb6^r~\xae\x95\x89\x891@\x001p!\x12<C\x8eo~UfNN\xa2\xbe\xa9\x9e\x81\x82Fxf^J~y\xb1\x82_\x88\x82\x99\x9e\xa1\xb5B\xb8\x7f\xb8\x99\x89\xa6\x82#\xd0K\xa9\xe1\xa9I\xde\x99%\xfa\xa6\xc6\xe6z\xc6f\n\x1a\xde\x1e!\xbe>:\n9\x99\xd9\xa9\n\xee\xa9\xc9\xd9\xf9\x9a\n\xce\x19\xc0\xdc\x9b\xaaol\xa8g\xa0ghfj\xa0gf\xac\x10\x9c\x98\x96X\x94\t\xd5\xc5\xc0\x0e\xf5\x04\x03\x07\xcco\x00\x00\x00\x00\xff\xff'))
    # b'\x00\x05\x00\x04host\x00\x0cfacebook.com\x00\x06method\x00\x07CONNECT\x00\x03url\x00\x10facebook.com:443\x00\nuser-agent\x00lMozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36\x00\x07version\x00\x08HTTP/1.1'

    # Test 3: This was another name/value block data sent by chrome, which can not be decompressed
    print(decompress(b'"\xcd+\x00\x01\x94\x96\x98\x9c\x9a\x94\x9f\x9f\xad\x97\x9c\x9fkebb\x0c\x10@\x83\xca+\x00\x00\x00\x00\xff\xff'))
    # Error -3 while decompressing data: incorrect header check

I'm new to python3+zlib (use python 2.7 before this project) and spdy. I really appreciate your help.

Answer 1

You need to flush for both compression and decompression. Otherwise some or all of the data remains in the object. I.e.:

def decompress(buf):
    decompressor = zlib.decompressobj()
    result = decompressor.decompress(buf)
    return result + decompressor.flush()

def compress(buf):
    compressor = zlib.compressobj()
    result = compressor.compress(buf)
    return result + compressor.flush()