Question

I have a large amount of data to move using two PHP scripts: one on the client side using a command line PHP script and other behind Apache. I POST the data to the server side and use php://input stream to save it on the web-server end. To prevent from reaching any memory limits, data is separated into 500kB chunks for each POST request. All this works fine.

Now, to save the bandwidth and speed it up, I want to compress the data before sending and decompress when received on the other end. I found 3 pairs of functions that can do the job, but I cannot decide which one to use:

Which pair of functions would you recommend and why?

UPDATE: I just read zlib FAQ:

The gzip format (gzencode) was designed to retain the directory information about a single file, such as the name and last modification date. The zlib format (gzcompress) on the other hand was designed for in-memory and communication channel applications, and has a much more compact header and trailer and uses a faster integrity check than gzip.

Was it helpful?

Solution

All of these can be used. There are subtle differences between the three:

  • gzencode() uses the GZIP file format, the same as the gzip command line tool. This file format has a header containing optional metadata, DEFLATE compressed data, and footer containing a CRC32 checksum and length check.
  • gzcompress() uses the ZLIB format. It has a shorter header serving only to identify the compression format, DEFLATE compressed data, and a footer containing an ADLER32 checksum.
  • gzdeflate() uses the raw DEFLATE algorithm on its own, which is the basis for both of the other formats.

All three use the same algorithm under the hood. gzencode() adds the ability to include the original file name and other environmental data (this is unused when just compressing a string). gzencode() and gzcompress() both add a checksum, so the integrity of the archive can be verified, which can be useful over unreliable transmission and storage methods. If everything is stored locally and you don't need any additional metadata then gzdeflate() would suffice. For portability I'd recommend gzencode() (GZIP format) which is probably better supported than gzcompress() (ZLIB format) among other tools.

When compressing very short strings the overhead of each method becomes relevant since for very short input the overhead can comprise a significant part of the output. The overhead for each method, measured by compressing an empty string, is:

  • gzencode('') = 20 bytes
  • gzcompress('') = 8 bytes
  • gzdeflate('') = 2 bytes

OTHER TIPS

I am no PHP expert and cannot answer the question posed, but it seems like there is a lot of guessing going on here, and fuzzy information being proffered.

DEFLATE is the name of the compression algorithm that is used by ZLIB, GZIP and others. In theory, GZIP supports alternative compression algorithms, but in practice, there are none.

There is no such thing as "the GZIP algorithm". GZIP uses the DEFLATE algorithm, and puts framing data around the compressed data. With GZIP you can add things like the filename, the time of the file, a CRC, even a comment. This metadata is optional, though, and many gzippers just omit it.

ZLIB is similar, except with a different, more limited set of metadata, and a specific 2-byte header.

This is all in IETF RFCs 1950, 1951, and 1952.

To say that "the gzip algorithm compresses better than DEFLATE" is just nonsense. There is no gzip algorithm. And the algorithm used in the GZIP format is DEFLATE.

All methods are essentially the same, the difference between them is mostly in the headers. personally I'd use gzencode, this will produce output which is equal to a commandline invocation to the gzip utility.

I had to decompress a gzip compressed file in PHP with C++. I found out that the PHP gzencode and its counterpart gzdecode use the Z_NO_FLUSH method and at the end of the block of the data to encode/decode, applies Z_FINISH. The example/tutorial in C shown on the zlib website can be used to decompress and compress gziped files from PHP, as long as the windows bit and the memory level is changed to allow gzip deflations and inflations.

Extra: It seems most people in this thread have no idea what "Compression method" meant.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top