size limit gzipinputstream

Question

GZIP has a size limitation of 4GB, got it from http://www.gzip.org/#faq10 There are some patches mentioned in the above link to be able to read a file more than 4gb.

I am using GZIPInputStream to read a .gz file.

And I am able to read sizes more than 4GB also. I know I am mixing up the input stream in java and the actual gzip executable.

But I want to know whether that could be a problem at all?

for example - will there ever be a case where I am not able to read a 6Gb .gz file in java? till now I tried and I faced no issue.

In short, is there a relation between the gzip file size limitation and the gzipinputstream in java?

Answer 1

As you say, it is important to make the distinction between gzip the tool and gzip the file format.

The former did not support files larger than 4GB until verison 1.2.4.

The latter is specified in RFC 1952 and has no inherent 4GB limitation.

Java's implementation of RFC 1952 doesn't appear to have any 4GB limitation either. I routinely read large files using GZIPInputStream (Java 1.6/1.7), and have never had problems.

Answer 2

This limitation concerns only the gzip program. Therefore if written properly, your application should not have any problem reading / writing files over 4GB.

Yet, keep in mind that people trying to read a 4GB+ file you wrote may meet some problems if they use the gzip program.