Is there a good way to handle *fast* compression to LZMA in Java?

Question

The native Java LZMA SDK provided by the 7-Zip author works, but is not nearly fast enough for my needs. The high level implementation just doesn't seem to be able to handle the low level-work needed by the algorithm.

I've found 7-Zip-JBinding, which implements a JNI cover over the C++ dll which ships with 7-Zip. This handles decompression at a reasonable speed, but it's in beta and doesn't currently support compression.

Anyway, my question is two-part:

1. Is my assumption that a DLL wrapper is my best bet for a fast Java implementation?
2. If so, is there a pre-existing project I should be looking at or do I have to write something from the ground up?

Answer 1

It might not be an option for you, but I've had excellent success with simply executing an external 7z process using ProcessBuilder. Using LZMA2 and lowering compression level dramatically sped up compression time too. There are portability issues of course.

You might want to check out XZ for Java. XZ uses LZMA2 compression, but has a different format better suited for Unix environments. The latest 7zip should be able to extract .XZ archives.

Answer 2

Take a look at Apache Commons Compress, which has an excellent API for the LZMA2 format. The implementation used is XZ for Java so you need to add this dependency to your code as well. The Examples page has a code snippet which shows how the XZCompressorInputStream is used to decompress:

FileInputStream fin = new FileInputStream("archive.tar.xz");
BufferedInputStream in = new BufferedInputStream(fin);
FileOutputStream out = new FileOutputStream("archive.tar");
XZCompressorInputStream xzIn = new XZCompressorInputStream(in);
final byte[] buffer = new byte[buffersize];
int n = 0;
while (-1 != (n = xzIn.read(buffer))) {
    out.write(buffer, 0, n);
}
out.close();
xzIn.close();

You should be aware that LZMA is a deprecated format - Apache Commons Compress only has support to read this format and the same goes for XZ for Java. LZMA2 is wrapped in the XZ archive format, but this is definitely the way to go since it is the supported algorithm/format and is also available on other platforms. There is also a command line tool on Linux called "xz" which can be used to compress/decompress files.