Using streams to decrypt and unzip to limit memory usage?

Question

I have a very large zip file, 2.5gb, which is encrypted. I can't decrypt the entire file into memory and unzip there for production. So I'm trying to use streams to limit the amount of memory used.

I've hooked up the following to do it (error handling and stream closing left out for clarity):

SecretKeySpec keySpec = new SecretKeySpec(myKey "AES");
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");

FileInputStream fis = new FileInputStream(new File(pathToEncryptedFile));
CipherInputStream cis = new CipherInputStream(fis, cipher);

ZipInputStream zis = new ZipInputStream(new BufferedInputStream(cis));
ZipEntry ze = null;
while ((ze = zis.getNextEntry()) != null) {
    String filename = ze.getName();
    System.out.println("Found zip entry: " + filename);
}

This works for about 50% of my files, even though they're all zipped and encrypted the same way. The exception I'll get in the while() loop for the unzipping part:

java.util.zip.ZipException: unknown format (EXTSIG=f23f1090)
  at java.util.zip.ZipInputStream.readAndVerifyDataDescriptor(ZipInputStream.java:196)
  ...

If I decrypt the entire file to a byte buffer and write it to disk, then use ZipInputStream on the file, it works for all my test files.

It seems like the extra padding at the end of the encrypted file is causing some problems when trying to use streams, but I thought the "PKCS5Padding" specification would take care of that.

Thanks

Answer 1

Use ZipInputStream on the decrypted file without reading it into memory. If that fails, your file cannot be read anyways, and needs to be recreated (could be that it's slightly non-standard). If it succeeds, write out the results of the decryption stream (before passing it to ZipInputStream) and check for binary differences.