Reading and writing objects via GZIP streams?

Question

I am new to Java. I want to learn to use GZIPstreams. I already have tried this:

ArrayList<SubImage>myObject = new ArrayList<SubImage>(); // SubImage is a Serializable class

ObjectOutputStream compressedOutput = new ObjectOutputStream(
   new BufferedOutputStream(new GZIPOutputStream(new FileOutputStream(
   new File("....")))));
compressedOutput.writeObject(myObject);

and

ObjectInputStream compressedInput = new ObjectInputStream(
   new BufferedInputStream(new GZIPInputStream(new FileInputStream(
   new File("....")))));
myObject=(ArrayList<SubImage>)compressedInput.readObject();

When the program writes myObject to a file without throwing any exception, but when it reaches the line

myObject=(ArrayList<SubImage>)compressedInput.readObject();

it throws this exception:

Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input stream

How can I solve this problem?

Answer 1

You have to flush and close your outputstream. Otherwhise, at least, the BufferedOutputStream will not write everything to the file (it does in big chucks to avoid penalizing performance).

If you call compressedOutput.flush() and compressedOutput.close() it will suffice.

You can try writing a simple string object and checking if the file is well written.

How? If you write a xxx.txt.gz file you can open it with your preferred zip app and look at the xxx.txt. If the app complains, then the content is not full written.

Extended answer to a comment: compressing even more the data

Changing serialization

You could change the standard serialization of SubImage object if it's an object of your own. Check java.io.Serializable javadoc to know how to do it. It's pretty straightforward.

Writing just what you need

Serialization has the drawback that needs to write "it's a SubImage" just before every instance you write. It's not necessary if you know what's going to be there beforehand. So you could try to serialize it more manually.

To write your list, instead of writing an object write directly the values that conform your list. You will need just a DataOutputStream (but ObjectOutputStream is a DOS so you can use it anyway).

dos.writeInt(yourList.size()); // tell how many items
for (SubImage si: yourList) {
   // write every field, in order (this should be a method called writeSubImage :)
   dos.writeInt(...);
   dos.writeInt(...);
   ...
}

// to read the thing just:
int size = dis.readInt();
for (int i=0; i<size; i++) {
   // read every field, in the same order (this should be a method called readSubImage :)
   dis.readInt(...);
   dis.readInt(...);
   ...
   // create the subimage
   // add it to the list you are recreating
}

This method is more manual but if:

1. you know what's going to be written
2. you will not need this kind of serialization for many types

it's pretty affordable and definitively more compressed than the Serializable counterpart.

Have in mind that there are alternative frameworks to serialize objects or create string messages (XStream for xml, Google Protocol Buffers for binary messages, and so on). That frameworks could work directly to binary or writing a string that could be then written.

If your app will need more on this, or just curious, maybe you should look at them.

Alternative serialization frameworks

Just looked in SO and found several questions (and answers) addressing this issue:

https://stackoverflow.com/search?q=alternative+serialization+frameworks+java

I've found that XStream is pretty easy and straightforward to use. And JSON is a format pretty readable and succint (and Javascript compatible which could be a plus :).

I should go for:

Object -> JSON -> OutputStreamWriter(UTF-8) -> GZippedOutputStream -> FileOutputStream