DataInputStream.readInt() returns EOFException even if there is still data

Question

I am having lots of trouble reading a custom file in java. The custom file format just consists of a so called "magic" byte array, the file format version and a gzipped json-string.

Writing the file works like a charm - reading on the other side works not as intended. The EOFException gets thrown when I am trying to read the following data length.

I checked the produced file with a HEX editor, the data gets saved correctly. Something seems to go wrong while the DataInputStream tries to read the file.

The read file code:

DataInputStream in = new DataInputStream(new FileInputStream(file));

// Check file header
byte[] b = new byte[MAGIC.length];
in.read(b);

if (!Arrays.equals(b, MAGIC)) {
    throw new IOException("Invalid file format!");
}

short v = in.readShort();
if (v != VERSION) {
    throw new IOException("Old file version!");
}

// Read data
int length = in.readInt(); // <----- Throws the EOFException
byte[] data = new byte[length];
in.read(data, 0, length);

// Decompress GZIP data
ByteArrayInputStream bytes = new ByteArrayInputStream(data);
Map<String, Object> map = mapper.readValue(new GZIPInputStream(bytes), new TypeReference<Map<String, Object>>() {}); // mapper is the the jackson OptionMapper

bytes.close();

The write file code:

DataOutputStream out = new DataOutputStream(new FileOutputStream(file));

// File Header
out.write(MAGIC); // an 8 byte array (like new byte[] {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H'}) to identify the file format
out.writeShort(VERSION); // a short (like 1)

// GZIP that stuff
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
GZIPOutputStream gzip = new GZIPOutputStream(bytes);

mapper.writeValue(gzip, map);
gzip.close();

byte[] data = bytes.toByteArray();

out.writeInt(data.length);
out.write(data);

out.close();

I really hope that somebody can help me out with my problem (I am trying to solve this thing for the whole day already)!

Regards

Answer 1

I don't think you are properly closing both the fileOutputStream and the GZIPOutputStream.

GZIPOutputStream requires that you call close() on it when you are done to finish writing out the zipped data. This will require you to hold onto a reference to the GZIPOutputStream.

Here is what I think the code should be

DataOutputStream out = new DataOutputStream(new FileOutputStream(file));

// File Header
out.write(MAGIC); // an 8 byte array (like new byte[] {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H'}) to identify the file format
out.writeShort(VERSION); // a short (like 1)

// GZIP that stuff
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
GZIPOutputStream zippedStream =new GZIPOutputStream(bytes)
mapper.writeValue(zippedStream, /* my data */); // mapper is the   Jackson ObjectMapper, my data is a Map<String, Object>


zippedStream.close();

byte[] data = bytes.toByteArray();

out.writeInt(data.length);
out.write(data);

out.close();