DataInputStream available() behaviour for reading files bigger than Integer.MAX_VALUE bytes

Question

I have a binary file in custom format than I have written using DataOutputStream. The simplified format of the data in the file is: IntCharIntCharIntChar...IntChar

I am using DataInputStream to read from this file, and available() to determine whether or not the next read will be successful.

Everything works fine for small files. However, for big files, with filesize bigger than Integer.MAX_VALUE bytes, the available() call returns strange negative values after the first read. The file I am trying to read is 4751054632 bytes (about 4.8 gig).

simplified test code:

DataInputStream reader=new DataInputStream(new BufferedInputStream(new FileInputStream("/path/file")));

System.out.println("available at start:\t" + reader.available());

while(reader.available()>0){
    int a=reader.readInt();
    System.out.println("available after readInt:\t" + reader.available());

    char b=reader.readChar();
    System.out.println("available after readChar:\t" + reader.available());

    //do something
}

output:

available at start: 2147483647 //this is equal to Integer.MAX_VALUE
available after readInt:    -2147475461
available after readChar:   -2147475463

Instead of using available() I could just execute the readInt() and readChar() commands in a try block and catch the exception when the file is finished, but I am trying to understand why this behaviour is happening. Essentially I am looking for a method that will return true if there is data available to read and false if the file is finished/ the stream has ended. I thought available()>0 would do exactly that but I guess not?

Answer 1

I am using DataInputStream to read from this file, and available() to determine whether or not the next read will be successful.

Then you're doing it wrong. See the Javadoc: "Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream.". Many implementations returns zero, and the ones that don't return zero don't guarantee to return a positive number: they can't, when the number concerned exceeds Integer.MAX_VALUE.

Also, the size of the file could change between available() and read().

You should detect end of stream by catching EOFException on the methods that throw it, or -1 or null returned by the methods that don't (i.e. read(), all overloads, and readLine() respectively).