Byte array not fully consumed

Question

I am writing an FLV parser in Java and have come up against an issue. The program successfully parses and groups together tags into packets and correctly identifies and assigns a byte array for each tag's body based upon the BodyLength flag in the header. However in my test files it successfully completes this but stops before the last 4 bytes.

The byte sequence left out in the first file is :

00 00 14 C3

And in the second:

00 00 01 46

Clearly it is an issue with the final 4 bytes of both files however I cannot spot the error in my logic. I suspect it might be:

while (in.available() != 0)

However I also doubt this is the case as the program is successfully entering the loop for the final tag however it is just stopping 4 bytes short. Any help is greatly appreciated. (I know proper exception handling is as yet not taking place)

Parser.java

    import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.lang.reflect.Array;
import java.net.URI;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.InputMismatchException;

/**
 * 
 * @author A
 * 
 *      Parser class for FLV files
 */

public class Parser {

    private static final int HEAD_SIZE = 9;
    private static final int TAG_HEAD_SIZE = 15;
    private static final byte[] FLVHEAD = { 0x46, 0x4C, 0x56 };
    private static final byte AUDIO = 0x08;
    private static final byte VIDEO = 0x09;
    private static final byte DATA = 0x12;
    private static final int TYPE_INDEX = 4;

    private File file;
    private FileInputStream in;
    private ArrayList<Packet> packets;
    private byte[] header = new byte[HEAD_SIZE];

    Parser() throws FileNotFoundException {
        throw new FileNotFoundException();
    }

    Parser(URI uri) {
        file = new File(uri);
        init();
    }

    Parser(File file) {
        this.file = file;
        init();
    }

    private void init() {
        packets = new ArrayList<Packet>();
    }

    public void parse() {
        boolean test = false;
        try {
            test = parseHeader();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        if (test) {
            System.out.println("Header Verified");
            // Add header packet to beginning of list & then null packet
            Packet p = new Packet(PTYPE.P_HEAD);
            p.setSize(header.length);
            p.setByteArr(header);
            packets.add(p);
            p = null;

            try {
                parseTags();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        } else {
            try {
                in.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            // throw FileNotFoundException because incorrect file
        }
    }

    private boolean parseHeader() throws FileNotFoundException, IOException {
        if (file == null)
            throw new FileNotFoundException();

        in = new FileInputStream(file);
        in.read(header, 0, 9);

        return Arrays.equals(FLVHEAD, Arrays.copyOf(header, FLVHEAD.length));
    }

    private void parseTags() throws IOException {
        if (file == null)
            throw new FileNotFoundException();
        byte[] tagHeader = new byte[TAG_HEAD_SIZE];
        Arrays.fill(tagHeader, (byte) 0x00);
        byte[] body;
        byte[] buf;
        PTYPE pt;

        int OFFSET = 0;
        while (in.available() != 0) {
            // Read first 5 - bytes, previous tag size + tag type
            in.read(tagHeader, 0, 5);

            if (tagHeader[TYPE_INDEX] == AUDIO) {
                pt = PTYPE.P_AUD;
            } else if (tagHeader[TYPE_INDEX] == VIDEO) {
                pt = PTYPE.P_VID;
            } else if (tagHeader[TYPE_INDEX] == DATA) {
                pt = PTYPE.P_DAT;
            } else {
                // Header should've been dealt with - if previous data types not
                // found then throw exception
                System.out.println("Unexpected header format: ");
                System.out.print(String.format("%02x\n", tagHeader[TYPE_INDEX]));
                System.out.println("Last Tag");
                packets.get(packets.size()-1).diag();
                System.out.println("Number of tags found: " + packets.size());
                throw new InputMismatchException();
            }

            OFFSET = TYPE_INDEX;

            // Read body size - 3 bytes
            in.read(tagHeader, OFFSET + 1, 3);
            // Body size buffer array - padding for 1 0x00 bytes
            buf = new byte[4];
            Arrays.fill(buf, (byte) 0x00);
            // Fill size bytes
            buf[1] = tagHeader[++OFFSET];
            buf[2] = tagHeader[++OFFSET];
            buf[3] = tagHeader[++OFFSET];
            // Calculate body size
            int bSize = ByteBuffer.wrap(buf).order(ByteOrder.BIG_ENDIAN)
                    .getInt();

            // Initialise Array
            body = new byte[bSize];

            // Timestamp
            in.read(tagHeader, ++OFFSET, 3);
            Arrays.fill(buf, (byte) 0x00);
            // Fill size bytes
            buf[1] = tagHeader[OFFSET++];
            buf[2] = tagHeader[OFFSET++];
            buf[3] = tagHeader[OFFSET++];
            int milliseconds = ByteBuffer.wrap(buf).order(ByteOrder.BIG_ENDIAN)
                    .getInt();
            // Read padding
            in.read(tagHeader, OFFSET, 4);
            // Read body
            in.read(body, 0, bSize);

            // Diagnostics
            //printBytes(body);

            Packet p = new Packet(pt);
            p.setSize(tagHeader.length + body.length);
            p.setByteArr(concat(tagHeader, body));
            p.setMilli(milliseconds);
            packets.add(p);
            p = null;

            // Zero out for next iteration
            body = null;
            Arrays.fill(buf, (byte)0x00);
            Arrays.fill(tagHeader, (byte)0x00);
            milliseconds = 0;
            bSize = 0;
            OFFSET = 0;
        }

        in.close();
    }

    private byte[] concat(byte[] tagHeader, byte[] body) {
        int aLen = tagHeader.length;
        int bLen = body.length;

        byte[] C = (byte[]) Array.newInstance(tagHeader.getClass()
                .getComponentType(), aLen + bLen);
        System.arraycopy(tagHeader, 0, C, 0, aLen);
        System.arraycopy(body, 0, C, aLen, bLen);
        return C;
    }

    private void printBytes(byte[] b) {
        System.out.println("\n--------------------");
        for (int i = 0; i < b.length; i++) {
            System.out.print(String.format("%02x ", b[i]));
            if (((i % 8) == 0 ) && i != 0)
                System.out.println();
        }
    }

}

Packet.java

public class Packet {

    private PTYPE type = null;
    byte[] buf;
    int milliseconds;

    Packet(PTYPE t) {
        this.setType(t);
    }

    public void setSize(int s) {
        buf = new byte[s];
    }

    public PTYPE getType() {
        return type;
    }

    public void setType(PTYPE type) {
        if (this.type == null)
            this.type = type;
    }

    public void setByteArr(byte[] b) {
        this.buf = b;
    }

    public void setMilli(int milliseconds) {
        this.milliseconds = milliseconds;
    }

    public void diag(){
        System.out.println("|-- Tag Type: " + type);
        System.out.println("|-- Milliseconds: " + milliseconds);
        System.out.println("|-- Size: " + buf.length);
        System.out.println("|-- Bytes: ");
        for(int i = 0; i < buf.length; i++){
            System.out.print(String.format("%02x ", buf[i]));
            if (((i % 8) == 0 ) && i != 0)
                System.out.println();
        }
        System.out.println();
    }
}

jFLV.java

import java.net.URISyntaxException;

public class jFLV {

    /**
     * @param args
     */
    public static void main(String[] args) {
        // TODO Auto-generated method stub
        Parser p = null;
        try {
            p = new Parser(jFLV.class.getResource("sample.flv").toURI());
        } catch (URISyntaxException e1) {
            // TODO Auto-generated catch block
            e1.printStackTrace();
        }
        p.parse();

    }

}

PTYPE.java

public enum PTYPE {
    P_HEAD,P_VID,P_AUD,P_DAT
};

Answer 1

Both your use of available() and your call to read are broken. Admittedly I would have somewhat expected this to be okay for a FileInputStream (until you reach the end of the stream, at which point ignoring the return value for read could still be disastrous) but I personally assume that streams can always return partial data.

available() only tells you whether there's any data available right now. It's very rarely useful - just ignore it. If you want to read until the end of the stream, you should usually keep calling read until it returns -1. It's slightly tricky to combine that with "I'm trying to read the next block", admittedly. (It would be nice if InputStream had a peek() method, but it doesn't. You can wrap it in a BufferedInputStream and use mark/reset to test that at the start of each loop... ugly, but it should work.)

Next, you're ignoring the result of InputStream.read (in multiple places). You should always use the result of this, rather than assuming it has read the amount of data you've asked for. You might want a couple of helper methods, e.g.

static byte[] readExactly(InputStream input, int size) throws IOException {
    byte[] data = new byte[size];
    readExactly(input, data);
    return data;
}

static void readExactly(InputStream input, byte[] data) throws IOException {
    int index = 0;
    while (index < data.length) {
        int bytesRead = input.read(data, index, data.length - index);
        if (bytesRead < 0) {
            throw new EOFException("Expected more data");
        }
    }
}

Answer 2

You should use one of the read methods instead of available, as available() "Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream."

It is not designed to check how long you can read.