Proper handling of chuncked Http Response within Socket

Question

I have some major problem detecting the true end of a http response within Socket (I have to use sockets as it was requested). We're communicating with a webservice that sends chuncked responses. I have no problems reading a response if it returns in one piece. However when it's being split all hell breaks loose :). For example:

UserA -> RequestA -> Response[1] -> Processed
UserA -> RequestA -> Response[1] -> Processed
UserB -> RequestB -> a)Response[0,1] -> Processed[a.0]
UserB -> RequestB -> b)Response[0,1] -> Processed[a.1] <- From the previous line. And thus the true response to request B have to be processed again.

What is the preferred way to handle this kind of situation? Btw, the WS also returns the Content-Length header attribute, but honestly I have a head-ache from handling that. For that it seems like I have to read the headers fields to a ByteArrayOutputStream and check if it contains the Content-Length information. Then retrieve the actual length and wait until the is.available() reaches this value. Since the available method returns an estimation I do not trust it. So what would be the proper way?

Answer 1

The correct answer should be:

private static byte[] convert(final InputStream is) throws IOException {
    final byte[] END_SIG = new byte[]{"\r".getBytes()[0], "\n".getBytes()[0]};
    final List<Byte> streamBytes = new ArrayList<Byte>();
    int readByte;
    byte[] bytes;

    // Read HTTP header:
    while ((readByte = is.read()) != -1) {
        streamBytes.add((byte) readByte);
        if (streamBytes.size() > 4) {
            int sbsize = streamBytes.size();

            int rp = sbsize - 4;
            int np = sbsize - 2;
            int rn = sbsize - 3;
            int nn = sbsize - 1;

            if (END_SIG[0] == streamBytes.get(rp) && END_SIG[0] == streamBytes.get(np) && END_SIG[1] == streamBytes.get(rn) && END_SIG[1] == streamBytes.get(nn)) {
                break;
            }
        }
    }

    // Convert to byte[]
    bytes = new byte[streamBytes.size()];
    for (int i = 0, iMAX = bytes.length; i < iMAX; ++i) {
        bytes[i] = streamBytes.get(i);
    }

    // drop header
    streamBytes.clear();

    // Convert byte[] to String & retrieve the content-length:
    final String HEADER = new String(bytes);
    int startIndex = HEADER.indexOf("Content-Length:") + "Content-Length:".length() + 1;
    int length = 0;
    int I = startIndex;
    while (Character.isDigit(HEADER.charAt(I++))) {
        ++length;
    }
    final String CL = HEADER.substring(startIndex, startIndex + length);

    // Determine the number of bytes to read from now on:
    int ContentLength = Integer.parseInt(CL);
    while (streamBytes.size() < ContentLength) {
        byte[] buffer = new byte[256];

        int rc = is.read(buffer);
        for (int irc = 0; irc < rc; ++irc) {
            streamBytes.add(buffer[irc]);
        }
    }

    // Convert to byte[]
    bytes = new byte[streamBytes.size()];
    for (int i = 0, iMAX = bytes.length; i < iMAX; ++i) {
        bytes[i] = streamBytes.get(i);
    }

    return bytes;
}

And in one place this is the answer to a question.

Answer 2

The preferred way is to use existing code that already handles it: for example, HttpURLConnection or the Apache HTTP Client. There's no good reason for you to be reinventing the wheel.