how to get the size of chunk in http response using java if transfer encoding is chunked

Question

How to know the size of the chunk of HTTP response if transfer encoding is chunked.I am unable to get the logic. Please help me.And provide me some sample java code to get the size of chunk.I read in some books that size of each chunk is specified before the chunk itself.But using which logic can I get that. Please help me using java.

Thank you.

Answer 1

Not able to give ready to use code but Apache HttpClient library supports "everything" out of the box.

This is wikipedia example of chunked response. You don't know the exact byte size of data until body part is read. Please note size of each chunk is hexadecimal value, last 0 sized chunk is end of data. It too is terminated by 2-byte CRLF delimiter.

Transfer-Encoding: chunked\r\n
Content-Type: text/plain\r\n
\r\n
4\r\n
Wiki\r\n
5;extkey=extvalue\r\n
pedia\r\n
E\r\n
.in\r\n
\r\n
chunks.\r\n
0\r\n
AfterBodyHeader: some value\r\n
AfterBodyHeader2: any value\r\n
\r\n

• read bytes until CRLF (\r\n or 0D0A in hex)
• drop everything after ; delimiter or stop at trailing CRLF, some replys may add extensions at the line of chunk size. Ignore them unless you wait for known key-value pair.
• convert hex to integer, it tells you num of data bytes in chunk
• read x num of bytes and append to ByteArrayOutputBuffer
• read trailing CRLF delimiter bytes and ignore it
• if not 0 sized read again size+chunk data
• some responses may add AfterHeaders they are like headers at the start of response
• after 0 sized chunk an empty line (\r\n) indicates end of complete http response. If there is no AfterHeaders you should see bytes 0\r\n\r\n in the end. Please note regular chunks may contain empty lines so do not terminate parsing until 0-sized indicator.

This example uses \r\n placeholders to indicate 2-byte delimiter as specs require