Get Size of HTTP Response in Java

Question

I would like to know how much data was sent in response to a certain http-request. What I currently do is this:

   HttpURLConnection con = (HttpURLConnection) feedurl.openConnection();

//check the response for the content-size int feedsize = con.getContentLength();

The problem is, that content-legnth is not always set. E.g. when the server uses transfer-encoding=chunked I get back a value of -1.

I do not need this to display progress information. I just need to know the size of the data that was sent to me after it has been done.

Background: I need this information because I would like to compare it to the size of a response, that was sent using gzip encoding.

Answer 1

I'd use a commons-io CountingInputStream, which would do the job for you. A full but trivial example:

public long countContent(URL feedurl) {
  CountingInputStream counter = null;
  try {
     HttpURLConnection con = (HttpURLConnection) feedurl.openConnection();
     counter = new CountingInputStream(con.getInputStream());
     String output = IOUtils.toString(counter);
     return counter.getByteCount();
  } catch (IOException ex) {
     throw new RuntimeException(ex);
  } finally {
     IOUtils.closeQuietly(counter);
  }
}

Answer 2

You can extend FilterInputStream, overriding the read(), read(byte[],int,int), and skip methods so that after calling the super form, they update a counter with the number of bytes read.

Then wrap the input stream returned by URLConnection with one of these, and use the wrapper in place of the original stream. When you're done, you can query wrapper's its counter.

Other ("manual") approaches would be use a tool like YSlow to gather statistics in a browser, or Wireshark to examine the traffic on the network.