Zero Content-Length in get request gets 400 Bad Request from lighttpd (clj-http)

Question

Using Clojure 1.4 (Java 7) and the clj-http (0.6.0) library. When doing a get request the Content-Length header is automatically included and set to zero. Some servers (lighttpd for instance) don't like this and respond with Bad Request. Is it possible to remove the said header or make the library not include it by default? I couldn't find anything relevant in the docs and googling gave me only this, which doesn't really help.

Answer 1

If I try:

(client/get "http://thepiratebay.se" {:debug true})

I get:

Request: nil
{:scheme :http,
 :http-url "http://thepiratebay.se",
 :request-method :get,
 :query-string nil,
 :uri "",
 :server-name "thepiratebay.se",

 :headers {"accept-encoding" "gzip, deflate"},
 :debug true,
 :body-type nil,
 :server-port nil,
 :body nil,
 :user-info nil}
HttpRequest:
{:requestLine #<BasicRequestLine GET http://thepiratebay.se HTTP/1.1>,
 :protocolVersion #<HttpVersion HTTP/1.1>,
 :params
 #<BasicHttpParams org.apache.http.params.BasicHttpParams@5b14a306>,
 :method "GET",
 :entity nil,
 :class
 clj_http.core.proxy$org.apache.http.client.methods.HttpEntityEnclosingRequestBase$0,
 :allHeaders
 [#<BasicHeader Connection: close>,
  #<BasicHeader accept-encoding: gzip, deflate>],
 :aborted false,
 :URI #<URI http://thepiratebay.se>}   

Which yields a 400 error. I tried to reproduce it in Java, using Apache HttpClient directly:

import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.client.HttpClient;
import org.apache.http.HttpResponse;
import org.apache.http.Header;

public class Get {
  public static void main(String args[]) throws Exception {
    HttpClient httpclient = new DefaultHttpClient();
    HttpGet httpget = new HttpGet(args[0]);
    httpget.addHeader("Connection", "close");
    httpget.addHeader("accept-encoding", "gzip, deflate");
    Header[] headers = httpget.getAllHeaders();
    for (Header h : headers) {
      System.out.println(h.getName() + ", " + h.getValue());
    }
    System.out.println();
    HttpResponse response = httpclient.execute(httpget);
    System.out.println(response);
  }
}

However, this works fine. My guess is that before calling HttpClient, clj-http is doing something that forces an empty body in the response, so HttpClient sets the header Content-Length to 0. The header is not set by clj-http if you look at the source. I would file this as an issue for clj-http.

https://github.com/dakrone/clj-http/issues