Why doesn't apache java http library handle sites with Content-Encoding: none?

Question

I'm trying to send a GET via a proxy and some sites have the header: Content-Encoding: none, which causes Apache to throw an exception. I'm wondering if this is the intended behavior, and whether I should treat this as a bug or not:

Caused by: org.apache.http.HttpException: Unsupported Content-Coding: none
at org.apache.http.client.protocol.ResponseContentEncoding.process(ResponseContentEncoding.java:98)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:139)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:199)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
... 10 more

My code:

public CloseableHttpResponse getViaProxy(String url, String ip, int port, String username,
                                         String password) {
    CloseableHttpClient httpClient;
    if (username == null) {
        httpClient = HttpClients.custom()
                .setSSLSocketFactory(getCustomSslConnectionSocketFactory())
                .build();
    } else {
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(
                new AuthScope(ip, port),
                new UsernamePasswordCredentials(username, password));
        httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credsProvider)
                .setSSLSocketFactory(getCustomSslConnectionSocketFactory())
                .build();
    }

    RequestConfig config = RequestConfig.custom()
            .setProxy(new HttpHost(ip, port))
            .build();
    HttpGet httpGet = new HttpGet(url);
    httpGet.setConfig(config);

    return httpClient.execute(httpGet);
}

The error I'm referring to is from here. It seems like this method only supports headers with Content-Encoding: gzip, deflate, or identity.

http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/client/protocol/ResponseContentEncoding.html

Answer 1

HTTP protocol specification defines gzip, compress, deflate and identity as valid content coding schemes. One should be using identity instead to signal that no content transformation is expected.

Answer 2

If you only want to process the request with the Content-Encoding is none, you can use the custom method of HttpClent like so:

CloseableHttpClient httpClient = HttpClients.custom().disableContentCompression().build();

this code disable the two interceptors RequestAcceptEncoding and ResponseContentEncoding.

Answer 3

There is a possibility to fix that issue on client-side with interceptors: http://adamscheller.com/java/httpexception-unsupported-content-coding-none-solution/