Why is my HttpClient class freezing with certain URLs

Question

I am currently getting to grips with new HttpClient libraries to come up with a basic class to return the html/css/etc. of requested URL. Using the example taken from here

You can see the example below:

package test;

import org.apache.http.client.ResponseHandler;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;

public class Test {

    public final static void main(String[] args) throws Exception {

        HttpClient httpclient = new DefaultHttpClient();
        try {
            HttpGet httpget = new HttpGet("http://www.cwjobs.co.uk/");
            System.out.println("executing request " + httpget.getURI());

            // Create a response handler
            ResponseHandler<String> responseHandler = new BasicResponseHandler();
            String responseBody = httpclient.execute(httpget, responseHandler);

            System.out.println(responseBody);
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
    }
}

The issue I am having is that if I was to set the URL I want to request to something like http://www.google.com then it prints out the response I need. However, when I use a URL such as www.cwjobs.co.uk (used merely as an example) it freezes on the execute method.

I am fairly new to Java and I understand the basics of HTTP so I would love to know if: - Although I have used a basic example I have done it wrong by either missing something that needs to be added to access that specific URL - It is not possible to get the response I want from that specific URL due to environment setup on the Server side. - Any extra literature or links you can recommend for me to look over outside of the Apache.org site

Thanks, Mark

Answer 1

You have to set the timeout in DefaultHttpClient for the connection. See: http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/params/CoreConnectionPNames.html?is-external=true#SO_TIMEOUT

Answer 2

This code is now deprecated (get HttpParams, etc). A better way is:

RequestConfig defaultRequestConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.BEST_MATCH).setExpectContinueEnabled(true).setStaleConnectionCheckEnabled(true).setTargetPreferredAuthSchemes(Arrays.asList(AuthSchemes.NTLM, AuthSchemes.DIGEST)).setProxyPreferredAuthSchemes(Arrays.asList(AuthSchemes.BASIC)).build();

HttpGet httpGet = new HttpGet(url);    
RequestConfig requestConfig = RequestConfig.copy(defaultRequestConfig).setSocketTimeout(5000).setConnectTimeout(5000).setConnectionRequestTimeout(5000).build();
httpGet.setConfig(requestConfig);