How Can I make a HTTP connection and retrieve the mobile version of a website HTML?

Question

Let's say that I need to retrieve the HTML of a website making a Httpget in my app. But, if the site has a mobile version of the content, I want this version and not the usual/desktop version (for instance, if you access http://techcrunch.com/ in your mobile you will get a diffent html that you will get if you access this website on your desktop)

I have made the following code:

HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
request.setHeader("User-Agent", System.getProperty("http.agent"));
HttpResponse response = client.execute(request);

BufferedReader br = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
    sb.append(line);
}
String html = sb.toString();

but after that, the html string contains the "desktop" version of the website and not the HTML for the mobile version. My intuition says that I only need to configure the "User-Agent" for the httpget, but it seems that I need to configre anything else. Any help?

Answer 1

I have solved the problem.

My mistake is in the line

request.setHeader("User-Agent", System.getProperty("http.agent"));

To get the correct user agent I need to create a Webview and do the following:

request.setHeader("User-Agent", webview.getSettings().getUserAgentString());

This way the HTTPGet will retrieve the mobile version of the website content.

Answer 2

I think your on the right track since most sites look at the user agent to determine if the requested page should be the full or mobile version. I think your best bet would be to set the User Agent string to a hard coded value, perhaps just to test at first ? Maby something like:

Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30