Redirect with Curl/HttpClient: header “Location” missing
Question
when I get the following url with curl
curl -D headers.http "http://www.springerlink.com/index/10.1007/s00453-007-9157-8"
the file headers.http contains a "Location" header:
HTTP/1.1 302 Found
Date: Tue, 27 Oct 2009 17:00:20 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: http://www.springerlink.com/link.asp?id=c104731297q64224
Set-Cookie: CookiesSupported=True; expires=Wed, 27-Oct-2010 17:00:20 GMT; path=/
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 173
but when I used the apache httpclient library this "Location:" header was missing (?).
int status = httpClient.executeMethod(method);
if(status!=HttpStatus.SC_OK &&
status!=HttpStatus.SC_MOVED_TEMPORARILY &&
status!=HttpStatus.SC_MOVED_PERMANENTLY
)
{
throw new IOException("connection failure for "+url+" status:"+status);
}
Header header=method.getResponseHeader("Location");
if(header==null )
{
for(Header h:method.getResponseHeaders())
{
LOG.info(h.toString());
}
throw new IOException(
"Expected a redirect for "+url
);
}
I've listed the headers below:
INFO: Date: Tue, 27 Oct 2009 17:05:13 GMT
INFO: Server: Microsoft-IIS/6.0
INFO: X-Powered-By: ASP.NET
INFO: X-AspNet-Version: 2.0.50727
INFO: Set-Cookie: ASP.NET_SessionId=js1o5wqnuhuh24islnvkyr45; path=/; HttpOnly
INFO: Cache-Control: private
INFO: Content-Type: text/html; charset=utf-8
INFO: Content-Length: 17245
uhh ???
Solution
What's going on is that with curl
, you are getting a 302 which is actually a redirect, to the URL in the location header.
With the Apache httpclient it is doing the redirect for you, and returning the headers from the request to the redirected-to location.
To demonstrate this try
curl -D headers.http "http://www.springerlink.com/link.asp?id=c104731297q64224"
and compare the response.
edit: There are actually about 4 redirects in there if you follow each location header through with curl.
OTHER TIPS
http://www.springerlink.com/index/10.1007/s00453-007-9157-8 is actually a redirect. Since the -D
option means "headers only", the first one is not redirecting to the specified Location: ...
, while the second one is. Take a look at the Content-Length, it's much different.
What happens when you leave out the -D
?
Add this
method.setFollowRedirects(false);
Before you execute the method.
HttpClient follows the redirect automatically by default but Curl doesn't.