Using cURL to imitate the itunes user agent

Question

I’m trying to access the content of the itunes app ranks without using itunes. I’d like to be able to view the source content of a link such as this: ‘https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewTop?genreId=36&id=25204&popId=30.’

I understand that the content being linked to is stored in xml, and thus not easily rendered in a browser, but even being able to access the source code would be immensely useful.

So far I have gleaned that in order to do this, the program making the request must imitate the itunes user agent, so that itunes is “tricked” into thinking that the itunes program is making the request.

I’ve tried using cURL to make the request, following the instructions found here: http://innumero.wordpress.com/2011/02/08/scraping-the-apple-app-store/

However, when I try to execute the following from command prompt, I receive the ‘curl (6) couldn't resolve host’ error:

    curl -H ‘Host: itunes.apple.com’ -H ‘Accept-Language: en-us, en;q=0.50′ -H ‘X-Apple-Store-Front: 143444,5′ -H ‘X-Apple-Tz: 3600′-U ‘iTunes/9.2.1 (Macintosh; Intel Mac OS X 10.5.8) AppleWebKit/533.16”http://itunes.apple.com/WebObjects/MZStore.woa/wa/customerReviews?s=143444&id= 632827808&displayable-kind=11′

The question is, why is cURL unable to resolve ‘Host: itunes.apple.com’?

And if there are any other creative ways to access the itunes ranks, I would love to hear them :)

Thanks!

Answer 1

Your curl command has several errors. I have modified your curl command:

curl -L -k -H "X-Apple-Store-Front: 143444,5" -H "X-Apple-Tz: 3600" -A "iTunes/9.2.1 (Macintosh; Intel Mac OS X 10.5.8) AppleWebKit/533.16" "http://itunes.apple.com/WebObjects/MZStore.woa/wa/customerReviews?s=143444&id=632827808&displayable-kind=11′"