How to accessing xml response when site also issues an HTTP error code

Question

the following url returns the expected resonse in the browser: http://ws.audioscrobbler.com/2.0/?method=user.getinfo&user=notonfile99&api_key=8e9de6bd545880f19d2d2032c28992b4

<lfm status="failed">
<error code="6">No user with that name was found</error>
</lfm>

But I am unable to access the xml in Python via the following code as an HTTPError exception is raised: "due to an HTTP Error 400: Bad Request"

import urllib
urlopen('http://ws.audioscrobbler.com/2.0/?method=user.getinfo&user=notonfile99&api_key=8e9de6bd545880f19d2d2032c28992b4')

I see that I can work aound this via using urlretrieve rather than urlopen, but the html response gets written to disc.

Is there a way, just using the python v2.7 standard library, where I can get hold of the xml response, without having to read it from disc, and do housekeeping?

I see that this question has been asked before in a PHP context, but I don't know how to apply the answer to Python: DOMDocument load on a page returning 400 Bad Request status

Answer 1

Copying from here: http://www.voidspace.org.uk/python/articles/urllib2.shtml#httperror

The exception that is thrown contains the full body of the error page:

#!/usr/bin/python2

import urllib2

try:
    resp = urllib2.urlopen('http://ws.audioscrobbler.com/2.0/?method=user.getinfo&user=notonfile99&api_key=8e9de6bd545880f19d2d2032c28992b4')
except urllib2.HTTPError, e:
    print e.code
    print e.read()