How to get details of DownloadError?

Question

I do the following:

try:
    result = urlfetch.fetch(url=some_url, 
    ...
except DownloadError:
    self.response.out.write('DownloadError')
    logging.error('DownloadError')                                            
except Error:
    self.response.out.write('Error')
    logging.error('Error')

Is there any way to get some more detailed description on what happened?

Answer 1

You should use logging.exception to add the Exception to the ERROR logging message:

try:
    result = urlfetch.fetch(url=some_url, 
    ...
except DownloadError, exception:
    self.response.out.write('Oops, DownloadError: %s' % exception)
    logging.exception('DownloadError')                                         
except Error:
    self.response.out.write('Oops, Error')
    logging.exception('Error')

Answer 2

In short, no. A download error is usually a timeout in our experience - something on the back end taking too long to respond (first byte). If it is receiving data, it looks like GAE will wait and throw a Deadline exception instead after your 10 seconds is up.

Does it ever succeed? Your choices on how to deal with d/l exceptions will vary depending on the back-end.

If you're taking the simplistic route and just retrying, beware of quotas and limiters - chances are your requests are indeed reaching the other system, and just aren't coming back in time. Very easy to blow past limiters this way.

J