GAE - AppEngine - DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL:

Question

I have a Google AppEngine application which runs great on my local machine. The app posts a image (from a url) to my facebook wall. However, when I deploy it to Google's servers, I get an error:

DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL: 

The offending code is:

facebook_access_token = facebook_info['access_token']

facebook_post_url = 'https://graph.facebook.com/me/photos?access_token=%s&url=%s&name=%s&method=post' % (facebook_access_token, url, caption)
facebook_post_url = facebook_post_url.replace(" ", "+");
facebook_result = urlfetch.fetch(facebook_post_url)

if facebook_result.status_code == 200:
  facebook_result_object = json.loads(facebook_result.content) 
  output_ids['facebook'] = facebook_result_object['id']
else:
  output_ids['facebook'] = ''

Ans the full error trace is:

Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", line 710, in __call__
    handler.get(*groups)
  File "/base/data/home/apps/s~digibackapi/1.362663258877230387/main.py", line 512, in get
    facebook_result = urlfetch.fetch(facebook_post_url)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 266, in fetch
    return rpc.get_result()
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 604, in get_result
    return self.__get_result_hook(self)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 404, in _get_fetch_result
    'Deadline exceeded while waiting for HTTP response from URL: ' + url)
DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL: https://graph.facebook.com/me/photos?access_token=<ACCESS_TOKEN>&url=http://digiback.cc/ej7&name=Trees&method=post

Again, the code looks solid to me, and it works ok on my local machine. Could it have something t do with timeouts? When I try the facebook_post_url in a browser, it returns instantly.

Does anyone have any ideas? I am ata complete loss here.

Many thanks!

Answer 1

Try setting the deadline for urlfetch to 30seconds or more(depending on whether you're calling urlfetch from within a task handler or request handler)

More info about urlfetch:Url Fetch Docs

Answer 2

Simple answer: the default deadline for url fetching is set to 5 seconds.

How to fix:

from google.appengine.api import urlfetch

urlfetch.set_default_fetch_deadline(60)

Answer 3

An alternative to setting the deadline globally is to set it on the specific fetch call. In your case:

result = urlfetch.fetch(
    url=your_url,
    deadline=60
)

Answer 4

I am not familiar with the facebook API's. But the way the urlfetch is constructed looks a bit strange. Normaly the method (Post) is a urlfetch argument and the post payload is urlencoded. This results in:

params = urllib.urlencode([
    ('access_token', facebook_access_token),
    ('url', url),
    ('name', caption),
  ])
 response = urlfetch.fetch(url='https://graph.facebook.com/me/photos', payload=params, method=urlfetch.POST,
                      headers={'Content-Type': 'application/x-www-form-urlencoded'})

Answer 5

Possibly related: I was experiencing the same deadline exceeded when testing local urlfetch code. It turns out that while a direct GET of a resource on the dev server via a browser would work, e.g., http://localhost:8080/content/test.jpg, trying the same via urlfetch was doomed to fail every time. I can only assume that fetching from localhost, which is translated to 127.0.0.1 at runtime, is not supported by urlfetch.

It turns out that yours was truly a timeout issue, whereas my answer is to not use urlfetch in dev.