urlfetch.fetch() from Google App Engine not showing up in Fiddler2

Question

I'm testing a Google App Engine app on my Windows machine, running locally on localhost:8084. Fiddler2 shows all my activity when I navigate around my app, but when requesting an external url with urlfetch.fetch() it doesn't show up in Fiddler at all, even when using an http, not an https address, and with a successful status code 200 in the response.

What do I need to do to get the urlfetch.fetch() request from Google App Engine to show up in Fiddler2?

Answer 1

My understanding is that Fiddler2 runs as an HTTP proxy; browser requests go through this proxy instead of directly to the internet resource. This allows Fiddler2 to capture information about the request and the response.

According to the Fiddler2 docs, "You can configure any application which accepts a HTTP Proxy to run through Fiddler so you can debug its traffic". So I think you would need to change the URLFetch API call to use a proxy, supplying the Fiddler URL and port. However, the URLFetch documentation doesn't specify exactly how to do this. You might be able to use urllib2 as specified in this question.

Answer 2

Irussell is generally right, but I'd like to make the answer more specific.

As proxies aren’t supported within Google AppEngine production environment, it’s not directly supported by development engine either. It seems that the only way to overcome this limitation is to modify the code of AppEngine development server.

You'll have to modify the urlfetch_stub.py file, by adding the following lines:

connection = connection_class('127.0.0.1', 8888)

and

full_path = protocol + "://" + host + full_path

You may find the detailed explanation in my blog post Use Fiddler to debug urlfetch requests in Google AppEngine