Using Google AppEngine Urlfetch instead of urllib2

Question

What is the difference between Google's urlfetch over the python lib urllib2 ?

When I came upon Google's urlfetch I thought maybe there were security reasons. Perhaps Google is safer in terms of malicous urls or something?

Is there any reason why I should choose Google's urlfetch over urllib2?

Answer 1

I don't work for Google, so this is just a guess from various GAE posts I've read. App Engine instances don't face the internet directly, but are buried behind layers of Google infrastructure. When a browser makes an HTTP request, it doesn't go straight to your instance, but rather it hits a Google edge server which eventually routes the request to a GAE instance.

Likewise when making an HTTP request out, your instance doesn't just open a socket (which urllib2 will normally do), but rather it sends the HTTP request to some other Google edge server which goes makes that HTTP request. Using urllib2 on GAE will use a GAE specific version which runs on top of urlfetch.

Answer 2

Note that in GAE urllib, urllib2 and httplib are just wrappers around UrlFetch (See Fetching urls in Python).

One difference of the urlfetch module is that provides you with an interface for making Asynchronous requests.

Answer 3

There is no problem to use standard libraries in App Engine. Url Fetch Api is just a service to make HTTP request more "easily" than urlib2. It is more understable for a novice in Python and you can easily use a non blocking request for example.

I suggest you to read some complementary information here: https://developers.google.com/appengine/docs/python/urlfetch/overview

If google found some security problem on a Python standard libraries. I guess It will send a fix ;)

Answer 4

The difference is : urlfetch only has a functional interface and urllib and httplib have a OO interface. An OO interface can be very usefull. I have seen a good example in the oauth2 client lib, where the request instance is passed to the client lib to check if the token is valid and authorized.