Python: urllib2 or Pycurl?

Question

I have extensive experience with PHP cURL but for the last few months I've been coding primarily in Java, utilizing the HttpClient library.

My new project requires me to use Python, once again putting me at the crossroads of seemingly comparable libraries: pycurl and urllib2.

Putting aside my previous experience with PHP cURL, what is the recommended library in Python? Is there a reason to use one but not the other? Which is the more popular option?

Answer 1

CURL has a lot more features as stated in its web page, so if you need, say fast concurrent connections, safe threading, etc then its for you. However, its not included in the distribution. If you foresee that your task is very simple, then use urllib2 and those HTTP modules that come with the distribution.

Answer 2

urllib2 is part of the standard library, pycurl isn't (so it requires a separate step of download/install/package etc). That alone, quite apart from any difference in intrinsic quality, is guaranteed to make urllib2 more popular (and can be a pretty good pragmatical reason to pick it -- convenience!-).

Answer 3

Nowadays there are other excellent alternatives - urllib3 and requests

Answer 4

Use urllib2. It's got very good documentation in python, while pycurl is mostly C documentation. If you hit a wall, switch to mechanize or pycurl.