Question

Background: Due to the environment limitation I am stuck with python 2.4. So requests is out of the question.

I want to be able to use urllib2.HTTPBasicAuthHandler and ProxyHandler at the same time to open a url.

If I do something like:

proxy = urllib2.ProxyHandler({'http': 'http://myproxy.local'})
proxy_opener = urllib2.build_opener(proxy)

...
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
pass_handler = urllib2.HTTPBasicAuthHandler(passman)
...
 urllib2.install_opener(urllib2.build_opener([proxy_opener, pass_handler]))

The code will stuck at this line:

urllib2.urlopen(target_url)

So, What is the proper way to install two handlers?

EDIT:

My original version has a syntax error. The line

urllib2.install_opener(urllib2.build_opener(pass_handler), proxy_opener)

should be

urllib2.install_opener(urllib2.build_opener(pass_handler, proxy_opener)) # note the parenthesis

But as atupal suggests, it should be

urllib2.install_opener(urllib2.build_opener([proxy_opener, pass_handler]))
Was it helpful?

Solution

Read the docs-install_opener and docs-build_opener

urllib2.install_opener(opener)

Install an OpenerDirector instance as the default global opener.

and urllib2.build_opener([handler, ...])

Return an OpenerDirector instance, which chains the handlers in the order given. handlers can be either instances of BaseHandler, or subclasses of BaseHandler (in which case it must be possible to call the constructor without any parameters). Instances of the following classes will be in front of the handlers, unless the handlers contain them, instances of them or subclasses of them: ProxyHandler (if proxy settings are detected), UnknownHandler, HTTPHandler, HTTPDefaultErrorHandler, HTTPRedirectHandler, FTPHandler, FileHandler, HTTPErrorProcessor.

So you should first build a opener use the proxy handler and auth handler. And install it globaly if you want:

proxy_handler = urllib2.ProxyHandler({'http': 'http://myproxy.local'})

...
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
pass_handler = urllib2.HTTPBasicAuthHandler(passman)
...
urllib2.install_opener(urllib2.build_opener(proxy_handler, pass_handler))

Update: I test the following snippet, and it works as expect. Don't forget replace the proxy, url, username and password with your own:

import urllib2

proxyhandler = urllib2.ProxyHandler({'http': 'http://219.93.183.106:8080'})

url = "http://atupal.org:9001"
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, "myusername", "mypassword")
pass_handler = urllib2.HTTPBasicAuthHandler(passman)

opener = urllib2.build_opener(
    proxyhandler,
    pass_handler,
    )
urllib2.install_opener(opener)

print urllib2.urlopen(url).read()
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top