Proxy connection with Python

Question

I have been attempting to connect to URLs from python. I have tried: urllib2, urlib3, and requests. It is the same issue that i run up against in all cases. Once I get the answer I imagine all three of them would work fine.

The issue is connecting via proxy. I have entered our proxy information but am not getting any joy. I am getting 407 codes and error messages like: HTTP Error 407: Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )

However, I can connect using another of other applications that go through the proxy, git for example. When I run git config --get htpp.proxy it returns the same values and format that I am entering in Python namely

http://username:password@proxy:8080

An example of code in requests is

import requests
proxy = {"http": "http://username:password@proxy:8080"}
url = 'http://example.org'
r = requests.get(url,  proxies=proxy)
print r.status_code

Thanks for your time

Answer 1

I have solved my issue by installing CNTLM. Once this is setup and configured I set the HTTP_PROXY etc.

Answer 2

In the requests module, proxy authentication is performed as shown:

import requests
proxies = {'http':'http://x.x.x.x', 'https':'https://x.x.x.x'}
auth = requests.auth.HTTPProxyAuth('username', 'password')
r = requests.get('http://www.example.com', proxies = proxies, auth = auth)
print r.status_code, r.reason

Answer 3

You can use the HTTP_PROXY in python in order to connect to your proxy server. You can find more detailed information on this link provided.

http://www.wkoorts.com/wkblog/2008/10/27/python-proxy-client-connections-requiring-authentication-using-urllib2-proxyhandler/

The above link is showed the example using urllib2. Actually I have used this some time back to connect to two servers simultaneously. Hope this will help you

Answer 4

import requests
proxies = {'https':'https://username:password@1.4.1.2:8080'}
r = requests.get('http://www.example.com', proxies = proxies)
print r.status_code, r.reason

A working Solution! Tested on my server