Establishing a TCP connection through an authenticated HTTP proxy?

Question

I'm on a network that requires me to connect through an authenticated HTTP proxy to access anything outside the network. What I need to do is basically to make a socket (or equivalent) to connect to the Internet, but sending all the data through the proxy instead of trying to send it directly. Any ideas as to how this could be done?

Answer 1

As I didn't find any actual modules or other code that I could use, I ended up writing my own function that connects through the proxy:

def http_proxy_connect(address, proxy = None, auth = None, headers = {}):
  """
  Establish a socket connection through an HTTP proxy.

  Arguments:
    address (required)     = The address of the target
    proxy (def: None)      = The address of the proxy server
    auth (def: None)       = A tuple of the username and password used for authentication
    headers (def: {})      = A set of headers that will be sent to the proxy

  Returns:
    A 3-tuple of the format:
      (socket, status_code, headers)
    Where `socket' is the socket object, `status_code` is the HTTP status code that the server
     returned and `headers` is a dict of headers that the server returned.
  """
  import socket
  import base64

  def valid_address(addr):
    """ Verify that an IP/port tuple is valid """
    return isinstance(addr, (list, tuple)) and len(addr) == 2 and isinstance(addr[0], str) and isinstance(addr[1], (int, long))

  if not valid_address(address):
    raise ValueError('Invalid target address')

  if proxy == None:
    s = socket.socket()
    s.connect(address)
    return s, 0, {}

  if not valid_address(proxy):
    raise ValueError('Invalid proxy address')

  headers = {
    'host': address[0]
  }

  if auth != None:
    if isinstance(auth, str):
      headers['proxy-authorization'] = auth
    elif auth and isinstance(auth, (tuple, list)):
      if len(auth) == 1:
        raise ValueError('Invalid authentication specification')

      t = auth[0]
      args = auth[1:]

      if t.lower() == 'basic' and len(args) == 2:
        headers['proxy-authorization'] = 'Basic ' + base64.b64encode('%s:%s' % args)
      else:
        raise ValueError('Invalid authentication specification')
    else:
      raise ValueError('Invalid authentication specification')

  s = socket.socket()
  s.connect(proxy)
  fp = s.makefile('r+')

  fp.write('CONNECT %s:%d HTTP/1.0\r\n' % address)
  fp.write('\r\n'.join('%s: %s' % (k, v) for (k, v) in headers.items()) + '\r\n\r\n')
  fp.flush()

  statusline = fp.readline().rstrip('\r\n')

  if statusline.count(' ') < 2:
    fp.close()
    s.close()
    raise IOError('Bad response')
  version, status, statusmsg = statusline.split(' ', 2)
  if not version in ('HTTP/1.0', 'HTTP/1.1'):
    fp.close()
    s.close()
    raise IOError('Unsupported HTTP version')
  try:
    status = int(status)
  except ValueError:
    fp.close()
    s.close()
    raise IOError('Bad response')

  response_headers = {}

  while True:
    tl = ''
    l = fp.readline().rstrip('\r\n')
    if l == '':
      break
    if not ':' in l:
      continue
    k, v = l.split(':', 1)
    response_headers[k.strip().lower()] = v.strip()

  fp.close()
  return (s, status, response_headers)

Answer 2

I'm not sure if this will be of much help, but have you taken a look to pycurl? This may help you to connect to the Proxy providing a username/password authentication system (see this and this)