local python proxy for web browser

Question

Hello Guys! Using firefox plugins I can capture headers. I want to do it in python: I need to change the browser proxy settings to localhost:8080 (or any port), then every request made by browser should pass through python script which is actually running on my machine. This script should be able capture headers, capture links in web page and so on. I know that web application scanners do so, but how can I do it in python. Can you suggest any start place, reading? I just want to learn about it and implement one.

Answer 1

Have a look at python-proxy. Googling for "python proxy" also yields tons of results.

If you want to write one from scratch it's also not too hard. You can use BaseHTTPServer to listen for new connections, make it multithreaded via SocketServer.ThreadingMixIn and then implement do_GET and do_CONNECT (possibly also do_POST and do_HEAD). In those methods you need to extract the URL from self.path, send a HTTP request to that URL (preferably using the requests package, it's much more comfortable than urllib) and send the response back to the client.