How do I display a web page with Python?

Question

I want to read user input in Python to get a url (e.g. http://www.google.com) and then print the web page in HTML formatting (text only) to the terminal. I tried using pexpect.spawn('elinks') but elinks doesn't seem to write to stdout. I also looked at the HTMLParser module, but I don't know how I format the resulting text into something resembling a webpage. Any advice?

Answer 1

This is no small challenge. The fact that you want to spawn elinks makes me wonder why you don't just use it instead. See what extensibility/plugin/addon options it has, or try rewriting it to suit your specific needs.

Ultimately, you'll need to write your own browser layout engine with a curses backend. If you're using python, urwid is a popular choice for curses layouts.

Answer 2

with python urllib

enter the url --> urllib --> the page --> print in console

# example in the python urllib page

import urllib

opener = urllib.FancyURLopener({})
f = opener.open("http://www.python.org/")
f.read()

# modify:

html = f.read()

# add:

print html

# to print in terminal

its similar to "curl" in unix

Answer 3

import requests
r = requests.get('http://www.google.com/')
print(r.content)