You don't need to encode at all. BeautifulSoup has already decoded the response bytes to Unicode values, and print()
can take care of the rest here.
However, the page is malformed, as there are no closing </option>
tags. This can confuse the standard HTML parser. Install lxml or the html5lib
package, and the page can be parsed correctly:
parsedHtml = BeautifulSoup(pageContent, 'lxml')
or
parsedHtml = BeautifulSoup(pageContent, 'html5lib')
Next, you can select the <option>
tags with one CSS selector:
possibleMatches = parsedHtml.select('select[name=languageid] option')
for match in possibleMatches:
print(match.text, "-", match.get('value'))
Demo:
>>> possibleMatches = soup.select('select[name=languageid] option')
>>> for match in possibleMatches:
... print(match.text, "-", match.get('value'))
...
Language - 0
Bahasa Indonesia - 32
Català - 34
Deutsch - 4
Eesti - 41
English - 1
Español - 2
Esperanto - 22
Filipino - 21
Français - 3
Italiano - 11
Język polski - 13
LINGUA LATINA - 35
Magyar - 14
Nederlands - 7
Norsk - 18
Português - 8
Română - 27
Suomi - 20
Svenska - 17
čeština - 31
Русский - 10
देवनागरी - 39
ภาษาไทย - 38
中文 - 5
日本語 - 6