Python urllib2.urlopen with @ symbol in URL

Question

I'm playing around in Python and and there's a URL that I'm trying to use which goes like this

https://[username@domain.com]:[password]@domain.com/blah

This is my code:

response =urllib2.urlopen("https://[username@domain.com]:[password]@domain.com/blah")
html = response.read()
print ("data="+html)

This isn't going through, it doesn't like the @ symbols and probably the : too. I tried searching, and I read something about unquote, but that's not doing anything. This is the error I get:

 raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: 'password@updates.opendns.com'

How do I get around this? The actual site is "https://updates.opendns.com/nic/update?hostname=

thank you!

Answer 1

use urlencode! Not sure if urllib2 has it, but urllib has an urlencode function. One sec and i'll get back to you.

I did a quick check, and it seems that you need to use urrlib instead of urllib2 for that...importing urllib and then using urllib.urlencode(YOUR URL) should work!

import urllib
url = urllib.urlencode(<your_url_here>)

EDIT: it's actually urlllib2.quote()!

Answer 2

URIs have a bunch of reserved characters separating distinguishable parts of the URI (/, ?, &, @ and a few others). If any of these characters appears in either username (@ does in your case) or password, they need to be percent encoded or the URI becomes invalid.

In Python 3:

>>> from urllib import parse
>>> parse.quote("p@ssword?")
'p%40ssword%3F'

In Python 2:

>>> import urllib
>>> urllib.quote("p@ssword?")
'p%40ssword%3F'

Also, don't put the username and password in square brackets, this is not valid either.