Python reading google search result with urllib

Question

I have the following code:

def downloadData(queryString):
    with urllib.request.urlopen(queryString) as url:
        encoding = url.headers.get_content_charset()
        result = url.read().decode(encoding)
    return result

However for google, which returns https documents, I get an error:

urllib.error.HTTPError: HTTP Error 403: Forbidden

Here is the query string:

https://www.google.com/search?q=Dow+Jones+Industrial+Average+Quote+with+volume

How would I need to change it to process https? Thank you!

Answer 1

You probably need to set a user agent:

https://stackoverflow.com/questions/16627227/http-error-403-in-python-3-web-scraping