Parse JSON from response with python requests

Question

I am trying to parse JSON data from a response:

If use my browswer to go to: /url/gold/stuff/here

I get a response in the browser as:

window.data_gold = [{'gold':' room 123' } , {'better gold':'room 1234'} , {"best gold": "in cellar"}];

How can I extract the JSON data out of the response

window.data_gold = json 

My code:

import requests,json

url     = '/url/gold/stuff/here'
r       = requests.get(url,timeout = 30)
newJSON = r.text

The above returns all the text, .json does not work.

Answer 1

Try something like

data = json.loads(r.text[19:-1])

Edit: it doesn't like that, but this works:

import ast

data = ast.literal_eval(r.text[19:-1])

which gives

[{'gold': ' room 123'},
 {'better gold': 'room 1234'},
 {'best gold': 'in cellar'}]

Answer 2

Here's a hideous looking one-liner that will pull the json out of that particular response and put it in a dict.

d = json.loads(r.text.split("=")[1].replace("'", '"')[:-1])

It's pulling the json itself out of the javascript statement, replacing the single-quotes with double-quotes (because the json module won't accept single-quotes), and then feeding it the json module.

Edit As pointed out by Hugh Bothwell, using ast.literal_eval instead of the json module avoids the single-quote issue, so you're left with

d = ast.literal_eval(r.text.split("=")[1].strip(" ;"))

The .strip(" ;") Will strip the ';' off the end and the whitespace from the beginning.