Convert tweepy JSON object to dict

Question

I'd like to convert the result of a Tweepy api.trends_location(woeid) call to a dict (or a dict of dicts), so I can work with the values (really, I want to end up with a dict of the 'name' values). The Tweepy documentation says that the result is 'a JSON object' (see here), but when I retrieve it, type(retrieved) evaluates to list. Sure enough, retrieved has a len of 1, and retrieved[0] gives me a single item:
[{'trends': [{'url': 'http://search.twitter.com/search?q=%23questionsidontlike', 'query': '%23questionsidontlike', 'events': None, 'promoted_content': None, 'name': '#questionsidontlike'}, ], (more of the same), 'created_at': '2011-01-31T22:39:16Z', 'as_of': '2011-01-31T22:47:47Z', 'locations': [{'woeid': 23424977, 'name': 'United States'}]}].

I can call json.dumps, which will give a nicely-formatted representation, but that's not much use to me, and json.loads gives me: __init__() got an unexpected keyword argument 'sort_keys'

How should I proceed?

Link to full code: https://gist.github.com/805129

Answer 1

Okay this should do it! It is even tested (thanks for posting the additional info).

>>> names = [trend["name"] for trend in retrieved[0]["trends"]]
>>> names
['#wishuwould', '#questionsidontlike', '#februarywish', 'Purp & Patron', 'Egyptians', 'Kool Herc', 'American Pie', 'Judge Vinson', 'Eureka Nutt', 'Eddie House']

I think most of the confusion came from the documentation referring to the output as a JSON object, which is different than a JSON string which would need to be converted using the json module.

How this works: retrieved is a list that contains a single item, which is the dictionary that contains the trends key, so retrieved[0]["trends"] is the list of trend dictionaries, where each trend dictionary contains the name key you are interested in.

Answer 2

Would something like this work for you?

def searchKeys(struct, keys, result = None, recursive = True):
        if result is None:
                result = []

        if isinstance(struct, dict):
                for k in keys:
                        if struct.has_key(k):
                                result.append(struct[k])

                if recursive:
                        for i in struct.values():
                                searchKeys(struct = i, keys = keys, result = result, recursive = recursive)
        elif isinstance(struct, list):
                if recursive:
                        for i in struct:
                                searchKeys(struct = i, keys = keys, result = result, recursive = recursive)

        return result

Usage example:

>>> searchKeys(struct = a, keys = ['name'])
['United States', '#questionsidontlike']

It recursively walks down a dict/list hierarchy searching for a set of dict keys and stores the corresponding values to a list.

Answer 3

To convert a Tweepy 'Status' object to a Python dictionary (JSON), access the private member "_json" on the object.

tweets = tweepy_api.user_timeline(screen_name='seanharr11')
json_tweets = map(lambda t: t._json, tweets)

Answer 4

>>> import simplejson
>>> a = {"response":[{"message":"ok"},{"message":"fail"}]}
>>> json = simplejson.dumps(a)
>>> simplejson.loads(json)
{'response': [{'message': 'ok'}, {'message': 'fail'}]}

http://docs.python.org/library/json.html