appending successfully to a python list

Question

This may seem like the worlds simplest python question... But I'm going to give it a go of explaining it.

Basically I have to loop through pages of json results from a query.

the standard result is this

{'result': [{result 1}, {result 2}], 'next_page': '2'}

I need the loop to continue to loop, appending the list in the result key to a var that can be later accessed and counted the amount of results within the list. However I require it to loop only while next_page exists as after a while when there are no more pages the next_page key is dropped from the dict.

currently i have this

next_page = True
while next_page == True:
    try:
        next_page_result = get_results['next_page'] # this gets the next page
        next_url = urllib2.urlopen("http://search.twitter.com/search.json" + next_page_result)# this opens the next page
        json_loop = simplejson.load(next_url) # this puts the results into json
        new_result = result.append(json_loop['results']) # this grabs the result and "should" put it into the list
    except KeyError:
        next_page = False   
        result_count = len(new_result)

Answer 1

Alternate (cleaner) approach, making one big list:

results = []
res = { "next_page": "magic_token_to_get_first_page" }
while "next_page" in res:
    fp = urllib2.urlopen("http://search.twitter.com/search.json" + res["next_page"])
    res = simplejson.load(fp)
    fp.close()
    results.extend(res["results"])

Answer 2

new_result = result.append(json_loop['results'])

The list is appended as a side-effect of the method call. append() actually returns None, so new_result is now a reference to None.

Answer 3

You want to use

result.append(json_loop['results']) # this grabs the result and "should" put it into the list
new_result = result

if you insist on doing it that way. As Bastien said, result.append(whatever) == None

Answer 4

AFAICS, you don't need the variable new_result at all.

result_count = len(result)

will give you the answer you need.

Answer 5

you cannot append into a dict..you can append into your list inside your dict,you should do like this

result['result'].append(json_loop['results'])

if you want to check if there is no next page value in your result dict,and you want to delete the key from the dict,just do like this

if not result['next_page']:
    del result['next_page']