Skipping JSON output in a loop

Question

I'm using Python to extract data using Steam's API. I have a randomly generated list of Steam ID's. The problem is, not all of these ID's point to valid accounts. When I try to access the account for which there is no object value, the program gives me a "KeyError". The Steam API will output a ['response'] level object for any requests, so when i print using that topmost level, I will get a response for every request. However, if I go one more level in (e.g. ['response']['game_count']), the program will fail when it gets to an account which does not have any ['game_count'] value. How do I tell python to just skip these accounts?

Example output:

account with game (Removed extra returns for readability)

{
"response": {
    "game_count": 1,
    "games": [
        {
            "appid": 205790,
            "playtime_forever": 0
        }
    ]
}

account without game

{
    "response": {
    }
}

My current code:

import urllib2, random, sys, json

list = open('d:\python\SteamUserIDs.txt').read().splitlines()
SteamID = str(list)


for SteamID in list[0:10:1]:
    request = urllib2.Request('http://api.steampowered.com/IPlayerService/GetOwnedGames/v0001/?key=05475AE5A8410CE01236A8A29E1DEE8E&steamid=%s&format=json' %SteamID, headers={"User-Agent": "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36"})
    response = json.load(urllib2.urlopen(request))
    request2 = urllib2.Request('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0001/?key=05475AE5A8410CE01236A8A29E1DEE8E&steamids=%s&format=json' %SteamID, headers={"User-Agent": "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36"})
    response2 = json.load(urllib2.urlopen(request2))

Answer 1

There are a couple options.

• The first option is in or not in. An important note about this one, if you see recommendations to use has_key(), don't. It has been deprecated in Python 2.x and removed in Python 3.x. Instead, you use the in operator.

Return True if d has a key key, else False.

if 'game_count' in response:
    # Do something

Or, the not version:

if 'game_count' not in response:
    # skip/pass on this record
else:
    # Do something

• The next option, like mentioned in another answer, is the get method

Return the value for key if key is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a KeyError.

if response.get('game_count', None):
    # Do something
else:
    # skip/pass on this record

Update: You are not doing anything with your responses. That's why it looks like nothing is happening.

There are also a few things about your code that needs to be fixed:

• You are making unneeded calls to the API. You can pass up to 100 SteamIDs per call to the GetPlayerSummaries. You don't need to call it one at a time for each ID.
• Don't name your variable list
• SteamID = str(list) is pointless, as you are reusing the SteamID variable in your loop

This is a slightly modified version of your code

steam_ids = open('d:\python\SteamUserIDs.txt').read().splitlines()
steam_str = ','.join(steam_ids)    # This make a comma separated string that we will use in our API calls 

owned_games = {}

for s in steam_ids:
    request = urllib2.Request('http://api.steampowered.com/IPlayerService/GetOwnedGames/v0001/?key=<<YOURKEY>>&steamid=%s&format=json' % s,headers={"User-Agent": "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36"})
    response = json.load(urllib2.urlopen(request))['response']
    if response.get('game_count', None):
        owned_games[s] = response['game_count']
    else:
        owned_games[s] = None

# This call will only work if you have less than 100 steam ids. If you have more, you
# will need to build steam_str differently for batches of 100
request2 = urllib2.Request('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0001/?key=<<YOURKEY>>&steamids=%s&format=json' %steam_str, headers={"User-Agent": "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36"})
response2 = json.load(urllib2.urlopen(request2))

At the end of this, you will end up with a owned_games dictionary that looks like this (Profile IDs aren't accurate in this example).

{'76561197992702437': 63,
 '76561197992702438': 0,
 '76561197992702439': 0,
 '76561197995066524': 62,
 '76561197995066525': 0,
 '76561197995066526': 0,
 '76561197995066527': 0}

This dictionary is showing you how many games each profile owns. Then, in your response2 object, you can parse through each item in response2['response']['players'] to do your comparisons. Alternatively, you can rebuild the players list into a dictionary (or combine it with games_owned so that you don't need to loop through the list each time to find the appropriate profile. Either of these two exercises is beyond the scope of your initial question though.

Now, you will need to look at the data in response and response2 for your parsing.

Answer 2

Something like this?

You are getting a key error because the key game_count doesn't exist. Check for that key before doing something (such as printing)

for response in steam_id_responses:
    if response.get('game_count', None):
        print response