Python 3.4 Requests Body wrong format

Question

In Python 3.4 I'm using Requests and a for loop to combine the body for a multiple item API request fetching JSON content. It yields a HTTP 400 error even if the body variable is recognized as a str class with type(). However if I print and copy the content into a new variable it is successful. What kind of formatting is going on?

import requests,json
list_length = len(namelist) #namelist arranged earlier in code
payload='['
for x in range(0, list_length):
    payload += '{"name": "'+ namelist[x] + '"}'
    if x<list_length-1:
        payload += ', '
payload += ']'
url = 'http://api.turfgame.com/v4/users'
headers = {'Content-Type': 'application/json'}
req = requests.post(url, data=json.dumps(payload),headers=headers)

>>> payload
'[{"name": "sune"}, {"name": "Demon"}, {"name": "kingenin"}]'

Answer 1

You are creating a JSON string, then encode it as a JSON string. This double encoding is not what you'll want here:

>>> payload = '[{"name": "sune"}, {"name": "Demon"}, {"name": "kingenin"}]'
>>> print(json.dumps(payload))
"[{\"name\": \"sune\"}, {\"name\": \"Demon\"}, {\"name\": \"kingenin\"}]"

That's a JSON string, containing a quoted JSON list..

Build a list, and pass that to json.dumps():

payload = [{'name': name} for name in namelist]
url = 'http://api.turfgame.com/v4/users'
headers = {'Content-Type': 'application/json'}
req = requests.post(url, data=json.dumps(payload),headers=headers)

This sends a proper JSON list instead:

>>> payload
[{'name': 'sune'}, {'name': 'Demon'}, {'name': 'kingenin'}]
>>> print(json.dumps(payload))
[{"name": "sune"}, {"name": "Demon"}, {"name": "kingenin"}]

You could also send payload as you built it, without passing it to json.dumps(), but why have a dog and bark yourself at all?

If you were to use requests version 2.4.2 or up, you can have it handle the JSON encoding for you; pass in the Python object into the json keyword argument and it'll even set the correct Content-Type header:

payload = [{'name': name} for name in namelist]
url = 'http://api.turfgame.com/v4/users'
req = requests.post(url, json=payload)