python requests

Question

I'm writing tests for RESTful API POST-verb which sends multi-part form data to the server. I'd like to json-encode the data. What would be the correct way to do that? Below are 3 tests, of which the first 2 pass and the third one (the scenario I need) fails. Any help would be appreciated.

import requests
import json

print "test 1, files+data/nojson"
requests.post('http://localhost:8080', files={'spot[photo]': open('test.jpg', 'rb')}, data={'spot': 'spot_description'})

print "test 2, only data/json"
requests.post('http://localhost:8080',data=json.dumps({'spot': 'spot_description'}))

print "test 3, only files+data/json"
requests.post('http://localhost:8080', files={'spot[photo]': open('test.jpg',
'rb')}, data=json.dumps({'spot': 'spot_description'}))

The code outputs:

$ /cygdrive/c/Python27/python.exe -B test.py
test 1, files+data/nojson
test 2, only data/json
test 3, only files+data/json
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    'rb')}, data=json.dumps({'spot': 'spot_description'}))
  File "C:\Python27\lib\site-packages\requests\api.py", line 98, in post
    return request('post', url, data=data, **kwargs)
  File "C:\Python27\lib\site-packages\requests\safe_mode.py", line 39, in wrapped
    return function(method, url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 51, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 241, in request
    r.send(prefetch=prefetch)
  File "C:\Python27\lib\site-packages\requests\models.py", line 532, in send
    (body, content_type) = self._encode_files(self.files)
  File "C:\Python27\lib\site-packages\requests\models.py", line 358, in _encode_files
    fields = to_key_val_list(self.data)
  File "C:\Python27\lib\site-packages\requests\utils.py", line 157, in to_key_val_list
    raise ValueError('cannot encode objects that are not 2-tuples')
ValueError: cannot encode objects that are not 2-tuples

Answer 1

The error is because your data parameter is a string.

in models.py::send():

    # Multi-part file uploads.
    if self.files:
        (body, content_type) = self._encode_files(self.files)

in models.py::_encode_files():

    fields = to_key_val_list(self.data)
    files = to_key_val_list(files)

In utils.py::to_key_val_list():

if isinstance(value, (str, bytes, bool, int)):
    raise ValueError('cannot encode objects that are not 2-tuples')

This is getting hit on the call with self.data. You're passing in a string representation of a dictionary, but it's expecting a dictionary itself, like so:

requests.post('http://localhost:8080',
              files={'spot[photo]': open('test.jpg', 'rb')},
              data={'spot': 'spot_description'})

So, if anything is assigned to the files param, then the data param cannot be of type str, bytes, bool, or int. You can follow along in the source code: https://github.com/kennethreitz/requests/blob/master/requests/models.py#L531

Answer 2

POSTing JSON data can be worked around by embedding it in a form-encoded field like this:

post(url, data={'data': json.dumps(actual_data)}, files={'myfile': open('foo.data')})

also, I think you can just put everything in files like this:

post(url, files={'data': json.dumps(actual_data), 'myfile': open('foo.dat')})

...which should be equivalent to the first snippet.