Submit form and upload file with requests

Question

I'm struggling with submitting a particular form with python requests. Other forms on the site I want to use it work fine and I'm able to submit the login form etc. It's just the file upload I'm having problems with. Apparently submitting the form works fine because I get a message from the site saying "Please go back and select at least one image file to upload." but the file isn't posted to the site.

If I look at the request with the Firefox Plugin Tamperdata it looks like this after I submit the empty form:

-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="userfile[]"; filename=""\r
Content-Type: application/octet-stream\r
\r
\r
-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="userfile[]"; filename=""\r
Content-Type: application/octet-stream\r
\r
\r
-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="userfile[]"; filename=""\r
Content-Type: application/octet-stream\r
\r
\r
-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="userfile[]"; filename=""\r
Content-Type: application/octet-stream\r
\r
\r
-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="userfile[]"; filename=""\r
Content-Type: application/octet-stream\r
\r
\r
-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="upload_to"\r
\r
0\r
-----------------------------201075691119887339851216603987\r
Content-Disposition: form-data; name="upload_type"\r
\r
standard\r
-----------------------------201075691119887339851216603987--\r

(I replaced all the \n with line breaks to make it a little bit more readable)

My python upload code looks like this:

index = s.get("https://example.com/")
file = {'userfile[]': open('tmp/cover/cover.jpg', 'rb')}

res = s.post(url='https://example.com/upload.php',
                data=file,
                headers={'Content-Type': 'application/octet-stream'})

The HTML form looks like this:

<form action="upload.php" method="post" id="upload_form" enctype="multipart/form-data">
    <p>
        <input name="userfile[]" type="file" size="30"> <br>
        <input name="userfile[]" type="file" size="30"> <br>
        <input name="userfile[]" type="file" size="30"> <br>
        <input name="userfile[]" type="file" size="30"> <br>
        <input name="userfile[]" type="file" size="30"> <br>

        <span id="more_file_inputs"></span>
        <br>
        <span id="upoptions_hidden">
            Uploading Options: <a href="javascript:void(0);" onclick="toggle('upoptions_hidden'); toggle('upoptions_shown');">Show</a>
        </span>

        <span id="upoptions_shown" style="display: none;">
            Uploading Options: <a href="javascript:void(0);" onclick="toggle('upoptions_hidden'); toggle('upoptions_shown');">Hide</a>
            <br><br>

                                Upload to: 
                <select name="upload_to">

                    <option value="0">Root Album</option>

                </select>
                <br><br>


           Output Layout: <input type="radio" name="upload_type" value="standard" checked="checked"> <span onclick="toggle_lightbox('index.php?layoutprev=std', 'upload_layout_preview_lightbox');" title="Click to preview" class="help">Standard</span> <input type="radio" name="upload_type" value="normal-boxed"> <span onclick="toggle_lightbox('index.php?layoutprev=bx', 'upload_layout_preview_lightbox');" title="Click to preview" class="help">Boxed</span>
        </span>
        <br><br>

        <input class="button1" type="button" value="Add File" onclick="new_file_input();"> 
        <input class="button1" style="font-weight:bold;" type="button" value="Start Upload" onclick="toggle_lightbox('index.php?act=upload_in_progress', 'progress_bar_lightbox'); $('form[id=upload_form]').submit();">
    </p>
</form>

Any thoughts on what's wrong here or a better way to debug this? Thank you!

Edit: I just realized that I didn't really submit the upload_type and upload_to fields but even if I set them like:

file = {'userfile[]': open('tmp/cover/cover.jpg', 'rb'), 'upload_to': '0', 'upload_type': 'standard'}

it's still not working.

Answer 1

Use the files keyword to make your upload, not data. Don't set the content-type header either; the form is posted as multipart form data, and requests should use the correct content type for that for you:

res = s.post(
    url='https://example.com/upload.php',
    data={'upload_type': 'standard', 'upload_to': '0'},
    files=file)

and put the rest of the form fields in a mapping in data.

If you want to upload more than one file under the same name, use a sequence of files:

files = {
    'userfile[]': [
        open('tmp/cover/cover1.jpg', 'rb'),
        open('tmp/cover/cover2.jpg', 'rb'),
    ]
}

res = s.post(
    url='https://example.com/upload.php',
    data={'upload_type': 'standard', 'upload_to': '0'},
    files=files)

Answer 2

I see two problems with your code:

• You are setting the Content-Type manually. Don't. The library will take care of that. This is esp. because during the multipart form data post, the headers are like: Content-Type: multipart/form-data; boundary=<blah>.
• According the docs, you should be using a files named argument for uploading a file, not data.

Use:

res = s.post(url='https://example.com/upload.php',
            data=<dict containing form params>,
            files=file)