How to debug a file upload?

Question

I'm trying to upload a PDF file to a website using Hot Banana's content management system using a Python script. I've successfully logged into the site and can log out, but I can't seem to get file uploads to work.

The file upload is part of a large complicated web form that submits the form data and PDF file though a POST. Using Firefox along with the Firebug and Tamper Data extensions I took a peek at what the browser was sending in the POST and where it was going. I believe I mimicked the data the browser was sending in the code, but I'm still having trouble.

I'm importing cookielib to handle cookies, poster to encode the PDF, and urllib and urllib2 to build the request and send it to the URL.

Is it possible that registering the poster openers is clobbering the cookie processor openers? Am I doing this completely wrong?

---

Edit: What's a good way to debug the process? At the moment, I'm just dumping out the urllib2 response to a text file and examining the output to see if it matches what I get when I do a file upload manually.

Edit 2: Chris Lively suggested I post the error I'm getting. The response from urllib2 doesn't generate an exception, but just returns:

<script>
    if (parent != window) { 
        parent.document.location.reload(); 
    } else { 
        parent.document.location = 'login.cfm'; 
    }
</script>

I'll keep at it.

Answer 1

"What's a good way to debug [a web services] process?"

At the moment, I'm just dumping out the urllib2 response to a text file and examining the output to see if it matches what I get when I do a file upload manually.

Correct. That's about all there is.

HTTP is a very simple protocol -- you make a request (POST, in this case) and the server responds. Not much else involved and not much more you can do while debugging.

What else would you like? Seriously. What kind of debugger are you imagining might exist for this kind of stateless protocol?

Answer 2

A tool like WireShark will give you a more complete trace at a much lower-level than the firefox plugins.

Often this can be something as simple as not setting the content-type correctly, or failing to include content-length.

Answer 3

You might be better off instrumenting the server to see why this is failing, rather than trying to debug this on the client side.