XML formatted fileDataBodyPart not uploading via ApacheHttpClient to CherryPy

Question

Please excuse my probably inaccurate vocabulary. I am using an ApacheHttpClient, through Jersey, to send a file over to our Berkeley DBXML server, handled by CherryPy.

The transmission is sent via Chunked and not streaming encoding - because we added a ProgressBar. Files are sent using a GUI, where you specify the server address, the file, and whether or not to overwrite. Files can be XML or XLS/X.

The problem I am experiencing is that, when submitting XML AND choosing the "overwrite" option, CherryPy only recieves 'dict: {"overwrite","1"}' in the request parameters, and claims that the "data" field -- that should contain the actual XML file -- is empty.

On the java side of things, if I pull out the BodyPartsof the FormData, I see it has the file, and the overwrite command. Only the overwrite command makes it to Cherrypy.

Now, deselecting overwrite, the data does get sent, but, obviously, will not overwrite at that point.

A strange thing about this is that it ONLY applies to XML files----XLS/X files can be submitted, and resubmitted with overwriting. They are handled exactly the same in the client side -- and only treated differently during server-side parsing but it doesn't make it that far.

Functionality was working fine before we switched to Chunked encoding. Here are code excerpts:

The file is added to an instance of FormDataMultiPart, as a fileDataBodyPart. UploadWorker handles

public void add(File file, boolean overwrite, UploadWorker worker) 
 {
    guilog = worker;  // if non null, output by publishing to worker
    //start filter here, pass in bytes
    long fbytes = file.length();
    String basename = Util.stripExt(file);

    //we recreate filters so that progress is reported properly
    //necessary for folder imports
    client.removeAllFilters();
    client.addFilter(new ConnectionListenerFilter(new ListenerFactory(fbytes,worker.progress,basename)));

    // Build a multipart POST form to be submitted

    // Prepare the metadata file for attachment.
    // We do this before creating the multipart form as we
    // won't have to close it out if there's problems here
    // and we bail.


    FileDataBodyPart bodypart;
    try {
        bodypart = fileDataBodyPart("data", file);
    } catch (Throwable t) {
        bodypart = null;
    }
    if (bodypart == null) {
        return;  // error msg handled as a side effect
    }



    MediaType media = MultiPartMediaTypes.createFormData();
    FormDataMultiPart formdata = new FormDataMultiPart();
    formdata.setMediaType(media);

            //THIS IS SIMPLIFIED EXAMPLE HARDCODED OVERWRITE

    formdata = formdata.field("overwrite", "1");

        //I have also tried
    //FormDataBodyPart bp = new FormDataBodyPart("overwrite","1");
    //formdata.bodyPart(bp);


    formdata.bodyPart(bodypart);  // attach metadata
            WebResource add = resourceCollection.path("add");

    try {
        response = add.type(media)
                .accept(MediaType.APPLICATION_XML,MediaType.TEXT_PLAIN)  //TEXT_PLAIN
                .post(ClientResponse.class, formdata);
        resIn = response.getEntityInputStream();
        responseText = convertStreamToString(resIn);
        status = response.getStatus();

With various catches for the try. As far as I can tell the JavaClient-side of things is working fine, but once the server code receives the POST, the "data" field of formdata is missing, and only overwrite exists! Reiterating, this only applies to XML and not XLS.

Any tips? is this some inherent bug or am I doing something wrong?

The CherryPy we use is straight out of the box -- no modificaitons.

Thank you! Sean

Answer 1

For those with a similar issue...I have "solved" it.

Turns out the XML filesize was too small (<1Kb) to process via chunked encoding.

why? No idea. But using larger files I can overwrite and submit them fine, with all formvars present...

EDIT: default chunked size is 8192bytes, or 8kb. Uploads would fail consistently until the filesize was >= 8kb, so I believe apache expects atleast one chunk to be uploaded.

so..whatever. Thanks for reading!