File Upload with Java (with progress bar)

Question

I'm extremely new to Java, and have mostly just been teaching myself as I go, so I've started building an applet. I'd like to make one that can select a file from the local disk and upload it as a multipart/form-data POST request but with a progress bar. Obviously the user has to grant permission to the Java applet to access the hard drive. Now I've already got the first part working: the user can select a file using a JFileChooser object, which conveniently returns a File object. But I'm wondering what comes next. I know that File.length() will give me the total size in bytes of the file, but how do I send the selected File to the web, and how do I monitor how many bytes have been sent? Thanks in advance.

Answer 1

I ended up stumbling across an open source Java uploader applet and found everything I needed to know within its code. Here are links to a blog post describing it as well as the source:

Article
Source Code

Answer 2

To check progress using HttpClient, wrap the MultipartRequestEntity around one that counts the bytes being sent. Wrapper is below:

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import org.apache.commons.httpclient.methods.RequestEntity;

public class CountingMultipartRequestEntity implements RequestEntity {
    private final RequestEntity delegate;

    private final ProgressListener listener;

    public CountingMultipartRequestEntity(final RequestEntity entity,
            final ProgressListener listener) {
        super();
        this.delegate = entity;
        this.listener = listener;
    }

    public long getContentLength() {
        return this.delegate.getContentLength();
    }

    public String getContentType() {
        return this.delegate.getContentType();
    }

    public boolean isRepeatable() {
        return this.delegate.isRepeatable();
    }

    public void writeRequest(final OutputStream out) throws IOException {
        this.delegate.writeRequest(new CountingOutputStream(out, this.listener));
    }

    public static interface ProgressListener {
        void transferred(long num);
    }

    public static class CountingOutputStream extends FilterOutputStream {

        private final ProgressListener listener;

        private long transferred;

        public CountingOutputStream(final OutputStream out,
                final ProgressListener listener) {
            super(out);
            this.listener = listener;
            this.transferred = 0;
        }

        public void write(byte[] b, int off, int len) throws IOException {
            out.write(b, off, len);
            this.transferred += len;
            this.listener.transferred(this.transferred);
        }

        public void write(int b) throws IOException {
            out.write(b);
            this.transferred++;
            this.listener.transferred(this.transferred);
        }
    }
}

Then implements a ProgressListener which updates a progress bar.
Remember that the progress bar update must not run on the Event Dispatch Thread.

Answer 3

A simpler countingEntity would not depend on a specific entity type but rather extend HttpEntityWrapped:

package gr.phaistos.android.util;

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.http.HttpEntity;
import org.apache.http.entity.HttpEntityWrapper;

public class CountingHttpEntity extends HttpEntityWrapper {

    public static interface ProgressListener {
        void transferred(long transferedBytes);
    }


    static class CountingOutputStream extends FilterOutputStream {

        private final ProgressListener listener;
        private long transferred;

        CountingOutputStream(final OutputStream out, final ProgressListener listener) {
            super(out);
            this.listener = listener;
            this.transferred = 0;
        }

        @Override
        public void write(final byte[] b, final int off, final int len) throws IOException {
            //// NO, double-counting, as super.write(byte[], int, int) delegates to write(int).
            //super.write(b, off, len);
            out.write(b, off, len);
            this.transferred += len;
            this.listener.transferred(this.transferred);
        }

        @Override
        public void write(final int b) throws IOException {
            out.write(b);
            this.transferred++;
            this.listener.transferred(this.transferred);
        }

    }


    private final ProgressListener listener;

    public CountingHttpEntity(final HttpEntity entity, final ProgressListener listener) {
        super(entity);
        this.listener = listener;
    }

    @Override
    public void writeTo(final OutputStream out) throws IOException {
        this.wrappedEntity.writeTo(out instanceof CountingOutputStream? out: new CountingOutputStream(out, this.listener));
    }
}

Answer 4

The amount of bytes returned by the listener is different from the original file size. So, instead of having transferred++, I modified it so that transferred=len; that is the length of the actual amount of bytes being written to the output stream. And when I compute the addition of the total bytes transferred it is equal to the actual ContentLength returned by CountingMultiPartEntity.this.getContentLength();

public void write(byte[] b, int off, int len) throws IOException {
    wrappedOutputStream_.write(b,off,len);
    transferred=len;
    listener_.transferred(transferred);
}

Answer 5

You might find this article helpful. It explains in detail using HttpClient and FileUpload, both apache projects to do what you want. It also includes code samples.

Answer 6

Keep in mind that the progress bar might be misleading when an intermediate component in the network (e.g., an ISP's HTTP proxy, or a reverse HTTP proxy in front of the server) consumes your upload faster than the server does.

Answer 7

As noted by the article Vincent posted, you can use Apache commons to do this.

Little snipped

DiskFileUpload upload = new DiskFileUpload();
upload.setHeaderEncoding(ConsoleConstants.UTF8_ENCODING);

upload.setSizeMax(1000000);
upload.setSizeThreshold(1000000);

Iterator it = upload.parseRequest((HttpServletRequest) request).iterator();
FileItem item;
while(it.hasNext()){
    item = (FileItem) it.next();
    if (item.getFieldName("UPLOAD FIELD"){
       String fileName = item.getString(ConsoleConstants.UTF8_ENCODING);
       byte[] fileBytes = item.get();
    }
}

Answer 8

Just my 2c worth:

This is based off of tuler's answer(has a bug at time of writing). I modified it slightly, so here is my version of tuler and mmyers answer (I can't seem to edit their answer). I wanted to attempt to make this a bit cleaner and faster. Besides the bug(which I discuss in comments on their answer), the big issue I have with their version is that it creates a new CountingOutputStream with every write. This can get very expensive in terms of memory - tons of allocations and garbage collections. Smaller issue is that is uses a delegate when it could just expand the MultipartEntity. Not sure why they chose that, so I did it in a manner I was more familiar with. If anyone knows pros/cons of the two approaches that would be great. Finally, the FilterOutputStream#write(byte[], int,int) method just calls the FilterOutputStream#write(byte) in a loop. The FOS documentation recommends subclasses overriding this behavior and making this more efficient. The best way to do that here is to let the underlying OutputStream handle the writing request.

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;

public class CountingMultiPartEntity extends MultipartEntity {

    private UploadProgressListener listener_;
    private CountingOutputStream outputStream_;
    private OutputStream lastOutputStream_;

    // the parameter is the same as the ProgressListener class in tuler's answer
    public CountingMultiPartEntity(UploadProgressListener listener) {
        super(HttpMultipartMode.BROWSER_COMPATIBLE);
        listener_ = listener;
    }

    @Override
    public void writeTo(OutputStream out) throws IOException {
        // If we have yet to create the CountingOutputStream, or the
        // OutputStream being passed in is different from the OutputStream used
        // to create the current CountingOutputStream
        if ((lastOutputStream_ == null) || (lastOutputStream_ != out)) {
            lastOutputStream_ = out;
            outputStream_ = new CountingOutputStream(out);
        }

        super.writeTo(outputStream_);
    }

    private class CountingOutputStream extends FilterOutputStream {

        private long transferred = 0;
            private OutputStream wrappedOutputStream_;

        public CountingOutputStream(final OutputStream out) {
            super(out);
                    wrappedOutputStream_ = out;
        }

        public void write(byte[] b, int off, int len) throws IOException {
                    wrappedOutputStream_.write(b,off,len);
                    ++transferred;
            listener_.transferred(transferred);
        }

        public void write(int b) throws IOException {
            super.write(b);
        }
    }
}

Answer 9

Look into HTTP Client for uploadign the file to the web. It should be able to to do that. I am unsure how to get the progress bar, but it would involve querying that API somehow.

Answer 10

Apache common is very good option. Apache common allows you to configure following things.

1. Configure(xml file) the maximum file size/ upload file size
2. Destination path (where to save the uploaded file)
3. Set the temp. folder to swap the file , so that file upload would be fast.

Answer 11

From the other answers you can just override the AbstractHttpEntity class children or implementations public void writeTo(OutputStream outstream) method you are using if do not want to create a class.

An example using a FileEntity instance:

FileEntity fileEntity = new FileEntity(new File("img.jpg")){
    @Override
    public void writeTo(OutputStream outstream) throws IOException {
        super.writeTo(new BufferedOutputStream(outstream){
            int writedBytes = 0;

            @Override
            public synchronized void write(byte[] b, int off, int len) throws IOException {
                super.write(b, off, len);

                writedBytes+=len;
                System.out.println("wrote: "+writedBytes+"/"+getContentLength()); //Or anything you want [using other threads]
            }
        });
    }

};