Efficient Data Transfer mechanism

Question

I am developing a Java class to transfer data from any input stream to any output stream (mainly to be used on-line, but with a broader range of utilities).

After wandering through the Java 7 API specification I noticed two methods from the FileChannel class: transferFrom(...) and transferTo.

I developed my class around these methods and created the following transmit() method:

public void transmit() throws IOException {
    File tmp = File.createTempFile("transmit", ".tmp");
    RandomAccessFile raf = new RandomAccessFile(tmp, "rw");
    FileChannel fc = raf.getChannel();
    fc.force(true);
    fc.transferFrom(Channels.newChannel(src), 0, Long.MAX_VALUE);
    raf.seek(0);
    fc.transferTo(0, Long.MAX_VALUE, Channels.newChannel(dst));
    raf.close();
}

And then I decided to test it out a bit with the following test (the first version did not use a temporary byte array and as expected did worse than the transmit() method):

public static void main(String[] args) throws IOException {
    File from = File.createTempFile("source", ".tmp");
    File to = File.createTempFile("destination", ".tmp");

    FileOutputStream tmp = new FileOutputStream(from);

    for (int i = 0; i < (1 << 20); i++) {
        tmp.write(0);
    }

    tmp.close();

    FileInputStream fin = new FileInputStream(from);
    FileOutputStream fout = new FileOutputStream(to);

    DataTransmitter dt = new DataTransmitter(fin, fout);

    long time_1 = new Date().getTime();
    dt.transmit();
    time_1 = new Date().getTime() - time_1;

    fin.close();
    fout.close();

    to.delete();

    fin = new FileInputStream(from);
    fout = new FileOutputStream(to);


    int len;
    byte[] b = new byte[8192];
    long time_2 = new Date().getTime();
    while ((len = fin.read(b)) >= 0) {
        fout.write(b, 0, len);
    }
    time_2 = new Date().getTime() - time_2;

    fin.close();
    fout.close();

    System.out.format("Transmitter method: %s milliseconds%n", time_1);
    System.out.format("Direct method: %s milliseconds%n", time_2);
}

On a file located in the local hard drive the so called "direct method" was quite faster (192 milliseconds for the transmit() method and 8 milliseconds for the "direct method") meaning the "direct method" was highly efficient compared to the transmit() method.

However, since the purpose of this class is to download files from an on-line source I created a 40 Mb file that I uploaded to a personal cloud and (from a method similar to the above) got the following results:

Transmitter method: 126478 milliseconds
Direct method (8192 bytes): 134105 milliseconds

Which means the transmit() method was more efficient coming from an on-line source.

I was wondering if these results are accurate (since these methods are system dependent I state that I'm running Ubuntu 13.10) and if they are, how can I find a way to optimize the transmit() method to a general situation.

If there are alternative methods that are more efficient I would also appreciate if someone stated them (no code is required, just an explanation of the method and maybe the reason why it is more efficient).

Answer 1

Don't write your own data transfer code, unless you have a very specific reason to do so. There are many data transfer java libraries: IOUtils, some parts of Apache Mina, etc. You should not reinvent the wheel, unless you want to do it for a school project or you see a weakness in the existing solutions which you want to improve on.

Also, speaking of bad practices, you shouldn't be using java's Date, but rather a more sane Date functionality, like Joda's DateTime and you shouldn't be opening/closing your own streams but either use a library that does it for you or use Java 7's Closable and try-with-resources statement.

EDIT: Another bad practice you do in your example is that you use java.util.Date to obtain the current time. This is wrong on two levels: code readability and thread safety.

For the code readability part: what you are trying to do is 'give me the current time', which you can clearly do by System.getCurrentTimeMillis(). What you are actually doing is:

'allocate a semi-deprecated object, don't specify timezone or locale (which might change from the first call to the second call), don't be thread-safe, and give me its current time so I can use it as my current time'

. Creating two extra objects when you can easily avoid it is not a big deal, but since a method for giving you the current time exists, and it is more efficient, you should use it, otherwise you are developing bad practices.

For the thread safety part, you can read this: How to make Java.util.Date thread-safe

Just code using java.util.Date is inherently prone to errors and you shouldn't be using it. It might not seem like a big deal in this example, but details are important if you are developing yourself as a software engineer. In addition, you shouldn't be writing your own methods for data transfer unless you know what is wrong with the existing ones and you want to improve on that or you are doing it for the purpose of a homework or study.