Input stream reads large files very slowly, why?

Question

I am trying to submit a 500 MB file. I can load it but I want to improve the performance. This is the slow code:

File dest = getDestinationFile(source, destination);
if(dest == null) return false;

in = new BufferedInputStream(new  FileInputStream(source));
out = new BufferedOutputStream(new  FileOutputStream(dest));
byte[] buffer = new byte[1024 * 20];
int i = 0;

// this while loop is very slow
while((i = in.read(buffer)) != -1){
   out.write(buffer, 0, i); //<-- SLOW HERE
   out.flush();
}

How can I find why it is slow?
Isn't the byte array size / buffer size sufficient? Do you have any ideas to improve the performance or?

Thanks in advance for any help

Answer 1

You should not flush in loop. You are using BufferedOutputStream. This mean that after "caching" some amount of data it flushes data to file. Your code just kills performance by flushing data after writing a little amount of data.

try do this like that:

while((i = in.read(buffer)) != -1){
out.write(buffer, 0, i); <-- SLOW HERE
}
out.flush();

..:: Edit: in response of comment below ::..
In my opinion you should not use buffer at all. You are using Buffered(Output/Input)Stream which means that they have his own buffer to read "package" of data from disk and save "package" of data. Im not 100% sure about performance in using additional buffer but I want you to show how I would do that:

File dest = getDestinationFile(source, destination);
if(dest == null) return false;

in = new BufferedInputStream(new  FileInputStream(source));
out = new BufferedOutputStream(new  FileOutputStream(dest));

int i;
while((i = in.read()) != -1){
   out.write(i);
}
out.flush();

In my version you will just read a BYTE (no a int. Read doc:
http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read()
this method returns int but this is just a BYTE) but there is no need to read a whole buffer (so you don't need to be worry about size of it).

Probably you should read more about streams to better understand what is nessesary to do with them.