Throughput of arrays of different sizes vary

Question

I have arrays like

byte[] b = new byte[10];
byte[] b1 = new byte[1024*1024];

I populate them with some values. Say,

for(i=0;i<10;i++){
     b[i]=1;
}
for(i=0;i<1024*1024;i++){
     b1[i]=1;
}

Then I write it to a RandomAccessFile and read again from that file into the same array using,

randomAccessFile.write(arrayName);
              and
randomAccessFile.read(arrayName);

When I try to calculate the throughput of both these arrays(using the time calculated for file read and write) of varying sizes(10 bytes and 1Mb), throughput appears to be more for 1MB array.

Sample Output:
Throughput of 10kb array: 0.1 Mb/sec.
Throughput of 1Mb array: 1000.0 Mb/sec.

Why does this happen? I have Intel i7 with quad core processor. Will my hardware configuration be responsible for this? If not what could be the possible reason?

Answer 1

The reason for the big difference is the overheads involved in I/O that occur no matter what the size of data being transferred - it's like the flag fall of a taxi ride. Overheads, which are not restricted to java and includes many O/S operations, include:

• Finding the file on disk
• Checking O/S permissions on the file
• Opening the file for I/O
• Closing the file
• Updating file info in the file system
• Many other tasks

Also, disk I/O is performed in pages (size depends on O/S, but usually 2K), so I/O of 1 byte probably costs the same as I/O of 2048 bytes: A slightly fairer comparison would be a 2048 byte array with a 1Mb array.

If you are using buffered I/O, that can further speed up larger I/O tasks.

Finally, what you report as "10Kb" is in fact just 10 bytes, so your calculation is possibly incorrect.