Question

Why is java.nio.FileChannel transferTo() and transferFrom() faster than byte-by-byte transfer (stream based or using ByteBuffer) on some JVM/OS combinations???

Do these methods use direct memory access (DMA) as opposed to issuing interrupt requests (IRQ) for each byte transfer??

Was it helpful?

Solution

Do these methods use direct memory access (DMA) as opposed to issuing interrupt requests (IRQ) for each byte transfer?

The specifics are up to the actual implementation, and they are not required to make use of any particular mechanism. This is hinted at in the documentation for transferTo (emphasis mine):

This method is potentially much more efficient than a simple loop that reads from this channel and writes to the target channel. Many operating systems can transfer bytes directly from the filesystem cache to the target channel without actually copying them.

'Potentially', 'many'... so no guarantees.

It does make sense to assume that it's probably more efficient to use the method, if only marginally so, because you allow the JVM to shortcut through native code (if using supported channel types). The 'simple loop' they describe works sort of like below:

ByteBuffer buf = ByteBuffer.allocateDirect(BUF_SIZE);
while ( /* read more condition */ ) {
  source.read(buf);
  buf.flip();
  target.write(buf);
  buf.compact();
}

Note that, even though this snippet uses direct buffers, you're still poking back into Java for buffer management (read, flip, write, compact). An optimising compiler may be able to elide some of it, but it probably won't.

Using transferTo/transferFrom, however, leaves it up to the JVM to decide how to transfer the bytes. If the platform has native support for this kind of transfers, it may be able to do so without creating intermediary buffers. If there is no such support, the JVM can still implement a loop such as above.

Example: suppose a SocketChannel is told to read data directly from a FileChannel through transferFrom. The FileChannel was recently read from, and its contents are in the OS file cache. Rather than read and copy the bytes into a buffer, the SocketChannel can point directly into the OS file cache and start transmitting from there. At least one round of copying eliminated.

Now further suppose that the socket (A) is actually connected to some other local process, for instance using a sort of pipe, called SocketChannel B. When B starts reading what it got from A, it might actually be reading straight from the OS file cache again. If then B just uses transferTo to another channel... you get the idea.

OTHER TIPS

There is some explanation in FileChannel API

This method is potentially much more efficient than a simple loop that reads from the source channel and writes to this channel. Many operating systems can transfer bytes directly from the source channel into the filesystem cache without actually copying them.

BTW both transferTo() and transferFrom() are abstract, so it all depends on the actual implementation

It does use DMA/Zero-copy, thus saving transferring from the "From" buffer to the CPU and then the transfer from the CPU to the "to" buffer. For a more detailed explanation read this article from IBM

In the following article, on zero copy

https://www.ibm.com/developerworks/linux/library/j-zerocopy/

the authors explain that the new methods are more efficient on Linux because they reduce context switches and reduce unnecessary buffer copying from kernel to application and back.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top