Is it possible to read the Process stdout InputStream into an NIO ByteBuffer?
-
06-07-2019 - |
Question
Is it possible to use NIO to process the stdout from a Process? I have it working with java.io, but this is something of an exercise to learn a bit more about NIO and to explore the possibility of performance improvements.
Basically I want to stream a large volume of text from stdout into a buffer as fast as possible without blocking, and then process the contents of that buffer later. The trouble is, I can't seem to figure out the right voodoo to make it work with NIO. This is where I am right now:
ProcessBuilder pb = new ProcessBuilder( ... );
Process p = pb.start();
stdout = new StreamConsumer(p.getInputStream());
new Thread(stdout).start();
// other stuff omitted for brevity
The class StreamConsumer looks like this:
class StreamConsumer implements Runnable
{
private InputStream is;
public StreamConsumer(InputStream is)
{
this.is = is;
}
public void run()
{
try
{
ReadableByteChannel source = Channels.newChannel(is);
// Is it possible get a channel to a ByteBuffer
// or MappedByteBuffer here?
WritableByteChannel destination = ??;
ByteBuffer buffer = ByteBuffer.allocateDirect(128 * 1024);
while (source.read(buffer) != -1)
{
buffer.flip();
while (buffer.hasRemaining())
{
destination.write(buffer);
}
buffer.clear();
}
source.close();
destination.close();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
Solution
Believe it or not, I think the writable byte channel you want is
ByteArrayOutputStream ostream = new ByteArrayOutputStream(<some large number>);
WritableByteChannel destination = Channels.newChannel(ostream);
Then when done
ostream.toByteArray()
contains the bytes to process. Or, if you want a byte buffer,
ByteBuffer.wrap(ostream.toByteArray())
I don't see here how you get the output outside the runnable, but I suspect your original code had that. Otherwise you might want the StreamConsumer
to be a Callable<ByteBuffer>
.
OTHER TIPS
I have created an open source library that allows non-blocking I/O between java and your child processes. The library provides an event-driven callback model. It depends on the JNA library to use platform-specific native APIs, such as epoll on Linux, kqueue/kevent on MacOS X, or IO Completion Ports on Windows.
The project is called NuProcess and can be found here:
you might want the StreamConsumer to be a Callable.
Another non blocking option to try might be to use Guava's ListenableFuture giving you success and failure callbacks without interpreting errors of your own.