Write DataOutputStream in ByteArray / ByteBuffer with "dynamic" size?

Question

i ask myself if there is a nice solution for receiving data via TCP and reading the data in a DataInputStream and store the data in a bytearray of "dynamic" size. I thought about some solutions like writing in buffers and finally store it in a created array that is as large as the packet i received.

Here an example: the data i receive via TCP (byte by byte) is n*13 Byte large and the end of a packet is 13 Bytes of zeros (13 Byte of zeros are unique, cant be in the data before). The next packet is m*13 Byte + 13 Byte of zeros and so on. So i want to listen to the stream and store e.g. n*13 Byte in a bytearray without zeros (i dont know the size of one dataset before).

Can you tell me how to do it a slim way?

Thanks in advance!

Chris

Answer 1

I would read the data straight into a direct ByteBuffer. You don't need a DataInputStream as well. I would have the ByteBuffer larger than needed e.g. 1 MB or 10 MB and use the fact that unused direct buffers (like unused native memory) only uses virtual memory, not real memory. I would make sure this buffer is re-used as much as possible, ideally for the life of the application.

In a 64-bit environment you can make the buffers really large. e.g. 1 GB without much impact.

If you want to minimise memory consumption, I would process the data as you receive it. That way you avoid having to store it first. It sounds like the minimum size is 13 bytes and would make a reasonable buffer size like 512 bytes or 2 KB.