Read MemoryStream into Byte[] chunk at a time for processing

Question

I am passing a MemoryStream to a function - this could potentially end up being very large in size. This then needs to be split off into a small byte[] for processing and then added back to a stream. In my previous implementation I had roughly the following:

byte[] output = new byte[stream.Length];
output = stream.ToArray();
byte[] processingArray = new byte[16];

int index = 0;
int chunks = output / 16;
for(int i = 0; i < chunks; i++){
  for(int x = 0; x < 16; x++){
     processingArray[x] = output[x + index];
  }
  //do the processing on the array here
  index += 16;
}

I want to avoid having to use .ToArray() as this seems like a resource waste to me. How would I go about extracting 16 bytes at a time from the memorystream, processing it and then grabbing the next 16 bytes?

Thanks

Answer 1

You can use the Read method to specify a buffer and read block by block:

byte[] block = new byte[16];
int bytesRead = stream.Read(block, 0, block.Length);

In the above sample, bytesRead contains the number of bytes that was read in the call to Read. If there are no more bytes to read, bytesRead will be 0.

Answer 2

I don't quite understand why you want to extract parts of a MemoryStream for memory footprint. The MemoryStream itself is already in the memory, so why divide it in chunks?

If you call ToArray all the memory is copied, so why not directly write output to a new (Memory)Stream?

If you want to read the stream use this:

stream.Position = 0; // go to beginning of stream

byte[] buffer = new byte[16];
while(stream.Read(buffer, 0, buffer.Length) > 0)
{
    // process
}

Answer 3

GetBuffer() will return the internal array that MemoryStream is using. This allows you to bypass Read() to make an efficient zero-copy solution:

MemoryStream ms;

byte[] buffer = ms.GetBuffer();
int offset = ms.Position;
int left = ms.Length - offset;

while(left != 0)
{
    // process buffer starting at offset.
}