Memory efficient way to add part of an Array to an List

Question

Let's say I have the following variables:

byte[] fileData;
List<byte> bundleData;

and I'd like to take a contiguous portion of fileData and add it to bundleData. My current method is essentially the following:

int startIndex = 20, endIndex = 80;
byte[] transferredData = new byte[endIndex - startIndex];
Array.Copy(fileData, startIndex, transferredData, 0, transferredData.Length);
bundleData.AddRange(transferredData);

Creating the intermediate array works fine, but it uses what seems like an unnecessary copy. Is there any way to add the data directly, without the use of a redundant array?

For reference, I'm using .NET 2.0 on this project.

Answer 1

If you do not actually need a List<byte> instance, a better option might be the MemoryStream class:

MemoryStream Class

Creates a stream whose backing store is memory.

Example:

MemoryStream bundleData = new MemoryStream();
bundleData.Write(fileData, startIndex, endIndex - startIndex + 1);

Answer 2

Another (probably efficient) way would be to use LINQ:

bundleData.AddRange(fileData.Skip(startIndex).Take(endIndex - startIndex));

Answer 3

The List<T> class essentially just wraps a T array that is replaced with a larger T array when the array is full. The fastest way to append your byte array to List<byte> would be to copy the bytes directly into the internal byte array.

However, the List<T> class does not expose the internal array, so the best option is probably to ensure that the list has enough capacity and does not need to replace the internal array multiple times, and then add each byte one by one:

bundleData.Capacity += endIndex - startIndex + 1;

for (int i = startIndex; i <= endIndex; i++)
{
    bundleData.Add(fileData[i]);
}

You could also experiment with AddRange and providing a view of the byte array:

static IEnumerable<T> Range<T>(this T[] array, int offset, int count)
{
    for (int i = 0; i < count; i++)
    {
        yield return array[offset + i];
    }
}

bundleData.AddRange(fileData.Range(startIndex, endIndex - startIndex + 1));

Answer 4

List.AddRange method is implemented as below. I'll add some pseudo code to explain.

ICollection<T> is2 = collection as ICollection<T>;
if(is2!=null)
{
   //then use Array.Copy
}
else
{
 //Loop through GetEnumerator() and calls Insert()
}

So, Intermediate array will be the best idea IMHO since Array Implements ICollection. Hope this helps.