.NET Micro Framework, reading files on a device with limited memory

Question

On a ChipworkX device we would read files using:

File.ReadAllBytes(filename);

But if we try that on a NetDuino Plus which has a much smaller amount of memory,
we simply get an OutOfMemoryException.

The files are not that big, but I guess that's all relative in this case (1.5kb max).

What's the correct way to read files on a device like this?

Answer 1

Use a FileStream

using (var fileStream = new FileStream(filename, FileMode.Open))
{
    byte[] block = new byte[1024];
    int readLength;
    while ((readLength = fileStream.Read(block, 0, block.Length)) > 0)
    {
        Process(block, readLength);
    }
}

write your own Process method. The block length of 1024 is just an example, read as big chunks as you can process at a time. You can vary that depending on the data.

Answer 2

I am assuming that you believe that there should be sufficient memory. If this is so, I suspect that internal default buffer sizes are blowing things. Try explicitly stating buffer sizes when opening the file to keep it tight to the actual file length:

string path = //some path
byte[] buffer;
int bufferSize = (int)new FileInfo(path).Length;

using (FileStream fs = new FileStream(
    path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize))
{
    buffer = new byte[bufferSize];

    fs.Read(buffer, 0, buffer.Length);
}

//do stuff with buffer 

Answer 3

When you are using a device with limited memory, it is a good idea to use a buffer that is the size of a sector. What you are doing is trading speed for memory. When you have little memory, you must do things more slowly and a sector is the smallest unit you can use that makes any sense.

I suggest a buffer of 512 bytes.