Arduino+Ethernet-SD shield, listing all filenames existing files on the SD card

Question

I have: Arduino MEGA 2560; Ethernet+SD shield http://www.amazon.com/dp/B0022TWQ22/?tag=stackoverfl08-20 ; SD card 2GB FAT.

SD contains 400 files with names 00000000; 0000001; 0000002; ... 00000098; 0000099; 0000100; ... 00000398; 00000399.

I need to construct String var which will contain all the Filenames separated by ";" like this: sdata = "0000001;0000002;0000003 ... 00000398;00000399;";

Code:

#include <SdFat.h>
#include <SPI.h>


const uint16_t chipSelect = SS;
char cnamefile[9];
String sdata="";
SdFat sd;
SdFile file;

void setup() {


  Serial.begin(9600);
  Serial.println("hi");
  sdata="";
  if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();
    Serial.println("List files");
    while (file.openNext(sd.vwd(), O_READ)) {
      file.getFilename(cnamefile);
      file.close();
      sdata = sdata + String(cnamefile) + ";";

    }
  Serial.print(sdata);
}

void loop() { 
}

Listening to the COM port i see:

hi
List files
00000005;00000006;00000007;00000008;00000009;00000010;00000011;00000012;00000013;00000014;00000015;00000016;00000017;00000018;00000019;00000020;00000021;00000022;00000023;00000024;00000025;00000026;00000027;00000028;00000029;00000030;00000031;00000032;00000033;00000034;00000035;00000036;00000037;00000038;00000039;00000040;00000041;00000042;00000043;00000044;00000045;00000046;00000047;00000048;00000049;00000050;00000051;00000052;00000053;00000054;00000055;00000056;00000057;00000058;00000059;00000060;00000061;00000062;00000063;00000064;00000065;00000066;00000067;00000068;00000069;00000070;00000071;00000072;00000073;00000074;00000075;00000076;00000077;00000078;

How to fix this problem and put all filenames in one variable?

Information for: 400 names and 400 ";" its 3600 bytes. When i try to read any file and put all its contents (more than 3600 bytes) in "String sdata" it works normally. Problem only with listing. Please help me in sorting out this issue.

Answer 1

This seems about the correct place that your program will fail. This innocent line is your problem:

sdata = sdata + String(cnamefile) + ";";

String concatentation like this will use 2X the memory of the sdata for a short moment. This is the how you should view the sequence of operations in that one line

// compiler has done this for you:
String temp1234 = sdata + String();

// note that at this moment, memory usage is now 2x sdata

String temp1235 = temp1234 + ";";

// now you can have 3x the memory used

// assignment to variable
sdata = temp1235;

// now can delete temporary variable
// compiler will do this
//temp1234.delete()
//temp1235.delete()

You are trying to create strings up to 3k bytes but have only 8k total RAM, so will not be able to do the above.

This demonstrates a couple of points about Strings. Your concatenation above on one line, is not necessarily better than this two line form:

sdata = sdata + String(cnamefile);
sdata = sdata  + ";";

In this second form, you are ensured there will only be one temporary variable for the intermediate result.

This leads to the next hint. You should be thinking how am I going to escape the temporary variable. That is why we have += operator. Your best chance is to concatenate like this:

sdata += String(cnamefile);
sdata += ";";

If the += operator is available on the String class, the compiler will use this. That operator may be able to use a more memory efficient way of concatenation. For example, if the String was preallocated with some extra memory, then it could just place the new characters into the existing buffer.

In general this is a great learning method about strings in constrained memory spaces, because you must understand some compiler internals and operator details that are often ignored in large CPU environments.

Given the sizes you are suggesting, you will probably only be able to fit in RAM if you change to an approach of pre-constructing a String buffer and filling it with the file names. In other words: don't use String on a microcontroller.