Iterate through files and read records

Question

What is the fastest way to the following in C# 3.5 :

1. Iterate through files in a directory
2. Read the file's records (fixed length of 247 characters)
3. Convert the fixed length string of each record to a Struct or Class.

Thanks

Answer 1

This would be relatively fast to write:

var myStructs = 
from file in Directory.GetFiles(".", "*.*", SearchOption.TopDirectoryOnly)
select ConvertFileToStructs(File.ReadAllText(file));

If this is the fastest way possible, performance-wise? Probably not, but it won't make a huge difference. What will impact the performance is the implementation of the deserialization within the ConvertFileToStructs() function. But to answer this, we need to know the specific format of your files.

---

Just read your comments. I would suggest the following parsing:

List<MyStruct> ConvertFileToStructs(string content, int[] mapping)
{
    var records = new List<MyStruct>();
    int length = content.Length();
    for(int i = 0; i < length; i += 247)
         records.Add(ConvertRecordToStruct(content.Substring(i,247), mapping));
    return records;
}

MyStruct ConvertRecordToStruct(string record, int[] mapping)
{
    MyStruct s;
    s.Field1 =  record.Substring(mapping[0], mapping[1]);
    //set other fields
    return s;
}

This code could probably be optimized for performance, but I don't think it would change things dramatically, especially because I/O to disk is involved and Substring() is pretty fast (see http://dotnetperls.com/substring). Of course you will have to test this on your machine.

Answer 2

custom class to handle files

   class customFile
        {
            string fileText;
            public string FileText
            {
                get { return fileText; }
                set { fileText = value; }
            }
        }

read all text

        string[] filePaths = Directory.GetFiles(dirPath);
        List<customFile> customFiles = new List<customFile>();
        foreach (string file in filePaths)
        {
            customFiles.Add(new customFile { FileText = File.ReadAllText(file) });
        }