Reading textfile parsing rows into a Datatable

Question

I want to read a text file and parse each row into a datable.

The way I'd like the data stored in datatable is with the type for each column set as well i.e. int, varchar, date etc.

DataTable data = new DataTable();
data.Columns.Add("code", typeof(int));
data.Columns.Add("description", typeof(string));
data.Columns.Add("value", typeof(decimal));

but then when I am reading the textfile with columns tab delimited, how could I ensure the correct columns in the text file go into the right columns in the datatable.

Because my ultimate aim is to get this datatable into a sql server table with the same columns and types using sqlbulkcopy.

Answer 1

Don't reinvent the wheel, use an available CSV-parser like this.

DataTable data = new DataTable("CSV");
var fileInfo = new System.IO.FileInfo("Path");
using (var reader = new System.IO.StreamReader(fileInfo.FullName, Encoding.Default))
{
    // use  reader.ReadLine(); to skip all lines but header+data
    Char quotingCharacter = '\0';//'"';
    Char escapeCharacter = quotingCharacter;
    using (var csv = new CsvReader(reader, true, '\t', quotingCharacter, escapeCharacter, '\0', ValueTrimmingOptions.All))
    {
        csv.MissingFieldAction = MissingFieldAction.ParseError;
        csv.DefaultParseErrorAction = ParseErrorAction.RaiseEvent;
        //csv.ParseError += csv_ParseError;
        csv.SkipEmptyLines = true;
        // load into DataTable
        data.Load(csv, LoadOption.OverwriteChanges, csvTable_FillError);
    }
}

Answer 2

You will need to ensure that the columns in the file are always in the correct order. Or if the file could include a header, you can read the file into a list of dictionary rows where the key is the header name/datatable column name.

Reading the file would then be a case of building a list of lines, that can be split on the tab character.