How can I replace DataTable code with SqlCeDataReader or SqlCeResultSet?

Question

The CF Whisperer recommended this:

"Avoid DataTables. They are gigantic memory hogs, and under CE you can't afford a memory hog. Basically it duplicates the entire source table into RAM for you. SqlCeDataReader and SqlCeResultSet are your friends."

here

...regarding this code:

var dt = new DataTable();
dt.Columns.Add(new DataColumn("Id", typeof(Int32)));
dt.Columns.Add(new DataColumn("DeptNumber", typeof(Int32)));
dt.Columns.Add(new DataColumn("DeptName", typeof(string)));

foreach (HHSUtils.Department dept in depts)
{
    try
    {
        dt.Rows.Add(dept.Id, dept.DeptNumber, dept.DeptName);
        countAdded++;
    }
    catch (Exception ex)
    {
        MessageBox.Show(string.Format("deptId == {0}, deptNumber == {1},  
          deptName == {2} ({3})", dept.Id, dept.DeptNumber, dept.DeptName, ex.Message));
    }
}
// Now move it all into the table
using (var bc = new SqlCeBulkCopy(dataSource))
{
    bc.DestinationTableName = "Departments";
    bc.WriteToServer(dt);
}

I don't understand how I'm to replace DataTable with SqlCeDataReader or SqlCeResultSet here. My code above with DataTable is DDL and doesn't seem to be equate (in my mind) to SqlCeDataReader or SqlCeResultSet, as their purpose is (reputedly?) for working with queried data...???

Answer 1

I don't fully understand the first loop. If you have a set of Department items, why then duplicate all of them into a DataTable at all? You're effectively just making a second copy of each into memory, and if there are a lot of them, that will be a problem. It's also a waste of processing time.

I'd be far more inclined to do something like this (untested, uncompiled pseudocode):

var cmd = new SqlCeCommand(connection);
cmd.CommandText = "MyTableName";
cmd.CommandType = CommandType.TableDirect
using(var rs = cmd.ExecuteResultSet(
                      ResultSetOptions.Updatable | ResultSetOptions.Scrollable))
{
    foreach(var dept in depts)
    {
        var record = rs.CreateRecord();
        record.SetInt(0, dept.ID);
        // do other fields

        rs.Insert(record);
    }
}

We're doing better because:

1. We're not duplicating records
2. We're not effectively iterating the data set twice
3. We're going table direct