Handling a large ADODB record set in data table?

Question

I am loading a large result set of about 3 million rows (ADODB record set) into a data table. Its taking too long to even load the result set into a data table. I want to find out a way of extracting only part of a result set and then loading it into a DataTable. Alternatively, is there a way to directly read the recordset directly instead of loading it into a data table and then reading it ?

This is the code I use to fill my DataTable -

OleDbDataAdapter oleDA = new OleDbDataAdapter();
DataTable dt = new DataTable();
oleDA.Fill(dt, myADODBRecordset);

Answer 1

Here are some options to consider:

1. Get only the rows and columns you really need to work with.

2. Get some data and let the user ask for the next set of rows when they want to.

3. Write optimized SQL queries

4. Don't use a DataTable unless you have to because it contains more metadata information that other list-type objects.

5. Consider using a managed .NET provider.

Answer 2

Why load so large data into memory? A large data transaction must comsume large resource, so optimize your code, or use EF.