SQL ODBC adapter out of memory exception

Question

I am connecting and receiving data from IBM's I Series AS400 database, using odbc adapters and dataset.

I have an odbc connection like given below:

OdbcCommand cmd = new OdbcCommand(queryString, conn);           // Set Active Query
OdbcDataAdapter rt = new OdbcDataAdapter(queryString, conn);    // Active Data Transfer
DataSet ds = new DataSet();                                     // Create DataSet
rt.SelectCommand.CommandTimeout = 180;                          // Set Command Timeout
rt.Fill(ds);                                                    // Transfer All Data
var reader = ds.CreateDataReader();                             // Create Reader
reader.Read();                                                  // Read
while (reader.Read()) { ... }

and it strangely gives the System.OutOfMemoryException on line rt.Fill(ds);

If there are 1billion rows with 130 columns would it cause this error?

How can i avoid this error and recieve the data i want?

Answer 1

If there are 1billion rows with 130 columns would it cause this error?

That seems like a lot of data and would likely cause the error.

How can i avoid this error and recieve the data i want?

You should use the DataReader directly, bypassing the DataSet. If you do this, the DataReader will load only one row of data, only getting the next row when you call Read.

OdbcCommand cmd = new OdbcCommand(queryString, conn); // Set Active Query
conn.Open();                          // You may not need this if already open
var reader = cmd.ExecuteReader();     // Create Reader
while (reader.Read()) { ... }         // Process records, one at a time
reader.Close();                       // Close Reader after use

More info:

OdbcDataReader Class at MSDN

Contrasting the ADO.NET DataReader and DataSet