.SqlDataReader.ReadColumnHeader NullReferenceException

Question

This question is somewhat related to my previous one answer to which helped me to find the reason why this simple iteration thru SqlDataReader:

m_aFullIDList = New Generic.List(Of Integer)

While i_oDataReader.Read
    m_aFullIDList.Add(i_oDataReader.GetInt32(0))
End While

m_iTotalNumberOfRecords = m_aFullIDList.Count

does not return all the records. Turned out when Generic.List changes its Capacity to accommodate more elements (e.g from 2^19 to next one 2^20) at this point SqlDataReader simple quits, its Read method returns False as if there's no more records.

Most of the time it quits quietly, no exception is thrown whatsoever. But every now and then I'd get:

NullReferenceException {"Object reference not set to an instance of an object."}

at System.Data.SqlClient.SqlDataReader.ReadColumnHeader(Int32 i)
at System.Data.SqlClient.SqlDataReader.ReadColumn(Int32 i, Boolean setTimeout) at System.Data.SqlClient.SqlDataReader.GetInt32(Int32 i)

I know for a fact that all records returned by Stored Procedure used by the Reader (it's a single column) are integer values. If I remove line m_aFullIDList.Add and instead simple read value into an integer variable OR if I pre-allocate Generic List capacity to a know large number - this issue is not happening. Apparently it only happens when List reallocates Capacity - this affects the reader.

I also attempted to use other structures (ArrayList, even Array, using Array.Resize) as soon as capacity of this structure is reallocated beyond certain point - this breaks the SqlDataReader.

This ASP.NET project is kinda complex, so when I tried to recreate the issue in a standalone simple project consisting only of executing reader and reading into List - the issue is not happening. Any idea what is going on and how this could be fixed?

Answer 1

I think I finally figured it out.

The logic in my code had 2 steps - call the function that returns SqlDataReader and then use that reader in another function to fill the list:

Dim oReader as SqlDataReader = GetTheReader()

FillTheList(oReader)

Function GetTheReader() looked something like this:

Function GetTheReader() as SqlDataReader
   Dim oConn As New SqlConnection("Connection String") : oConn.Open()
   Dim oComm As New SqlCommand("Stored Procedure", oConn)

   Dim oReader As SqlDataReader = oComm.ExecuteReader(CommandBehavior.CloseConnection)

   Return oReader
End Function

It opened connection as a local variable which went out of scope when the function returned back to the caller. And when Generic List was being populated, after another Capacity allocation Garbage Collection claimed that memory by destroying outdated variables (and closing that connection). I was left with a SqlDataReader without a valid connection.

My current solution is to create connection outside GetTheReader function and pass it as one of the parameters.