Going from Dictionary<int, StringBuilder> To Table-Valued SqlParameter. How?

Question

I have code that has a Dictionary defined as:

Dictionary<int, StringBuilder> invoiceDict = new Dictionary<int, StringBuilder>();

Each Value in each KeyValuePair the Dictionary is actually three separate values currently created as follows:

invoiceDict.Add(pdfCount+i, new StringBuilder(invoiceID.Groups[1].ToString() + "|" + extractFileName + "|" + pdfPair.Key));

As you can see, the three values are separated by a '|'. The number of "rows" in the Dictionary can range between 600-1200. I would like to use a table-valued parameter to get all of that in my SQL Server 2008 DB in one operation.

The table-valued parameter is defined as follows:

CREATE TYPE dbo.BatchSplitterInvoices AS TABLE
(
    InvoiceID varchar(24),
    NotePath varchar(512),
    BatchID varchar(50)
)

CREATE PROCEDURE dbo.cms_createBatchSplitterInvoices (
  @Invoices dbo.BatchSplitterInvoices READONLY,
  @StaffID int
)

What's the best way to get from the Dictionary to something that can be passed into the table-valued param? Should I use something else than the Dictionary? Or, do I need to parse the Dictionary into another, better data structure?

SqlParameter invoicesParam = cmd.Parameters.AddWithValue("@Invoices", invoiceDict.Values);
invoicesParam.SqlDbType = SqlDbType.Structured;

Thanks.

Answer 1

Due to issue with somewhat poor documentation surrounding the TVPs (needs more than just IEnumerable), and having to parse the content of dictionaries anyways, I decided to loop through the Dictionary into a DataTable, which seems to be the preferred object to feed a TVP.

Answer 2

Both @narayanamarthi and @ChrisGessler are on the right track with regards to using the IEnumerable<SqlDataRecord> interface instead of a DataTable. Copying the collection to a DataTable is just wasting CPU and memory for no gain. But the iterator can be separate from the collection.

So some notes:

• Don't use AddWithValue when creating SqlParameters. It is just a bad pratice
• Don't concatenate the 3 distinct values that you want in the TVP into a String or StringBuilder. The entire purpose of a TVP is to pass in a strongly-typed record so serializing the fields into a CSV list defeats the purpose. Instead use the Invoice class as recommended by Chris.
• Replace your invoiceDict.Add(...) with List<Invoice>.Add(new Invoice(...));

• Create a method to iterate over the collection:

  private static IEnumerable<SqlDataRecord> SendRows(List<Invoice> Invoices)
  {
    SqlMetaData[] _TvpSchema = new SqlMetaData[] {
      new SqlMetaData("InvoiceID", SqlDbType.Int),
      new SqlMetaData("NotePath", SqlDbType.VarChar, 512),
      new SqlMetaData("BatchID", SqlDbType.VarChar, 50)
    };
    SqlDataRecord _DataRecord = new SqlDataRecord(_TvpSchema);
  
    foreach(Invoice _Invoice in Invoices)
    {
      _DataRecord.SetInt32(0, _Invoice.Id);
      _DataRecord.SetString(1, _Invoice.NotePath);
      _DataRecord.SetString(2, _Invoice.BatchId);
      yield return _DataRecord;
    }
  }
  

• Declare the parameter as follows:

  SqlParameter _InvoiceParam = cmd.Parameters.Add("@Invoices", SqlDbType.Structured);
  _InvoiceParam.Value = SendRows(Invoices);
  

---

I have additional notes and links in the following two answers with regards to TVPs in general:

• How can I insert 10 million records in the shortest time possible?
• Pass Dictionary<string,int> to Stored Procedure T-SQL

Answer 3

SqlBulkCopy will send it in one operation, but you will need to pass it a DataTable or an IDataReader instead of a Dictionary<int, StringBuilder> and you will also need to point directly to the table, instead of using a stored procedure.

Answer 4

Implementing the IEnumerable interface on the class holding your collection and then explicitly implementing the GetEnumerator Method for the IEnemerable will serve the purpose. There is an article at http://lennilobel.wordpress.com/2009/07/29/sql-server-2008-table-valued-parameters-and-c-custom-iterators-a-match-made-in-heaven/. Please go through this

Answer 5

You need to implement IEnumerable<SqlDataRecord> on a custom collection object. In your case, you should change your Dictionary<int, StringBuilder> to List<Invoice> or List<Tuple<int, string, int>>

For example:

class Invoice
{
  public int Id { get; set; }
  public string NotePath { get; set; }
  public int BatchId { get; set; }
}

class InvoiceCollection : List<Invoice>, IEnumerable<SqlDataRecord>
{
  IEnumerator<SqlDataRecord> IEnumerable<SqlDataRecord>.GetEnumerator()
  {
    SqlDataRecord r - new SqlDataRecord(
      new SqlMetaData("InvoiceId", SqlDataType.Int), 
      new SqlMetaData("NotePath", SqlDataType.VarChar), 
      new SqlMetaData("BatchId", SqlDataType.Int)
    );

    foreach(var item in this)
    {
      r.SetInt32(0, item.Id);
      r.SetString(1, item.NotePath);
      r.SetInt32(2, item.BatchId);
      yield return r;
    }
}

Then simply pass the custom list to a SqlParameter:

SqlCommand cmd = new SqlCommand("dbo.InvoiceInsUpd", connection);
cmd.CommandType = CommandType.StoredProcedure;

SqlParameter sqlParam = cmd.Parameters.AddWithValue("@Invoices", invoices);
sqlParam.SqlDbType = SqlDbType.Structured;