How do I improve the database performance when going from VBA to SQL Server 2008 R2?

Question

I am working on a financial application that performs CRUD operations from Excel to SQL Server 2008 R2. The application uses VBA and ADO. I am trying to optimize the performance of an operation that loops through thousands of rows and sends updates to the database. Currently, a stored procedure gets called once for each row.

When users have slow connections to the application the performance is slow too, enough that they have started complaining. To optimize the performance, I tried:

1. Batching all of the operations into an XML file and sending them all at once. Unfortunately the performance got worse.
2. Next I thought of batching all of the updates in a table-valued parameter. However, VBA and ADO can’t use table-valued parameters.

I am running out of things to try. How do I improve the database performance when going from VBA to SQL Server 2008 R2?

Answer 1

I figured it out. While there is no table-valued parameter (TVP) object available in ADO, you can execute raw SQL statements on a SQL Server connection. The solution is to:

1. Build a SQL statement that uses a TVP
2. Execute the SQL statement on the ADO connection.

For example:

Dim cnSQLServer As ADODB.Connection
Dim rst As ADODB.Recordset
Dim strSQL As String

Set cnSQLServer = New ADODB.Connection
cnSQLServer.Open "Provider=SQLOLEDB;Data Source=localhost;Initial Catalog=Northwind;Integrated Security = SSPI"

' Build a SQL statement that uses a TVP
strSQL = "DECLARE @ReturnMessage nvarchar(255) " + _
    "DECLARE @catType CategoryTableType " + _
    "INSERT INTO @catType" + _
    "EXEC ('" + _
    "SELECT ''Organic'', ''Organically grown produce and grains'' " + _
    "SELECT ''Conventional'', ''Non-organically grown produce and grains'' " + _
    "SELECT ''Irish'', ''Mrs. O''''Leary''''s creamery products'' " + _
    "') " + _
    "EXEC dbo.usp_InsertCategory @catType, @ReturnMessage OUTPUT; " + _
    "SELECT @ReturnMessage as ReturnMessage'"

' Execute the SQL statement on the ADO connection.
Set rst = cnSQLServer.Execute(strSQL)

The solution is described in more detail in an article about calling TVPs from MS Access: Using SQL Server 2008 Table-valued Parameters in Access 2007

Using a TVP gave me a nice performance boost. Hope this helps someone else. Cheers.

Answer 2

To be honest VBA/ADO speed is pretty craptacular (in comparison to web sites) in my experience. There is arcane overhead in this method of database connection especially when executing SQL statements, and not a lot to help to troubleshoot it.

One avenue to try is to update recordset objects rather than executing stored procedures, I have found this to help in some cases.

That said depending on the complexity of your updates, that may just make things worse.