Import Excel file to SQL Server row-by-row

StackOverflow https://stackoverflow.com/questions/4034470

Question

I'm importing an Excel file (about 1000 records only) to a dedicated SQL Server database. As I need to work on the incoming data from Excel (add a GUID per row, some data conversions) I want to do it row by row and don't want to bulk import (I have nothing against transactions, though).

I'm confused on how to do it correctly. I can either use an SQLCommand with parameters like so:

SqlCommand sqlCommand = new SqlCommand("insert into TestTable(GUID,Name,Pricing) values(@GUID,@Name,@Pricing)", sqlConn);
foreach (DataRow dr in ds.Tables[0].Rows) //<-- this is my Excel file to iterate through
{
 sqlCommander.Parameters.Clear();
 String refGUID = Guid.NewGuid().ToString();
 sqlCommander.Parameters.AddWithValue("GUID", refGUID);
 sqlCommander.Parameters.AddWithValue("Name", dr.ItemArray[0]);
 sqlCommander.Parameters.AddWithValue("Pricing", dr.ItemArray[1]);
 sqlCommander.ExecuteNonQuery();
}

Or I can use the "connected" mode like so:

SqlDataAdapter dataAdapter = new SqlDataAdapter("SELECT GUID, Name, Pricing FROM TestTable", sqlConn);
SqlCommandBuilder commandBuilder = new SqlCommandBuilder(dataAdapter);

DataSet myDataSet = new DataSet();
dataAdapter.Fill(myDataSet, "TestTable");
foreach (DataRow dr in ds.Tables[0].Rows) //<-- this is my Excel file to iterate through
{
 DataRow row = myDataSet.Tables[0].NewRow();
 row["GUID"] = refGUID;
 row["Name"] = dr.ItemArray[0];
 row["Pricing"] = dr.ItemArray[1];
 myDataSet.Tables[0].Rows.Add(row);
 dataAdapter.Update(myDataSet);
}

Now my questions are the following:

  1. Is it better to send an INSERT command for each row (that would be the SqlCommand method) or is it better to fill a special DataSet (2nd Method)? I guess it is rather "stupid" to have 1000's of inserts to an SQL server?
  2. dataAdapter.Update(myDataSet) <-- should I do that AFTER iterating through all Excel rows or for each row (as shown in the example code above), would this magically create a transaction?
  3. Which method should I use? There also is LINQ to SQL - why not use that (performance maybe, because of another layer)?
  4. What happens to the DataSet when some error occurs when reading out the Excel file - is the update still pushed to the SQL server or is everything lost?

In short: I want to import an Excel file to an SQL server row-by-row whilst making changes on the data to be imported (and I don't want to use SSIS packages [because besides data conversion I am doing a lot more with the Excel file like importing it to Sharepoint and firing of Workflows] or BizTalk)
»» How to do that beautifully?
In the end I went ahead and bought Aspose Cells. Aspose has a very nice set of tools at their disposal.

Was it helpful?

Solution

You mention not wanting to use SSIS - but have you considered SqlBulkCopy? Then there is no need for anything except .NET yet you can still use the fastest/most-direct import.

This will accept a DataTable, so you can prepare your data in a DataTable and then pull the trigger. Transactions are optionally supported IIRC. For larger data you can also implement IDataReader to provide fully streaming upload (while still processing each row in transit).

OTHER TIPS

Given your description of your problem: "I want to import an Excel file to an SQL server row-by-row whilst making changes on the data to be imported" - SSIS is the perfect tool for the job.

Importing Excel data with SQL Server Integration Services SSIS with unicode and non-unicode data issues

The dataSet will probably ending up sending the INSERT statements to the server, so in my opinion, is better to to just send the INSERT statements without the DataSet. You also can have more control over the process, like checking for individual rows for errors, logging, etc.

You could convert your processed data table into XML and pass that to a stored procedure in Sql server (in one query) and have your stored procedure parse the XML to create the records.

INSERT INTO [dbo].[TableName]
           ([ColumnName1]
           ,[ColumnName2])
)
SELECT [ColumnName1]
           ,[ColumnName2]

FROM OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0','Data Source= PathToFile.xls;Extended Properties=Excel 8.0')...[Sheet1$]
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top