What is the recommended way to normalize data during an import from CSV/Excel in ASP.NET (MVC)?

Question

I know there are a thousand data import related questions on Stack Overflow, and please accept my apologies if this has already been asked somewhere, but I wondered if anybody could advise of any tried and tested solutions for normalisaing data during an import from CSV/Excel in C#/ASP.NET MVC 3+.

I could code something to do the job but wondered if there were any open source libraries or tools which could help out with this.

My area of interest is as follows:

When importing data, I occasionally need to normalise some fields, a simplistic example of this is shown below:

My input may be:

Name, JobTitle
==============
Nick, Manager
Dan,  Coder

My table structure may be

Name, JobTitleId
================
Nick, 1
Dan,  2

If a job title doesn't exist, I'd like it to be created in my JobTitles table and the Id to be returned. If it does exist, I'd like to store the ID.

In sumamry, my questions are:

• Is there a technique or approach I should be taking to do this?
• Are there any open source/commercial libraries out there which handle this - no point in re-inventing the wheel

Answer 1

Is there a technique or approach I should be taking to do this?
The simplest technique that I can think of is a non-MVC way: use SSIS! You can have ASP.NET as the middle man to accept the Excel file and pass it to the database if security is a concern. There are many tutorials out there using Excel Connection Manager. e.g., Import Excel File. SSIS gives you incredible flexibity which might help you normalize your data.

Are there any open source/commercial libraries out there which handle this - no point in re-inventing the wheel
As you might already have invested in Microsoft products like Visual Studio and SQL Server, you might as well leverage them to suit your needs than trying out open source products out there.

Sigh!!
Besides that, if you really wanna check out tools, then I have heard people recommend these ones (not used any of them):

1. RelationalExcel but it is not free.
2. ExpertXLS Excel Library for .NET - this one is not free but works with ASP.NET.
3. This blog seems like a good reference to tools that he has tried.

Other alternatives could be:

1. Load your Excel using .NET into datasets and generate your SQL queries dynamically OR pass this to an SQL Server sp that does the heavy lifting
2. Parse your Excel using VBA and then use it as a data source to ASP.NET