Database redesign then reload approach using linq to excel and entity framework

Question

So I've got a few tables with a lot of data. Let's say they are table A, B and C. I want to add auto increment ID fields for each of those tables, normalize them by swapping some fields between the tables and add an additional table D. Gulp. There are three goals: 1) redesign the database, and reload the existing data. 2) Enable a data load from a spreadsheet to add/edit/delete the four tables. 3) Enable a web front end to add/edit/delete the four tables.

My current approach:

• I thought I would export a flat file for all the data in the 3 existing tables into a csv (spreadsheet).
• Then refactor the database design structure
• Then use linq to excel to read back the csv spreadsheet records into dto objects
• Then use the Entity Framework to transform those dto objects into Entities to update the database with the appropriate relationships between tables
• The spreadsheet would be re-used for future bulk data add/edit/deletes

What about the following tools? SSIS Bulk insert Stored procedures

Am I over complicating this? Is there a better way?

Answer 1

What's your definition of "a lot of data"? For some people it's 10,000 rows, for others it's billions of rows.

I would think that a few stored procedures should be able to do the trick, mostly made up of simple INSERT..SELECT statements. Use sp_rename to rename the existing tables, create your new tables, then move the data over.

If you already have to develop a bulk import process then it might make sense to get reuse out of that by doing an export, but I wouldn't create that whole process just for this scenario.

There might be cases where this approach isn't the best, but I don't see anything in your question that makes me think it would be a problem.

Make sure that you have a good back-up first of course.