How to import to SQL Server 2005 from flat file with data transformations

Question

I have a flat data file that I need to import into my SQL Server 2005 DB.

Many of the fields need to be split off into different, related tables. For example, the flat file has names, addresses and telephone numbers, all in one record. In my DB, the Person table has many Telephones and Addresses.

Is there a one-step process whereby I can import everything into my tables, or do I have to first import it into a new table in my DB (ugh - pollution if I forget to delete it), and import the data from there using SQL statements and temp tables?

Answer 1

I prefer the one import table, followed by splitting out into final tables. I'd also persist the import table rather than creating/deleting it every time.

• Easier to deal wth constraints (check before insert into final table or update existing row)
• Easier to leave error generating data in the import table after removing successful rows
• Server side transaction
• Data type safety: can you 100% trust your source?
• Easier to ISNULL or NULLIF in SQL to deal with empty strings and other such
• and other things that I can't recall right now...

Answer 2

This is totally a job for SQL Server Integration Services. It has some great functionality that will allow you to grab a flat file, do data manipulation on it, and eventually import it into your new db.

Unfortunately, there isn't an easy "quick fix" solution that I know of outside of that. There is the technology I would look into first, however.