SSIS Data Flow performance slow vs select into

Question

so I have a script:

select * 
into my_archive_05302013
from archive_A
where my_Date = '05/18/2013'

and:

insert into archive_B (ID,my_date,field_n )
select ID,  my_Date,field_n from my_archive_05302013

where the n in field_n is about 100 or so. so in other words there are more than 100 columns in the table that I am loading.

which run pretty fast the query inserts about 200000 records. my_date is a non-clustered index in table archive_A

Now when I create a dataflow using SSIS 2008 it takes HOURS to complete

I have the following in my OLE DB source:

SELECT * FROM Archive_A
WHERE My_Date = (SELECT MAX(My_Date) from Archive_A)

and for OLE DB Destination:

Data access mode of: "Table or view - fast load" Name of the table: archive_B Table lock and Check constraints is checked

Any one know what the problem could be?

Thanks in advance

Answer 1

The problem is that because you are using a data source and a data destination what you are doing is pulling all of the data out of the database to then put it all back in again whereas your INSERT statement keeps it all contained within the database. Use an Excute SQL Task with your INSERT statement instead.