Sql server bulk delete by known record ids

Question

I need to delete many rows from sql server 2008 database, it must be scalable so I was thinking about bulk delete, the problem is that the are not many references on this, at least in my case.

The first factor is that I will exactly know the ID of every row to delete, so any tips with TOP are not an option, also I will delete less rows that I want to retain so there will be no need for some "drop/temp table/re-create" methods.

So I was thinking to use WHERE IN , either suppling IDs or xml data with IDs, there is also an option to use MERGE to delete rows.

If I will have to delete 1000+ rows, sending all ids to WHERE IN could be a problem ? And what with MERGE - it is really a cure for all bulk insert/update/delete problems? What to choose?

Answer 1

One option would be to store the known ID's into a "controller" table, and then delete rows from your main data table that have their ID show up in the controller table.

That way, you could easily "batch" up your deletes, e.g.

DELETE FROM dbo.YourMainDataTable
WHERE ID IN (SELECT TOP (250) ID FROM dbo.DeleteControllerTable)

You could easily run this delete statement in e.g. a SQL Agent Job which comes around every 15 minutes to check if there's anything to delete. In the meantime, you could add more ID's to your DeleteController table, thus you could "decouple" the process of entering the ID's to be deleted from the actual deletion process.