Mark record inserted

Question

I have table, which I am inserting records to another table. What is the best way to mark record inserted, so it will not be attempted to being inserted again?

Answer 1

You can see the difference between the two tables like so:

SELECT * FROM tableFoo
LEFT JOIN tableBar ON tableFoo.commonColumn = tableBar.commonColumn
WHERE tableBar.commonColumn IS NULL

The idea is that both tables have a column to match up on, and the records that are joined when the column is null are the records that are only present in tableBar.

The reason this works is because left joins will returns records even if one of the tables has null values, unlike an inner join, which does the opposite.

After you get those records, you can insert based on the IDs returned.

Answer 2

insert only records not in your other table, either using a NOT EXISTS clause or by left joining and filtering all the not-NULL records from the result

Answer 3

The best way is to use a common key (or have the key of the first table be the leading part of the second table's key). That way, you simply select rows from TABLE1 where NOT EXISTS in table 2.

The best alternative, if you need to transform the key in some way, is to use an insert trigger on TABLE1: when you insert rows there, the trigger will fire, and you can insert data into TABLE2. This has the benefit -- and also the drawback -- of using a single transaction. It's a benefit in that you retain data consistency, a drawback if TABLE2 is used for reporting or other non-essential purposes.

DO NOT decide to use a flag in TABLE1 that indicates the row has been inserted. This is ugly from both a logical and physical design, because you're coupling data in the table to a process that uses the table.

Answer 4

There are two methods that I use frequently. Depending on the nature of the table, one way may be better than another, however if you go with #2, there are likely opportunities to make your table structure better.

1.) Make sure your tables are indexed and have a good primary key. Select all the records from table1 where the primary key does not exists in table2. This works will for normalized tables

2.) If your tables are not normalized and do not have good have good keys, you can add a ProcessedDate to table1. Insert all records that have a null ProcessDate then set the ProcessDate to the current datetime. You will just have to make sure that that no new records were inserted into table1 during the time you were inserting into table2.

Without knowing your table structure it is difficult to give a good answer.