Incremental extraction from DB2

Question

What would be the most efficient way to select only rows from DB2 table that are inserted/updated since the last select (or some specified time)? There is no field in the table that would allow us to do this easily. We are extracting data from the table for purposes of reporting, and now we have to extract the whole table every time, which is causing big performance issues. I found example on how to select only rows changed in last day:

SELECT * FROM ORDERS 
  WHERE ROW CHANGE TIMESTAMP FOR ORDERS > 
    CURRENT TIMESTAMP - 24 HOURS;

But, I am not sure how efficient this would be, since the table is enormous. Is there some other way to select only rows that are changed, that might be more efficient that this?

I also found solution called ParStream. This seems as something that can speed up demanding queries on the data, but I was unable to find any useful documentation about it.

Answer 1

I propose these options:

• You can use Change Data Capture, and this will replay automatically the modifications to another data source.
• Normally, a select statement does not assure the order of the rows. That means that you cannot use a select without a time reference in order to retrieve the most recent. Thus, you have to have a time column in order to retrieve the most recent. You can keep track of the most recent row in a global variable, and the next time retrieve the rows with a time bigger than that variable. If you want to increase performance, you can put the table in append mode, and in this way the new rows will be physically together. Keeping an index on this time column could be expensive to maintain, but it will speed (no table scan) when you need to extract the rows.

Answer 2

If your server is DB2 for i, use database journaling. You can extract after images of inserted records by time period or journal entry number from the journal receiver(s). The data entries can then be copied to your target file.