Oracle Data Pump export includes incorrect sequences

Question

Recently, we took a live/online backup of our production database, by issuing the following command:

expdp system/******@SID FULL=y DIRECTORY=data_pump_dir DUMPFILE=full_prod.dmp LOGFILE=full_prod_export.log JOB_NAME=prod_backup

There were no errors nor warnings.

We then took this dump and created a number of developer databases from it, by issuing:

impdp system/******@SID SCHEMAS=MY_SCHEMA DIRECTORY=data_pump_dir DUMPFILE=full_prod.dmp LOGFILE=full_prod_import.log

Again, no errors nor warnings.

Once the developer environments were set-up, the developers launched their applications and tried to insert some test data. The first few attempts failed with primary key violations. After a bit of digging, it turns out that the cached sequences (NOCACHE sequences were fine) were generally behind by one or two values.

An example is the sequence for our contact table:

CREATE SEQUENCE REQ_CONTACT_SEQ
START WITH 213041
MAXVALUE 999999999999999999999999999
MINVALUE 1
NOCYCLE
CACHE 20
NOORDER;

Oracle shows a last cached value of 213041 for this sequence. Now, when I run the following query:

SELECT MAX(id) FROM REQ_CONTACT;

Oracle returns 213042. It's obvious that this value is 1 higher than the sequence.

So.... My question is, did we only get this result because we took the Data Export while production was live - ie there were open and active connections to the database? Or does Data Pump have a problem with sequences that are cached? We are using Oracle 10.2.0.4.

Thanks, Muel.

Answer 1

I believe you are correct in your assumption that this occurs because the environment is live and has open connections. I have been doing the same process for our dev machines for the last few years, using Oracle 10.2.0.4, and have had to create a script that I run on imports of Production exports. I basically increment each of the problem sequences by 100, and then set the increment value back down to 1.

alter sequence [sequence_name] increment by 100;
select [sequence_name].nextval from dual;
alter sequence [sequence_name] increment by 1;
select [sequence_name].nextval from dual;
commit;

In our case, we see Primary Key constraint errors. Over time, I have added each of the sequences that has shown a PK constraint error to my script.

Every once in a while, I'll get caught off guard by a new PK constraint error, and will have to adjust the script and add the new sequence.

I'm considering creating a variation that increments every sequence, which would prevent any new PK constraint errors from cropping up. This is off the top of my head, but I think it would have be some sort of LOOP DBMS statement that loops through each value in the query "select sequence_name from user_sequences."

I've always found it odd that Oracle doesn't have a way to correct this, nor have I found an "easy" solution from anyone on the net. It would be some script that queried each max(id) and compared it to the current sequence value, and increment it the difference.

I've also heard of a way to run expdp (export datapump) with some sort of "state" variable, that causes it to maintain the state of things throughout the export. I'll update the post if I find anything. In the meantime, good luck!

EDIT: The parameter to add to expdp to maintain data across the export is "consistent = y" Apparently, it only maintains data for each table separately, so I'm not sure if the sequence values would be consistent.

Oracle datapump details

Answer 2

The best solution to this, is to add the flashback parameter to the export. This will keep all your data consistent from the moment you start the export.

FLASHBACK_TIME=SYSTIMESTAMP

Answer 3

You can also simply drop cascade the schemas that have sequences. I do that and set FLASHBACK_TIME or FLASHBACK_SCN

Note: consistent = y is for exp not expdp

Also note: If you are on 12.1 you might get errors related to this if you set a "FLASHBACK_x" parameter for a full export --as sysdba to work around bug 19238926 Doc ID 1676411.1

grant flashback any table to exp_full_database;
grant flashback on SYS.KU$_USER_MAPPING_VIEW to exp_full_database;
grant flashback on SYS.FGA_LOG$FOR_EXPORT to exp_full_database;
grant flashback on SYS.AUDTAB$TBS$FOR_EXPORT to exp_full_database;
grant flashback on SYS.DBA_SENSITIVE_DATA to exp_full_database;
grant flashback on SYS.DBA_TSDP_POLICY_PROTECTION to exp_full_database;
grant flashback on SYS.NACL$_ACE_EXP to exp_full_database;
grant flashback on SYS.NACL$_HOST_EXP to exp_full_database;
grant flashback on SYS.NACL$_WALLET_EXP to exp_full_database;