Question

I am using oracle database and what i do is

  1. Taking 1 record of table A. (table A has column P and lets say values of it are x,y,z)

  2. Putting that record to table B or C or D according to values x,y,z (if P=x then put record to table B , if P=y then put record into table C ...)

  3. Delete that record of A which we inserted to table B or C or D.

Note: size of A is like 200 million, B is 170 C is 20 D is 10 so and size of A is decreasing others are same (if a parameter of A record is negative then it is not inserted into to B,C,D it is exist in these tables so just deleted it from table) so there is no size change for B,C,D just size of A decreasing in time.

The problem is at the beginning everything is working nice, but in time, its becoming extremely slow. Approximately it is making 40 insert+delete in 1 second but in time its processing 1 insert+delete in 3 second.

  • All tables have index in corresponding columns.

  • Paralel run exists but there is no lock.

  • Table sizes are approximately 60 million record.

What other effects can make it - in time - if there is no lock or size increase for table??

note: it is not different processes, in same process i click "execute query" it is starting very fast but then extremely slow.

Was it helpful?

Solution

Inserting 200 million records from a staging table and inserting them into permanent tables in a single transaction is ambitious. It would be a useful if you had a scheme for dividing the records from table A into chunks which could be processed in discrete chunks.

Without seeing your code it's hard to tell but I have a suspicion you are attempting this RBAR rather than a more efficient set-based approach. I think the key here is to de-couple the insertions from clearing down table A. Insert all the records, than zap A at your leisure. Something like this

insert all
    when p = 'X' then into b
    when p = 'Y' then into c
    when p = 'Z' then into d
select * from a;

truncate table a;
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top