Question

Each time i want to process 5000 records like below.

First time i want to process records from 1 to 5000 rows. second time i want to process records from 5001 to 10000 rows. third time i want to process records from 10001 to 15001 rows like wise

I dont want to go for procedure or PL/SQL. I will change the rnum values in my code to fetch the 5000 records.

The given query is taking 3 minutes to fetch the records from 3 joined tables. How can i reduced the time to fetch the records.

select * from  (
SELECT to_number(AA.MARK_ID) as MARK_ID, AA.SUPP_ID as supplier_id, CC.supp_nm as SUPPLIER_NAME, CC.supp_typ as supplier_type, 
CC.supp_lock_typ as supplier_lock_type, ROW_NUMBER() OVER (ORDER BY AA.MARK_ID) as rnum 
from TABLE_A AA, TABLE_B BB, TABLE_C CC  
WHERE 
AA.MARK_ID=BB.MARK_ID AND 
AA.SUPP_ID=CC.location_id  AND 
AA.char_id='160' AND  
BB.VALUE_KEY=AA.VALUE_KEY AND 
BB.VALUE_KEY=CC.VALUE_KEY
AND AA.VPR_ID IS NOT NULL) 
where rnum >=10001  and rnum<=15000;

I have tried below scenario but no luck.

I have tried the /*+ USE_NL(AA BB) */ hints. I used exists in the where conditions. but its taking the same 3 minutes to fetch the records.

Below is the table details.

select count(*) from TABLE_B;
-----------------
2275

select count(*) from TABLE_A;
-----------------
2405276

select count(*) from TABLE_C;
-----------------
1269767

Result of my inner query total records is

SELECT count(*) 
from TABLE_A AA, TABLE_B BB, TABLE_C CC  
WHERE 
AA.MARK_ID=BB.MARK_ID AND 
AA.SUPP_ID=CC.location_id  AND 
AA.char_id='160' AND  
BB.VALUE_KEY=AA.VALUE_KEY AND 
BB.VALUE_KEY=CC.VALUE_KEY
AND AA.VPR_ID IS NOT NULL;
-----------------
2027055

All the used columns in where conditions are indexed properly.

Explain Table for the given query is...

Plan hash value: 3726328503

-------------------------------------------------------------------------------------------------------
| Id  | Operation                  | Name             | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT           |                  |  2082K|   182M|       | 85175   (1)| 00:17:03 |
|*  1 |  VIEW                      |                  |  2082K|   182M|       | 85175   (1)| 00:17:03 |
|*  2 |   WINDOW SORT PUSHED RANK  |                  |  2082K|   166M|   200M| 85175   (1)| 00:17:03 |
|*  3 |    HASH JOIN               |                  |  2082K|   166M|       | 44550   (1)| 00:08:55 |
|   4 |     TABLE ACCESS FULL      | TABLE_C          |  1640 | 49200 |       |    22   (0)| 00:00:01 |
|*  5 |     HASH JOIN              |                  |  2082K|   107M|    27M| 44516   (1)| 00:08:55 |
|*  6 |      VIEW                  | index$_join$_005 |  1274K|    13M|       |  9790   (1)| 00:01:58 |
|*  7 |       HASH JOIN            |                  |       |       |       |            |          |
|   8 |        INLIST ITERATOR     |                  |       |       |       |            |          |
|*  9 |         INDEX RANGE SCAN   | TABLE_B_IN2      |  1274K|    13M|       |  2371   (2)| 00:00:29 |
|  10 |        INDEX FAST FULL SCAN| TABLE_B_IU1      |  1274K|    13M|       |  4801   (1)| 00:00:58 |
|* 11 |      TABLE ACCESS FULL     | TABLE_A          |  2356K|    96M|       | 27174   (1)| 00:05:27 |
-------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("RNUM">=10001 AND "RNUM"<=15000)
   2 - filter(ROW_NUMBER() OVER ( ORDER BY "A"."MARK_ID")<=15000)
   3 - access("A"."SUPP_ID"="C"."LOC_ID" AND "A"."VALUE_KEY"="C"."VALUE_KEY")
   5 - access("A"."MARK_ID"="A"."MARK_ID" AND "A"."VALUE_KEY"="A"."VALUE_KEY")
   6 - filter("A"."MARK_CHN_IND"='C' OR "A"."MARK_CHN_IND"='D')
   7 - access(ROWID=ROWID)
   9 - access("A"."MARK_CHN_IND"='C' OR "A"."MARK_CHN_IND"='D')
  11 - filter("A"."CHNL_ID"=160 AND "A"."VPR_ID" IS NOT NULL)

Could you please anyone help me on this to tune this query as i am trying from last 2 days?

Was it helpful?

Solution

Each query will take a long time because each query will have to join then sort all rows. The row_number analytic function can only return a result if the whole set has been read. This is highly inefficient. If the data set is large, you only want to sort and hash-join once.

You should fetch the whole set once, using batches of 5k rows. Alternatively, if you want to keep your existing code logic, you could store the result in a temporary table, for instance:

CREATE TABLE TMP AS <your above query>
CREATE INDEX ON TMP (rnum)

And then replace your query in your code by

SELECT * FROM TMP WHERE rnum BETWEEN :x AND :y

Obviously if your temp table is being reused periodically, just create it once and delete when done (or use a true temporary table).

OTHER TIPS

How many unique MARK_ID values have you got in TABLE_A? I think you may get better performance if you limit the fetched ranges of records by MARK_ID instead of the artificial row number, because the latter is obviously not sargeable. Granted, you may not get exactly 5000 rows in each range but I have a feeling it's not as important as the query performance.

Firstly, giving obfuscated table names makes it nearly impossible to deduce anything about the data distributions and relationships between tables, so potential answerers are crippled from the start.

However, if every row in table_a matches one row in the other tables then you can avoid some of the usage of 200Mb of temporary disk space that is probably crippling performance by pushing the ranking down into an inline view or common table expression.

Monitor V$SQL_WORKAREA to check the exact amount of space being used for the window function, and if it is still excessive consider modifying the memory management to increase available sort area size.

Something like:

with cte_table_a as (
  SELECT
    to_number(MARK_ID) as MARK_ID,
    SUPP_ID as supplier_id,
    ROW_NUMBER() OVER (ORDER BY MARK_ID) as rnum 
  from
    TABLE_A
  where
    char_id='160' and
    VPR_ID IS NOT NULL)
select ...
from
  cte_table_a aa,
  TABLE_B BB,
  TABLE_C CC  
WHERE 
  aa.rnum      >= 10001          and
  aa.rnum      <= 15000          and
  AA.MARK_ID   =  BB.MARK_ID     AND 
  AA.SUPP_ID   =  CC.location_id AND 
  BB.VALUE_KEY =  AA.VALUE_KEY   AND 
  BB.VALUE_KEY =  CC.VALUE_KEY
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top