Ways to improve this SQL query

Question

I have this query

SELECT "items".*
FROM
    "items"
    INNER JOIN
    item_mods ON item_mods.item_id = items.id AND item_mods.mod_id = 15
WHERE (items.league_id = 1) AND (items.item_type_id = 11) AND (num_sockets >= 2)
ORDER BY item_mods.total_value DESC
LIMIT 25

This is the explain http://explain.depesz.com/s/dbf

"Limit  (cost=55739.84..55739.90 rows=25 width=554) (actual time=18065.470..18065.478 rows=25 loops=1)"
"  ->  Sort  (cost=55739.84..55741.90 rows=824 width=554) (actual time=18065.468..18065.471 rows=25 loops=1)"
"        Sort Key: item_mods.total_value"
"        Sort Method: top-N heapsort  Memory: 37kB"
"        ->  Nested Loop  (cost=5871.95..55716.59 rows=824 width=554) (actual time=285.806..18055.589 rows=610 loops=1)"
"              ->  Bitmap Heap Scan on items  (cost=5871.52..20356.70 rows=4339 width=550) (actual time=201.543..10028.684 rows=9945 loops=1)"
"                    Recheck Cond: ((item_type_id = 11) AND (num_sockets >= 2))"
"                    Rows Removed by Index Recheck: 4120"
"                    Filter: (league_id = 1)"
"                    Rows Removed by Filter: 1125"
"                    ->  BitmapAnd  (cost=5871.52..5871.52 rows=4808 width=0) (actual time=199.322..199.322 rows=0 loops=1)"
"                          ->  Bitmap Index Scan on index_items_on_item_type_id  (cost=0.00..289.61 rows=15625 width=0) (actual time=38.699..38.699 rows=16018 loops=1)"
"                                Index Cond: (item_type_id = 11)"
"                          ->  Bitmap Index Scan on index_items_on_num_sockets  (cost=0.00..5579.49 rows=301742 width=0) (actual time=158.441..158.441 rows=301342 loops=1)"
"                                Index Cond: (num_sockets >= 2)"
"              ->  Index Scan using index_item_mods_on_item_id on item_mods  (cost=0.43..8.14 rows=1 width=8) (actual time=0.803..0.803 rows=0 loops=9945)"
"                    Index Cond: (item_id = items.id)"
"                    Filter: (mod_id = 15)"
"                    Rows Removed by Filter: 9"
"Total runtime: 18065.773 ms"

How can I improve the speed of this query? I note there is a loop > 9000 times on the index scan

Answer 1

The reason your query is slow is because there are no indexes for the way you want to return the data.

Notice the "Bitmap Index Scan" where it is saying, I know you have an index but I have to look at the whole table to find the row I need (hence total row scan of up to 301742!). This is probably because of the combination of other columns you have asked for and the constraints you are applying i.e. item_mods.mod_id = 15

Try:

1. "items".* - Only select out the columns you need rather than everything.

2. Create an index on: item_mods.item_id AND item_mods.mod_id

3. Create an index on: items.league_id AND items.item_type_id AND num_sockets (assuming num_sockets is on the same table)

Any performance difference?

Answer 2

This only shortens and cleans the syntax, but doesn't change anything substantially:

SELECT i.*
FROM   items     i
JOIN   item_mods m ON m.item_id = i.id
WHERE  i.league_id = 1
AND    i.item_type_id = 11
AND    i.num_sockets >= 2
AND    m.mod_id = 15
ORDER  BY m.total_value DESC
LIMIT  25;

A query like this is very hard to optimize. Postgres cannot just read from the top of the index. since you are sorting by a column in item_mods, but the most selective conditions are on items, it is also hard to tailor an index that would help some more.

Of course you can optimize indexes on either table. But without additional information to feed to the query, it won't get cheap. All qualifying rows have to be read before Postgres knows the winners.

We have developed solutions under this related question on dba.SE. Sophisticated stuff:
Can spatial index help a “range - order by - limit” query

Answer 3

Below query should give a better performance as filtration is done before joining.

   SELECT t.* 
   FROM
   (
     SELECT items.* FROM items
     WHERE  (items.league_id = 1) AND (items.item_type_id = 11) 
   ) t 
   INNER JOIN 
   (
      SELECT item_mods.*
      FROM item_mods
      WHERE item_mods.mod_id = 15 
   ) s
   ON s.item_id = t.id 
   WHERE (num_sockets >= 2)
   ORDER BY item_mods.total_value DESC
   LIMIT 25

num_sockets>=2 can also be included in some inner query if it is known which table it belongs to.

Please let me know if it performed better or not.