Simple Postgres queries running slow on Heroku

Question

I have a Movie model with around 15,000 items and a Dvd model with 3500.

The following queries are simple Rails associations using a Crane Postgres db running on Heroku. I'm wondering why the following queries are taking so long, and how could I eventually reduce the time of it.

2012-10-24T05:42:19+00:00 app[postgres]: [30-1] [BLACK] LOG:  duration: 57.914 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 37 ORDER BY scene LIMIT 1
2012-10-24T05:42:20+00:00 app[postgres]: [31-1] [BLACK] LOG:  duration: 77.086 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 915 ORDER BY scene LIMIT 1
2012-10-24T05:42:20+00:00 app[postgres]: [32-1] [BLACK] LOG:  duration: 85.602 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 108 ORDER BY scene LIMIT 1
2012-10-24T05:42:21+00:00 app[postgres]: [33-1] [BLACK] LOG:  duration: 70.147 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 11 ORDER BY scene LIMIT 1
2012-10-24T05:42:21+00:00 app[postgres]: [34-1] [BLACK] LOG:  duration: 144.204 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 6 ORDER BY scene LIMIT 1
2012-10-24T05:42:22+00:00 app[postgres]: [35-1] [BLACK] LOG:  duration: 56.623 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 1956 ORDER BY scene LIMIT 1
2012-10-24T05:42:23+00:00 app[postgres]: [36-1] [BLACK] LOG:  duration: 64.860 ms  statement: SELECT  "movies".* FROM "movies"  WHERE "movies"."dvd_id" = 747 ORDER BY scene LIMIT 1

Answer 1

You can connect directly to your database CLI and then use the PostgresQL EXPLAIN command to get information on how it's running your query. This can show places where you can add indices to your tables to speed things up. The Postgres docs explain (see what I did there?) EXPLAIN in more detail.

Answer 2

What is your code that is responsible for these queries? Are you using eager loading? Your query should be something like:

DVD.includes(:movies).where(whatever) #assuming DVD has many movies

This should reduce the number of queries/requests. to your database. Your queries suggest that you may have a N+1 problem.

You should also have a database index on the movies table for dvd_id as it is a foreign key.

Answer 3

Make sure you at least have index on dvd_id. If you can create compound index on (dvd_id, scene), your query will be fully optimal and cannot be optimized any further.

In other words, just execute

CREATE INDEX movies_dvd_id_scene_idx ON movies (dvd_id, scene);

and you should be all set

Answer 4

duration: 57.914 ms
duration: 77.086 ms
duration: 85.602 ms
duration: 70.147 ms
duration: 144.204 ms
duration: 56.623 ms
duration: 64.860 ms

On average (~80ms) this is the usual duration needed for queries operating over different continents. Make sure both your Heroku app and DB are hosted in the same region (EU or US).