Lost automatic ActiveRecord caching by migrating from Rails 3 to Rails 4

StackOverflow https://stackoverflow.com/questions/18720355

Question

I have a rather intense method processing rather large batches of things. It depends on ActiveRecord to be conservative about hitting database with exact same query zillion of times.

This is what I had in Rails 3:

  CACHE (0.0ms)  SELECT `song_work_numbers`.* FROM `song_work_numbers` ...
  CACHE (0.0ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
  CACHE (0.0ms)  SELECT `songs`.* FROM `songs` ...
  CACHE (0.0ms)  SELECT `variance_statement_line_matches`.* ...
  CACHE (0.0ms)  SELECT COUNT(*) FROM `productions` ...
  CACHE (0.0ms)  SELECT `productions`.* FROM `productions` ...
  ProductionWorkNumber Load (0.6ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
  SQL (0.4ms)  SELECT COUNT(*) FROM `production_episodes` ...
  ProductionEpisode Load (0.4ms)  SELECT `production_episodes`.* FROM `production_episodes` ...
Processed Line [449] in 0.010037 s

  CACHE (0.0ms)  SELECT `song_work_numbers`.* FROM `song_work_numbers` ...
  CACHE (0.0ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
  CACHE (0.0ms)  SELECT `songs`.* FROM `songs` ...
  CACHE (0.0ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
  CACHE (0.0ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
  CACHE (0.0ms)  SELECT COUNT(*) FROM `productions` ...
  CACHE (0.0ms)  SELECT `productions`.* FROM `productions` ...
  CACHE (0.0ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
  CACHE (0.0ms)  SELECT COUNT(*) FROM `production_episodes` ...
  CACHE (0.0ms)  SELECT `production_episodes`.* FROM `production_episodes` ...
Processed Line [450] in 0.006033 s

And this is what I get in Rails 4

  SongWorkNumber Load (1.5ms)  SELECT `song_work_numbers`.* FROM `song_work_numbers` ...
  Variance::StatementLineMatch Load (0.4ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
  Song Load (0.5ms)  SELECT `songs`.* FROM `songs` ...
  ProductionWorkNumber Load (1.9ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
  Variance::StatementLineMatch Load (0.9ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
   (2.9ms)  SELECT COUNT(*) FROM `productions` ...
  Production Load (1.5ms)  SELECT `productions`.* FROM `productions` ...
  ProductionWorkNumber Load (1.7ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
   (0.6ms)  SELECT COUNT(*) FROM `production_episodes` ...
  ProductionEpisode Load (112.2ms)  SELECT `production_episodes`.* FROM `production_episodes` ...
[Processed line 449 in 0.137435 sec]

  SongWorkNumber Load (1.5ms)  SELECT `song_work_numbers`.* FROM `song_work_numbers` ...
  Variance::StatementLineMatch Load (0.4ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
  Song Load (0.5ms)  SELECT `songs`.* FROM `songs` ...
  ProductionWorkNumber Load (1.7ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
  Variance::StatementLineMatch Load (0.4ms)  SELECT `variance_statement_line_matches`.* FROM `variance_statement_line_matches` ...
   (0.5ms)  SELECT COUNT(*) FROM `productions` ...
  Production Load (1.5ms)  SELECT `productions`.* FROM `productions` ...
  ProductionWorkNumber Load (1.9ms)  SELECT `production_work_numbers`.* FROM `production_work_numbers` ...
   (0.6ms)  SELECT COUNT(*) FROM `production_episodes` WHERE ...
  ProductionEpisode Load (0.5ms)  SELECT `production_episodes`.* FROM `production_episodes` ...
[Processed line 450 in 0.023306 sec]

As you can see this is much much slower. When going over 50K+ lines those milliseconds really add up.

Queries are identical. Code is the same. The only difference is that in Rails 4 version that code is ran from the delayed_job and in Rails 3 it was handled during normal browser request.

Is there a setting or something that I missed?

Was it helpful?

Solution

I figured it out. SqlCache is used via middleware in Rails: use ActiveRecord::QueryCache This means that it's the ActionController that wraps action processing with the cache blanket. Obviously within delayed job there's no ActionController and no caching. All you need to do to bring caching back is this:

def method_ran_by_delayed_job
  ActiveRecord::Base.cache do
    # all your crazy code
  end
end
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top