What would make rails cache values that should live a day not stick around?

Question

I'm running a Rails application that uses the cache mechanism to stash some REST API call results that don't change, and are time consuming. I'm using "fetch" exclusively, and have set the default cache (MemoryStore) to expire in a day.

The problem I have is that it's really hit or miss whether cached items are found via fetch or not. Sometimes they are there, sometimes not. Sometimes they will show up in the cache for a few requests, then vanish. It's not the entire cache that vanishes, just some entries.

What would make cache values that should live for a whole day not stick around?

---

For background, this is occuring on a development server and closed environment running internally - no other users or hits that would interfere with cache results.

First, I'm configuring it as so in config/applications.rb...

# --- Expire cache items defaults to 1 day (in seconds)
config.cache_expires_in = 24*60*60
config.cache_store = ActiveSupport::Cache::MemoryStore.new(:expires_in => config.cache_expires_in)

(Yea I know "1.day" would work - was just trying this explicitly if it made a diff (which it didn't))

Second, I'm calling caching like this in my controller action...

cache_key = APP_CATEGORIES_CACHE_KEY_BASE + lang
result = Rails.cache.fetch(cache_key, {:expires_in => Rails.configuration.cache_expires_in}) do
   # --- Painful remote service call here
end

Additionally, there are no other references to the cache in the code - I'm not deleting it, clearing it, etc.

I can hit the URL with a web browser on my local server to fetch the data, and it all works fine. I get exactly what I want.

But if I look in my cache log, I see sometimes it's finding the existing cache value, sometimes it's not. Sometimes I can load it 10 times in a row and it's always fetching fine, other times a fetch finds nothing and I have to rewrite it.

It's the same URL, same cache key every time, etc. Yet it's hit or miss (as it were) with the cache. Sometimes I'll do a write and immediately have to write again, or find it repeatedly then not. I can wait a few minutes and do a hit, and still find it, or I can do a hit 2 seconds later and it's not found.

Here's a sequential sample of my cache log file from me hitting reload repeatedly on the same URL. I've broken it apart a bit to make it easier to work with.

Hit 1 - not found, creates

329 D, [2014-05-14T16:09:51.773340 #29158] -- : Cache read: app_categories_feed_cache_key_en ({:expires_in=>86400})
330 D, [2014-05-14T16:09:51.773506 #29158] -- : Cache generate: app_categories_feed_cache_key_en ({:expires_in=>86400})
331 D, [2014-05-14T16:09:52.901890 #29158] -- : Cache write: app_categories_feed_cache_key_en ({:expires_in=>86400})

Hit 2 - not found, creates

333 D, [2014-05-14T16:09:57.680192 #29155] -- : Cache read: app_categories_feed_cache_key_en ({:expires_in=>86400})
334 D, [2014-05-14T16:09:57.680446 #29155] -- : Cache generate: app_categories_feed_cache_key_en ({:expires_in=>86400})
335 D, [2014-05-14T16:09:58.741355 #29155] -- : Cache write: app_categories_feed_cache_key_en ({:expires_in=>86400})

Hit 3 - found!

337 D, [2014-05-14T16:10:03.093808 #29155] -- : Cache read: app_categories_feed_cache_key_en ({:expires_in=>86400})
338 D, [2014-05-14T16:10:03.094082 #29155] -- : Cache fetch_hit: app_categories_feed_cache_key_en ({:expires_in=>86400})

Hit 4 - found again!

340 D, [2014-05-14T16:10:19.907132 #29155] -- : Cache read: app_categories_feed_cache_key_en ({:expires_in=>86400})
341 D, [2014-05-14T16:10:19.907414 #29155] -- : Cache fetch_hit: app_categories_feed_cache_key_en ({:expires_in=>86400})

Hit 5 - another found!

343 D, [2014-05-14T16:10:25.785796 #29158] -- : Cache read: app_categories_feed_cache_key_en ({:expires_in=>86400})
344 D, [2014-05-14T16:10:25.786225 #29158] -- : Cache fetch_hit: app_categories_feed_cache_key_en ({:expires_in=>86400})

Hit 6 - wtf? not found!

346 D, [2014-05-14T16:10:31.190847 #29146] -- : Cache read: app_categories_feed_cache_key_en ({:expires_in=>86400})
347 D, [2014-05-14T16:10:31.191054 #29146] -- : Cache generate: app_categories_feed_cache_key_en ({:expires_in=>86400})
348 D, [2014-05-14T16:10:32.254776 #29146] -- : Cache write: app_categories_feed_cache_key_en ({:expires_in=>86400})

Answer 1

In the end, the problem was because the site runs using Unicorn, which creates multiple workers. And workers do not share caching when using MemoryStore.

So each time I hit my server, I randomly am connected to a worker. If I had hit a worker before, it would have my cached value available. If the worker had not been hit before, then there was no cache hit.

The solution for me is to shift to using memcached, which will be shared by all workers. That translates to just one worker being hit resulting in all workers having the data available.