Cache warming with RABL for JSON templates

Question

Okay, this post is a bit wordy before I got to the actual question, so the abridged version basically pertains to cache warming using RABL templates. When calling Rabl.render vs API calls, the caches generated do not have the same cache-keys. When using Rabl.render directly should I expect cache-keys to match, when the same template is called via an API?

K, now the wind-up..

I have a Rails API server on Heroku. I've done a lot of optimization with RABL using russian doll caching to improve the reuse of the underlying objects in collections. Though, I am still left with collection caches, that when generated by the user on first request, is a burden on the experience ( e.g. 1+ second api calls).

When debugging a sample API call, I get the following cache actions on a given object.

...api/v1/activities/26600 :

Cache read: rabl/activities/26600-20140423170223588554000//hash/d30440d18014c72014a05319af0626f7 
Cache generate: rabl/activities/26600-20140423170223588554000//hash/d30440d18014c72014a05319af0626f7 
Cache write: rabl/activities/26600-20140423170223588554000//hash/d30440d18014c72014a05319af0626f7

so for the same object when calling ...api/v1/activities ( after above call ) I get the desired cache hit:

Cache read: rabl/activities/26600-20140423170223588554000//hash/d30440d18014c72014a05319af0626f7 
Cache fetch_hit: rabl/activities/26600-20140423170223588554000//hash/d30440d18014c72014a05319af0626f7

This works great. The next step is to avoid having the first/any API call spend time generating the cache.
So I have been pursuing cache warming techniques to generate these collections prior to the user accessing them. One suggestion is using wget as a way to hit the API directly ( see https://stackoverflow.com/a/543988/451488 ). But this adds a load on the Heroku web dynos, so I want to background cache warming via sidekiq workers.

RABL provides a way to render a template from code directly ( https://github.com/nesquena/rabl#rendering-templates-directly ) which definitely seems like the right approach for this use case. Therefore my intent is to call the RABL engine through some event prior to an API call (for example - a User login event).

So for the above API example, I'd call the following in rails console and expect a cache hit.

irb(main):002:0> @activity = Activity.find(26600)
irb(main):003:0> Rabl.render(@activity, 'api/v2/activities/show_no_root', :view_path => 'app/views', :format => :json) 
Cache read: rabl/activities/26600-20140423170223588554000//hash 
Cache generate: rabl/activities/26600-20140423170223588554000//hash 
Cache write: rabl/activities/26600-20140423170223588554000//hash

Unexpectedly, I did not get a cache hit, but it is obvious that the cache-keys are not the same since the trailing hash signature is missing. I'm not sure why the cache-keys would be different in this case. I am left with no way to warm caches for RABL templates.

UPDATE

Turns out the hash in the trailing cache key is the template hash.

 Cache digest for api/v1/activities/_show.rabl: d30440d18014c72014a05319af0626f7

Though this tells me the source of that hash, its still not clear why calling Rabl::Renderer directly would not use this as well.

Answer 1

I wasn't able to use Rabl::Render due to the missing template digest hash in the cache-key. However by creating a sidekiq worker as below, I am able to warm the cache by calling the api as a background process, which works nicely.

 class CacheWarmApi
   include Sidekiq::Worker
   sidekiq_options :queue => :warmers

   def perform( url_helper, args, params={},method='get')
     if method == 'get'
       session = ActionDispatch::Integration::Session.new(Rails.application)
       session.get(session.send(url_helper, *args), params)
     end
   end
 end

For example :

 CacheWarmApi.perform_async( :api_v2_expensiveapi_url, args_array , params_hash)

I think this is a bit too heavy of a solution, and still think there is a solution out there with Rabl::Render.

Answer 2

I've managed to accomplish this by calling Rabl.render(ref) directly, storing the result in cache directly (Redis in my case) and using that from the request handler in the controller:

Worker:

#... do some long work to calculate object_to_render_with
#.
#.
#.

#render the result using Rabl 
render_result= Rabl.render(object_to_render_with, rabl_view_relative_path,
                 view_path: File.join(Rails.root, 'app/views'),
                 format: :json) #already the default, being explicit
Redis.current.set(cache_key, render_result, ex: (DEFAULT_CACHE_EXPIRY))

Controller:

def request
    #check if we have a recent result
    res = Redis.current.get cache_key
    if res.nil?
      # No result in cache, start the worker
     Worker.perform_async
    end

    # use that cached result or an empty result
    render json: res || {status: 'in_progress'}
end

Bonus: also added a layer of tracking progress in the same manner (using other key and other request, updating the Redis manually along the progress of work).

HTH