Django - Johnny Cache for multiple processes

Question

I've configured johnny cache with one of my applications that is hosted on apache. It is configured with memcached as the backend which runs on the same machine on the default port.

The caching works fine when multiple web clients go through apache. They all read from the cache and any update is invalidating the cache. But when a python program/script reads from the DB using django (same settings.py that has johnny configuration), it doesn't read from the cache and hence any updates made by that program wont affect the cache. Which leaves me with the web clients reading stale data from the cache.

I haven't found anything in johnny cache's documentation related to this. Any thoughts on this situation?

I'm using johnny cache 0.3.3, django 1.2.5 and python 2.7.

Edit: to answer one of the quetions in the comments, I read from the DB in the script this way-

>>> cmp = MyModelClass.objects.get(id=1)
>>> cmp.cust_field_2
u'aaaa'

I know it doesn't read from the cache because I update the table directly by firing an update sql statement and the updated value is not reflected in my web client as it still reads from the cache. Whereas my script shows the updated value when I re-fetch the object using MyModelClass.objects.get(id=1)

Thanks,

Answer 1

It appears that middleware is not called when you run scripts/management comands which is why you are seeing the difference. This makes sense when reading the documentation on middleware because it processes things like request and views, which don't exist in a custom script.

I found a way around this, and there is an issue regarding it in the Johnny Cache bitbucket repo. In your script put the following before you do anything with the database:

from johnny.middleware import QueryCacheMiddleware
qcm = QueryCacheMiddleware()

# put the code for you script here

qcm.unpatch()

You can see more on that here:

https://bitbucket.org/jmoiron/johnny-cache/issue/49/offline-caching

and here:

https://bitbucket.org/jmoiron/johnny-cache/issue/50/johhny-cache-not-active-in-management

Answer 2

That is the recommended way from the documentation:

from johnny.cache import enable
enable()

Update:

What I observed, as if your tasks.py files have this in the beginning, you can not disable johnny cache using settings.py anymore.

I have reported the issue: https://github.com/jmoiron/johnny-cache/issues/27