plone.memoize cache depending on function's return value

Question

i'm trying to cache the return value of a function only in case it's not None.

in the following example, it makes sense to cache the result of someFunction in case it managed to obtain data from some-url for an hour.

if the data could not be obtained, it does not make sense to cache the result for an hour (or more), but probably for 5 minutes (so the server for some-domain.com has some time to recover)

def _cachekey(method, self, lang):
    return (lang, time.time() // (60 * 60))

@ram.cache(_cachekey)
def someFunction(self, lang='en'):
    data = urllib2.urlopen('http://some-url.com/data.txt', timeout=10).read()

    except socket.timeout:
        data = None
    except urllib2.URLError:
        data = None

    return expensive_compute(data)

calling method(self, lang) in _cachekey would not make a lot of sense.

Answer 1

In this case, you should not generalize "return as None", as decorator cached results can depend only on input values.

Instead, you should build the caching mechanism inside your function and not rely on a decorator.

Then this becomes a generic non-Plone specific Python problem how to cache values.

Here is an example how to build your manual caching using RAMCache:

https://developer.plone.org/performance/ramcache.html#using-custom-ram-cache

Answer 2

as this code would be too long for a comment, i'll post it here in hope it'll help others:

#initialize cache
from zope.app.cache import ram
my_cache = ram.RAMCache()
my_cache.update(maxAge=3600, maxEntries=20)
_marker = object()


def _cachekey(lang):
    return (lang, time.time() // (60 * 60))


def someFunction(self, lang='en'):

    cached_result = my_cache.query(_cacheKey(lang), _marker)

    if cached_result is _marker:
        #not found, download, compute and add to cache
        data = urllib2.urlopen('http://some-url.com/data.txt', timeout=10).read()
        except socket.timeout:
            data = None
        except urllib2.URLError:
            data = None

        if data is not None:
            #cache computed value for 1 hr
            computed = expensive_compute(data)
            my_cache.set(data, (lang, time.time() // (60 * 60) )
        else:
            # allow download server to recover 5 minutes instead of trying to download on every page load
            computed = None
            my_cache.set(None, (lang, time.time() // (60 * 5) )

        return computed


    return cached_result