Question

I've implemented a memoize decorator, which allows to cache a function. The cache key includes the function arguments. Similarly the cached decorator caches a function, but ignores the arguments. Here is the code:

class ApplicationCache (Memcached):

The make_key method: UltraJSON delivers quickly a string, which SHA512 hashes into a crisp hex digest:

    def make_key (self, *args, **kwargs):

        kwargs.update (dict (enumerate (args)))
        string = ujson.encode (sorted (kwargs.items ()))
        hashed = hashlib.sha512 (string)

        return hashed.hexdigest ()

The memoize decorator: Since Python 2.x sucks w.r.t. fully qualified function names, I simply force the user to deliver a reasonable name:

    def memoize (self, name, timeout=None):
        assert name

        def decorator (fn):
            @functools.wraps (fn)
            def decorated (*args, **kwargs):

                key = self.make_key (name, *args, **kwargs)
                cached = self.get (key)

                if cached is None:
                    cached = fn (*args, **kwargs)
                    self.set (key, cached, timeout=timeout)

                return cached
            return decorated
        return decorator

The cached decorator: It's almost a verbatim copy of memoize with the single exception where make_key ignores the arguments:

    def cached (self, name, timeout=None):
        assert name

        def decorator (fn):
            @functools.wraps (fn)
            def decorated (*args, **kwargs):

                key = self.make_key (name) ## no args!
                cached = self.get (key)

                if cached is None:
                    cached = fn (*args, **kwargs)
                    self.set (key, cached, timeout=timeout)

                return cached
            return decorated
        return decorator

Now, my problem with cached is, that it screams for re-factoring: It should use memoize and the idea would be to eliminate the arguments of fn (using functools.partial maybe?), like:

    def cached (self, name, timeout=None):

        ## Reuse the more general `memoize` to cache a function,
        ## but only based on its name (ignoring the arguments)

I'm actually not sure if I'm overdoing the DRY principle here, and if reuse is even possible, since the current implementation of cached ignores the arguments only while building the key (but obviously not while invoking the decorated function).

Was it helpful?

Solution

I'd get rid of the name argument and provide the key function as an argument:

def memoize(self, timeout=None, keyfunc=self.make_key):
    ...
    key = keyfunc(function.__name__, *args, **kwargs)
    ...

cache will then just become:

def cache(self, timeout=None):
    return self.memoize(timeout, keyfunc=lambda f, *args, **kwargs: f.__name__)
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top