Python: strategies for persistently memoizing functions with function arguments?

Question

I have written a little class to persistently memoize some expensive functions that do various statistical analyses of random networks.

These are all pure functions; all the data is immutable. However, some of the functions take functions as arguments.

Making keys based on these arguments is a small problem, since in Python function object equality is equivalent to function object identity, which does not persist between sessions, even if the function implementation does not change.

I am hacking around this for the time being by using the function name as a string, but this raises its own swarm of issues when one starts thinking about changing the implementation of the function or anonymous functions and so on. But I am probably not the first to worry about such things.

Does anybody have any strategies for persistently memoizing functions with function arguments in Python?

Answer 1

Have a look at using this as the identity of the function

[getattr(func.__code__,s) 
 for s in ['co_argcount', 'co_cellvars', 'co_code', 'co_consts', 
           'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars',
           'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize',
           'co_varnames']
]

that should correctly handle changing the implementation in any way...

Answer 2

One option would be to use marshal.dumps(function.func_code)

It'll produce a string representation for the code of the function. That should handle changing implementations and anonymous functions.