Is it possible to decorate a Python class whose instances will be pickled?

Question

The following code run correctly:

import pickle

class MyClass():   
    def __init__(self, arg):
        self.arg = arg

a = MyClass('my arg')
with open('/home/mahikeulbody/mypickle', 'wb') as file:
    pickle.dump(a, file)

but adding a decorator to get a multiton class :

import pickle

def multiton(cls):
    instances = {}
    def getinstance(arg):
        if arg not in instances:
            instances[arg] = cls(arg)
        return instances[arg]
    return getinstance

@multiton
class MyClass():   
    def __init__(self, arg):
        self.arg = arg

a = MyClass('my arg')
with open('/home/michel/mypickle', 'wb') as file:
    pickle.dump(a, file)

produces the following error:

pickle.dump(a, file)
_pickle.PicklingError: Can't pickle <class '__main__.MyClass'>: it's not the same object as __main__.MyClass

What is wrong ?

Answer 1

Pickle must be able to load the class directly. Your decorator replaces the class with a factory function, making it impossible for pickle to import the class itself.

Use a separate factory function, not a decorator, returning a 'private' class (but still importable directly):

class _MyClass():   
    def __init__(self, arg):
        self.arg = arg

def MyClass(arg, instances={}):
    if arg not in instances:
        instances[arg] = _MyClass(arg)
    return instances[arg]

Answer 2

To do this, I'd use dill, which can serialize almost anything in python.

>>> def multiton(cls):
...     instances = {}
...     def getinstance(arg):
...         if arg not in instances:
...             instances[arg] = cls(arg)
...         return instances[arg]
...     return getinstance
... 
>>> @multiton
... class MyClass():   
...     def __init__(self, arg):
...         self.arg = arg
... 
>>> import dill
>>>                       
>>> a = MyClass('my arg')
>>> b = dill.loads(dill.dumps(a))
>>> a
<__main__.MyClass instance at 0x4d64558>
>>> b
<__main__.MyClass instance at 0x4d64800>

Dill also has some good tools for helping you understand what is causing your pickling to fail when your code fails.