Python, how to use __setattr__ on dictionary object that is part of the class that overloads the method?

Question

As illustrated in the code below, why can't I use __setattr__ to set values on a dict that is part of the class that overloads the method? I expected that b.hello would not exist.

class MyClass():

    datastore = {}

    def __init__(self):
        self.datastore = {}

    def __getattr__(self, key):
        return self.datastore[key]

    def __setattr__(self, key, value):
        self.datastore[key] = value

a = MyClass()
b = MyClass()

a.hello = "err"

print a.hello # err
print b.hello # err

Answer 1

b.hello prints your string "err" because datastore is an attribute of the class itself, not of objects of the class. Therefore, when you initialize it in a, b can also access it.

Therefore, remove the datastore = {} from the class.

Furthermore, from the Python docs:

if __setattr__() wants to assign to an instance attribute, it should not simply execute self.name = value — this would cause a recursive call to itself. Instead, it should insert the value in the dictionary of instance attributes, e.g., self.__dict__[name] = value. For new-style classes, rather than accessing the instance dictionary, it should call the base class method with the same name, for example, object.__setattr__(self, name, value).

So, change your code to:

class MyClass(object): # Use new style classes
    def __init__(self):
        object.__setattr__(self, 'datastore', {}) # This prevents infinite recursion when setting attributes

    def __getattr__(self, key):
        return self.datastore[key]

    def __setattr__(self, key, value):
        self.datastore[key] = value

a = MyClass()
b = MyClass()

a.hello = "err"

print a.hello # Works
print b.hello # Gives an error

Answer 2

Let me first explain why this occurs:

class MyClass():

    datastore = {}

    def __init__(self):
        self.datastore = {} # calls __setattr__

As you can see, your first variable definition in __init__ ends up calling __setattr__

    def __setattr__(self, key, value):
        self.datastore[key] = value

The instance does not yet have the datastore attribute because it is still in the process of being defined. So this line self.datastore[key] = value first trys to look up datastore in the instance's __dict__ but can't find it! Then it looks up one level in the class tree, where it does find it, as a class attribute!

Remember this:

class MyClass():

    datastore = {}

This is pretty confusing to begin with, since you have both an instance variable and a class variable with the same name, you should not have both.

So you can change your __init__ to this:

object.__setattr__(self, 'datastore', {})

Like @Dhara suggested, or you can use the more general approach which I would recommend:

super(MyClass, self).__setattr__('datastore', {})

Where the latter option only works for new-style classes (which are better in every way!) which you should be using! Just add object as a superclass

class MyClass(object): # In Py3k you don't need to since all classes are newstyle

One thing to note:

    def __setattr__(self, key, value):
        self.datastore[key] = value

Only works because you are setting the key of a dictionary and not an attribute of the instance. Be careful not to do things like

    def __setattr__(self, key, value):
        self.datastore = {} # for example

because that will result in infinite recursion, if you ever want to do something similar in __setattr__ use the same trick from before:

    def __setattr__(self, key, value):
        super(MyClass, self).__setattr__('datastore', {})

The final result should look like:

class MyClass(object):

    def __init__(self):
        super(MyClass, self).__setattr__('datastore', {})

    def __getattr__(self, key):
        return self.datastore[key]

    def __setattr__(self, key, value):
        self.datastore[key] = value


a = MyClass()
b = MyClass()
a.hello = "err"
print a.hello
print b.hello