Python: dynamic class generation: overwrite members

Question

I have a python class hierarchy, that I want to extend at runtime. Furthermore every class in this hierarchy has a static attribute 'dict', that I want to overwrite in every subclass. Simplyfied it looks like this:

'dict' is a protected (public but with leading underscore) member

class A(object):
    _dict = {}

    @classmethod
    def getdict(cls):
        return cls._dict

    @classmethod
    def setval(cls, name, val):
        cls._dict[name] = val

    @classmethod
    def addchild(cls, name):
        return type(name, (cls, ), { '_dict' : {} })

B = A.addchild('B')
A.setval(1, 5)

print A.getdict()
# prints: {1: 5}
# like expected

print B.getdict()
# prints: {}
# like expected

This works just like expected. The question now is: why doesnt it work anymore if I declare the attribute private:

Now the same thing with 'dict' beeing a private member

class C(object):
    __dict = {}

    @classmethod
    def getdict(cls):
        return cls.__dict

    @classmethod
    def setval(cls, name, val):
        cls.__dict[name] = val

    @classmethod
    def addchild(cls, name):
        return type(name, (cls, ), { '__dict' : {} })

D = C.addchild('D')
C.setval(1, 5)

print C.getdict()
# prints: {1: 5}
# like expected

print D.getdict()
# prints: {1: 5}
# why!?

Suddenly D, the subclass of C, has the same values in 'dict' as its superclass!?

Could anyone be so kind and explain to me, what the reason for this is? Thanks in advance!

Answer 1

phild, as you know, when you prefix an attribute name with double-underscore __, the python interpreter automagically changes (mangles) attribute name from __attribute to _CLS__attribute, where CLS is the class name.

However, when you say

return type(name, (cls, ), { '__dict' : {} })

the keys in the dictionary { '__dict' : {} } do not get mangled. __dict remains the same.

Thus D ends up with both D._C__dict and D.__dict:

(Pdb) dir(D)
['_C__dict', '__class__', '__delattr__', '__dict', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'addchild', 'getdict', 'setval']

D._C__dict refers to C's class attribute. So when you run

C.setval(1, 5)

you are changing D._C__dict as well as C._C__dict. They are one and the same.

Answer 2

Here is a chapter in documentation about "private" attributes. And I commented you class definition to make it more clear:

class C(object):
    __dict = {} # This creates C.__dict__['_C__dict']

    @classmethod
    def getdict(cls):
        return cls.__dict # Uses cls.__dict__['_C__dict'] 

    @classmethod
    def setval(cls, name, val):
        cls.__dict[name] = val # Uses cls.__dict__['_C__dict'] 

    @classmethod
    def addchild(cls, name):
        return type(name, (cls, ), { '__dict' : {} }) # Creates child.__dict__['__dict']

I.e. all childs have their own __dict attribute, but only one from base class is used.

Answer 3

The Java or C++ concepts of "protected" and "private" do not apply. The naming convention Python does a little, but not what you're imagining.

The __name does some name mangling, making it hard to access because the name is obscured.

Your _dict and __dict are simply class-level attributes that are simply shared by all instances of the classes.