Is it possible to add __slots__ in decorator of class that already defines __slots__?

Question

Let me start off by saying that I understand how slots and metaclasses work in Python. Playing around with the two, I've run into an interesting problem. Here's a minimal example:

def decorator(cls):
    dct = dict(cls.__dict__)
    dct['__slots__'] = ('y',)
    return type('NewClass', cls.__bases__, dct)

@decorator
class A(object):
    __slots__= ('x',)
    def __init__(self):
        self.x = 'xx'

A()

This produces the following exception:

Traceback (most recent call last):
  File "p.py", line 12, in <module>
    A()
  File "p.py", line 10, in __init__
    self.x = 'xx'
TypeError: descriptor 'x' for 'A' objects doesn't apply to 'NewClass' object

Now, I know why this happens: the descriptor created for the slot x must be able to reference the reserved space for the slot. Only instances of class A, or instances of subclasses of A, have this reserved space, and therefore only those instances can use the descriptor x. In the above example, the metaclass creates a new type that is a sublcass of A's base classes, but not of A itself, so we get the exception. Simple enough.

Of course, in this simple example, either of the following two definitions of decorator will work around the problem:

def decorator(cls):
    dct = dict(cls.__dict__)
    dct['__slots__'] = ('y',)
    return type('NewClass', (cls,) + cls.__bases__, dct)

def decorator(cls):
    class NewClass(cls):
        __slots__ = ('y',)
    return NewClass

But these work-arounds aren't exactly he same as the original, as they both add A as a base class. They can fail in a more complicated set up. For example, if the inheritance tree is more complicated, you might run into the following exception: TypeError: multiple bases have instance lay-out conflict.

So my very specific question is:

Is there are way to create a new class, via a call to type, that modifies the __slots__ attribute of an existing class, but does not add the existing class as a base class of the new class?

Edit:

I know that strict metaclasses are another work around for my examples above. There are lots of ways to to make the minimal examples work, but my question is about creating a class via new that is based on an existing class, not about how to make the examples work. Sorry for the confusion.

Edit 2:

Discussion in the comments has led me a more precise question than what I originally asked:

Is it possible to create a class, via a call to type, that uses the slots and descriptors of an existing class without being a descendant of that class?

If the answer is "no", I'd appreciate a source as to why not.

Answer 1

No, unfortunately there is no way to do anything with the __slots__ after the class is created (and that's when the decorators on them are called). The only way is to use a metaclass, and modify/add __slots__ before calling the type.__new__.

An example of such a metaclass:

class MetaA(type):
    def __new__(mcls, name, bases, dct):
        slots = set(dct.get('__slots__', ()))
        slots.add('y')
        dct['__slots__'] = tuple(slots)
        return super().__new__(mcls, name, bases, dct)

class BaseA(metaclass=MetaA):
    pass

class A(BaseA):
    __slots__ = ('x',)

    def __init__(self):
        self.x = 1
        self.y = 2

print(A().x, A().y)

Without metaclasses, you can do some magic and copy everything from the defined class and create a new one on the fly, but that code smells ;)

def decorator(cls):
    slots = set(cls.__slots__)
    slots.add('y')
    dct = cls.__dict__.copy()
    for name in cls.__slots__:
        dct.pop(name)
    dct['__slots__'] = tuple(slots)
    return type(cls)(cls.__name__, cls.__bases__, dct)

@decorator
class A:
    __slots__ = ('x',)
    def __init__(self):
        self.x = self.y = 42

print(A().x, A().y)

The main disadvantage of such code, is that if someone applies another decorator, before yours one, and, let's say, creates a reference to the decorated class somewhere, then they will end up storing reference to a different class. Same for metaclasses - they will execute twice. So, the metaclass approach is better, since there are no side-effects.

---

The definitive answer of why you can't really change __slots__ after the class is created depends on implementation details of the python interpreter you're working with. For instance, in CPython, for each slot you defined, class has a descriptor (see PyMemberDescr_Type & PyMemberDef struct in CPython source code), that has an offset parameter of where the slot value is aligned in internal object storage. And you simply have no instruments of manipulating such things in public Python API. You trade flexibility for less memory usage (again, in CPython, as in PyPy you get the same memory effect automatically for all your classes).

If modification of __slots__ is absolutely required, you can, probably, write a C extension (or work with ctypes) and do it, but that's hardly a reliable solution.

Answer 2

You can do that with metaclasses:

class MetaSlot(type):
    def __new__(mcs, name, bases, dic):
        dic['__slots__'] += ('y',)
        return type.__new__(mcs, name, bases, dic)


class C(metaclass=MetaSlot):  # Python 3 syntax
    __slots__ = ('x',)

Now both x and y can be used:

>>> c = C()
>>> c.y = 10
>>> c.x = 10