Python : subclass `type` to create specialized types (e.g. a “list of int”)

Question

I am trying to subclass type in order to create a class allowing to build specialized types. e.g. a ListType :

>>> ListOfInt = ListType(list, value_type=int)
>>> issubclass(ListOfInt, list)
True
>>> issubclass(list, ListOfInt)
False
>>> # And so on ...

However, this ListOfInt will never be used to create instances ! I just use it as an instance of type that I can manipulate to compare with other types ... In particular, in my case I need to look-up for a suitable operation, according to the type of input, and I need the type to contain more precisions (like list of int or XML string, etc ...).

So here's what I came up with :

class SpzType(type):

    __metaclass__ = abc.ABCMeta

    @classmethod
    def __subclasshook__(cls, C):
        return NotImplemented

    def __new__(cls, base, **features):
        name = 'SpzOf%s' % base.__name__
        bases = (base,)
        attrs = {}
        return super(SpzType, cls).__new__(cls, name, bases, attrs)

    def __init__(self, base, **features):
        for name, value in features.items():
            setattr(self, name, value)

The use of abc is not obvious in the code above ... however if I want to write a subclass ListType like in the example on top, then it becomes useful ...

The basic functionality actually works :

>>> class SimpleType(SpzType): pass
>>> t = SimpleType(int)
>>> issubclass(t, int)
True
>>> issubclass(int, t)
False

But when I try to check if t is an instance of SpzType, Python freaks out :

>>> isinstance(t, SpzType)
TypeError: __subclasscheck__() takes exactly one argument (0 given)

I explored with pdb.pm() what was going on, and I found out that the following code raises the error :

>>> SpzType.__subclasscheck__(SimpleType)
TypeError: __subclasscheck__() takes exactly one argument (0 given)

WeIrD ?! Obviously there is an argument ... So what does that mean ? Any idea ? Did I misuse abc ?

Answer 1

Thanks to comment from kindall, I have refactored the code to the following :

class SpzType(abc.ABCMeta):

    def __subclasshook__(self, C):
        return NotImplemented

    def __new__(cls, base, **features):
        name = 'SpzOf%s' % base.__name__
        bases = (base,)
        attrs = {}
        new_spz = super(SpzType, cls).__new__(cls, name, bases, attrs)
        new_spz.__subclasshook__ = classmethod(cls.__subclasshook__)
        return new_spz

    def __init__(self, base, **features):
        for name, value in features.items():
            setattr(self, name, value)

So basically, SpzType is now a subclass of abc.ABCMeta, and subclasshook is implemented as an instance method. It works great and it is (IMO) elegant !!!

EDIT : There was a tricky thing ... because __subclasshook__ needs to be a classmethod, so I have to call the classmethod function manually... otherwise it doesn't work if I want to implement __subclasshook__.

Answer 2

I'm not quite sure what you want to achieve. Maybe it is better to use collections module instead of using abc directly?

There is more info about generic collection classes in PEP 3119

Answer 3

The sort of thing you want to do could probably be done more easily using a class factory function like the following. At least for me, it makes it more straightforward to keep straight the various levels at which I'm trying to operate.

def listOf(base, types={}, **features):
    key = (base,) + tuple(features.items())
    if key in types:
        return types[key]
    else:

        if not isinstance(base, type):
            raise TypeError("require element type, got '%s'" % base)

        class C(list):

             def __init__(self, iterable=[]):
                 for item in iterable:
                     try:    # try to convert to desired type
                         self.append(self._base(item))
                     except ValueError:
                         raise TypeError("value '%s' not convertible to %s"
                            % (item, self._base.__name__))

              # similar methods to type-check other list mutations

        C.__name__ = "listOf(%s)" % base.__name__
        C._base = base
        C.__dict__.update(features)  
        types[key] = C
        return C

Note that I'm using a dict as a cache here so that you get the same class object for a given combination of element type and features. This makes listOf(int) is listOf(int) always True.

Answer 4

Here's a decorator version of my other answer that works with any class. The decorator returns a factory function that returns a subclass of the original class with the desired attributes. The nice thing about this approach is that it does not mandate a metaclass, so you can use a metaclass (e.g. ABCMeta) if desired without conflicts.

Also note that if the base class uses a metaclass, that metaclass will be used to instantiate the generated subclass. You could, if you wanted, hard-code the desired metaclass, or, you know, write a decorator that makes a metaclass into a decorator for template classes... it's decorators all the way down!

If it exists, a class method __classinit__() is passed the arguments passed to the factory, so the class itself can have code to validate arguments and set its attributes. (This would be called after the metaclass's __init__().) If __classinit__() returns a class, this class is returned by the factory in place of the generated one, so you can even extend the generation procedure this way (e.g. for a type-checked list class, you could return one of two inner classes depending on whether the items should be coerced to the element type or not).

If __classinit__() does not exist, the arguments passed to the factory are simply set as class attributes on the new class.

For convenience in creating type-restricted container classes, I have handled the element type separately from the feature dict. If it's not passed, it'll be ignored.

As before, the classes generated by the factory are cached so that each time you call for a class with the same features, you get the same class object instance.

def template_class(cls, classcache={}):

    def factory(element_type=None, **features):

        key = (cls, element_type) + tuple(features.items())
        if key in classcache:
            return classcache[key]

        newname  = cls.__name__
        if element_type or features:
            newname += "("
            if element_type:
                newname += element_type.__name__
                if features:
                    newname += ", "
            newname += ", ".join(key + "=" + repr(value)
                                 for key, value in features.items())
            newname += ")"

        newclass = type(cls)(newname, (cls,), {})
        if hasattr(newclass, "__classinit__"):
            classinit = getattr(cls.__classinit__, "im_func", cls.__classinit__)
            newclass = classinit(newclass, element_type, features) or newclass
        else:
            if element_type:
                newclass.element_type = element_type
            for key, value in features.items():
                setattr(newclass, key, value)

        classcache[key] = newclass
        return newclass

    factory.__name__ = cls.__name__
    return factory

An example type-restricted (type-converting, actually) list class:

@template_class
class ListOf(list):

    def __classinit__(cls, element_type, features):
        if isinstance(element_type, type):
            cls.element_type = element_type
        else:
            raise TypeError("need element type")

    def __init__(self, iterable):
        for item in iterable:
            try:
                self.append(self.element_type(item))
            except ValueError:
                raise TypeError("value '%s' not convertible to %s"
                        % (item, self.element_type.__name__))

    # etc., to provide type conversion for items added to list 

Generating new classes:

Floatlist = ListOf(float)
Intlist   = ListOf(int)

Then instantiate:

print FloatList((1, 2, 3))       # 1.0, 2.0, 3.0
print IntList((1.0, 2.5, 3.14))  # 1, 2, 3

Or just create the class and instantiate in one step:

print ListOf(float)((1, 2, 3))
print ListOf(int)((1.0, 2.5, 3.14))