In Python, how do I call the super class when it's a one-off namedtuple?

Question

So, I have a large number of message Payload classes for a serial API, each of which has a number of immutable fields, a parse method, and some methods which are shared. The way I'm structuring this is that each will inherit from a namedtuple for the field behaviours, and receive the common methods from a parent class. However, I'm having some difficulties with the constructors:

class Payload:
    def test(self):
        print("bar")

class DifferentialSpeed(Payload, namedtuple('DifferentialSpeed_', 
    'left_speed right_speed left_accel right_accel')):
    __slots__ = ()
    def __init__(self, **kwargs):
        super(DifferentialSpeed, self).__init__(**kwargs)
        # TODO: Field verification
        print("foo")

    @classmethod
    def parse(self, raw):
        # Dummy for now
        return self(left_speed = 0.0, right_speed = 0.1,
                    left_accel = 0.2, right_accel = 0.3)

    def __str__(self):
        return "Left Speed: %fm/s\nRight Speed: %fm/s\n"\
            "Left Acceleration: %fm/s^2\nRight Acceleration: %fm/s^2" % (
            self.left_speed, self.right_speed, self.left_accel, self.right_accel)


payload = DifferentialSpeed.parse('dummy')
print(payload)

This works, but I get the following warning:

DeprecationWarning: object.__init__() takes no parameters
  super(DifferentialSpeed, self).__init__(**kwargs)

If I remove **kwargs from the call, it still seems to work, but why? How are those arguments to the constructor getting passed through to the namedtuple? Is this guaranteed, or a random result of how the mro gets established?

If I wanted to stay away from super, and do it the old way, is there some way I can access the namedtuple to call its constructor? I'd rather not have to do this:

DifferentialSpeed_ = namedtuple('DifferentialSpeed_', 
    'left_speed right_speed left_accel right_accel')
class DifferentialSpeed(Payload, DifferentialSpeed_):

Seems kind of verbose and unnecessary.

What's my best course of action here?

Answer 1

For starters, namedtuple(whatever) inherits from tuple, which is immutable, and immutable types don't bother with __init__, because by the time __init__ is called the object is already constructed. If you want to pass arguments to the namedtuple base class you'll have to override __new__ instead.

You can see the definition of the result of namedtuple() by passing in a verbose=true argument; I find it educational.

Answer 2

You have three base classes: Payload, your namedtuple DifferentialSpeed_, and the common base class object. Neither of the first two have an __init__ function at all, except for the one inherited from object. namedtuple doesn't need an __init__, since the initialization of immutable classes is done by __new__, which is called before __init__ is run.

Since super(DifferentialSpeed, self).__init__ resolves to the next __init__ in the call chain, the next __init__ is object.__init__, which means you're passing arguments to that function. It doesn't expect any--there's no reason to be passing arguments to object.__init__.

(It used to accept and silently ignore arguments. That behavior is going away--it's gone in Python 3--which is why you get a DeprecationWarning.)

You can trigger the problem more clearly by adding a Payload.__init__ function that takes no arguments. When you try to pass along `*kwargs, it'll raise an error.

The correct thing to do in this case is almost certainly to remove the **kwargs argument, and just call super(DifferentialSpeed, self).__init__(). It doesn't take any arguments; DifferentialSpeed is passing Payload its own arguments that Payload, and functions further down the call chain, know nothing about.

Answer 3

As others have pointed-out, tuples are an immutable type, which must be initialized in their __new__() instead of their __init__() method -- so you need to add the former in your subclass (and get rid of the latter). Below is how this would be applied to your example code. The only other change was adding a from import... statement to the beginning.

Note: cls has to be passed twice in the super() call in __new__() because it's a static method although it is special-cased so you don't have to declare it to be one.

from collections import namedtuple

class Payload:
    def test(self):
        print("bar")

class DifferentialSpeed(Payload, namedtuple('DifferentialSpeed_',
    'left_speed right_speed left_accel right_accel')):
    #### NOTE: __new__ instead of an __init__ method ####
    def __new__(cls, **kwargs):
        self = super(DifferentialSpeed, cls).__new__(cls, **kwargs)
        # TODO: Field verification
        print("foo")
        return self

    @classmethod
    def parse(self, raw):
        # Dummy for now
        return self(left_speed = 0.0, right_speed = 0.1,
                    left_accel = 0.2, right_accel = 0.3)

    def __str__(self):
        return "Left Speed: %fm/s\nRight Speed: %fm/s\n"\
            "Left Acceleration: %fm/s^2\nRight Acceleration: %fm/s^2" % (
            self.left_speed, self.right_speed, self.left_accel, self.right_accel)


payload = DifferentialSpeed.parse('dummy')
print(payload)