Faster attribute access in python

Question

Consider the following class:

class MyObject(object):

    __slots__ = ('_att1', '_att2')

    def __init__(self):
        self._att1 = None
        self._att2 = None

    @property
    def att1(self):
        """READ-ONLY property. """
        return self._att1

    @property
    def att2(self):
        """att2 property description. """
        return self._att2

    @att2.setter
    def att2(self, val):
        self._att2 = val

An advantage of using the property decorator is that we can add some documentation

a = MyObject()
help(a)
Help on MyObject in module __main__ object:

class MyObject(__builtin__.object)
 |  Methods defined here:
 |  
 |  __init__(self)
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  att1
 |      READ-ONLY property.
 |  
 |  att2
 |      att2 property description.

If the class is meant to be used by the final user then it is fine to access the attributes by using the properties.

a.att2 = "new value"

However, if I were to derive a class from MyObject, is it acceptable to use the "protected" variables in the derived class? That is,

 class Derived(MyObject):

     __slots__ = ()

     def __init__(self):
         self._att1 = 1
         self._att2 = 0

     @property
     def att2(self):
         """att2 adds 1 to itself in every call. """
         self._att2 += 1
         return self._att2

     @att2.setter
     def att2(self, val):
         self._att2 = val

I'm aware that if MyObject was an object from a third party then the attributes with underscore are subject to change and thus my code would break if they ever do change. However, since I am using MyObject as a base class I would think that it is fine to use this with my own derived classes.

I recently started using pylint and it made me aware of the fact that I am using "protected" variables in my derived classes. The reason I decided to post a question here is to know if this is something acceptable to do so that I can just suppress those warnings in pylint. If it isn't, then what is the standards to deal with this?

Another point I would like to make is about the attribute access. What would be faster

a.att1

or

a._att1

I'm under the impression that by doing a.att1 python would first look in the objects dictionary, or slots as in the example I am working with. If that isn't located there when it looks for functions (as is in the case with the __getattr__). When doing a lot of computations in my class definition I rather access something located in the dictionary or in the slots as opposed to the ones I defined with the decorators. Is this considered bad practice among the python community? I'm only asking this since the default configuration of pylint tells me otherwise and I would like to continue working having good standards in mind.

EDIT:

Let us try not to get into the discussion of __slots__ and premature optimization. If possible assume that in my original post I did not use __slots__ and that everything is in the object's dictionary.

Answer 1

Python doesn't have a formal distinction between private and protected class members. Even the distinction between private and public is pretty weak, since anything can be accessed if a user tries hard enough. If a private versus protected distinction would make sense for your application, you can simply document that _foo (or foo) is not part of the public user interface, but can be relied upon by subclasses to always have certain semantics. (What those semantics are is up to you.)

Now, I don't know if it's really worth the documentation effort to do this for a simple read-only property (the function call overhead is likely not too bad, so subclasses can just use the property like everyone else). However, if you had a property that was doing a lot more work, like a database query or HTTP request, it might make some sense for the internals to be exposed a bit for subclasses to get at just the parts they need.

If you do think that exposing the "protected" value is desirable, the only question is whether you should put the underscore in front of the variable name or not. The underscore has a few effects on things like dir and perhaps in documentation generating tools, but it doesn't change anything about how the code runs.

So it's really an issue of code style, and it really doesn't matter much either way. You can either leave the underscore off and put big warning text in the documentation so that users know they shouldn't be messing with the internals, or you could use the underscore and silence the pylint warnings (and if necessary, force there to be some extra docs for subclass implementors).

Answer 2

Attributes with underscore are not “protected”, but “private”—they are subject to change. Even if your class is derived from it. You should use parent property to access this data.

Speaking about performance. Of course, property access is a bit slower than attribute access, simply because it involves a function call, but you shouldn't care about it. By the way it has nothing to do with __getattr__ and all this stuff. Properies are also looked up in the dictionary as normal attributes, they just implement the descriptor protocol.

Answer 3

Short answer: "Premature optimization is the root of all evil."

Trust pylint. See if you really have a performance problem. If yes, profile and than go for the bottlenecks. Changing a loop or a data structure is likely much more effective.