What are the potential dangers of using this kind of reflection cheat in Python?

Question

In a program I'm working on, I need to do a substantial amount of real-time reflection in order to maintain a list of known "attributes" throughout the program structure (for use by a sort of "virtual programming assistant"). I came up with kind of a "cheaty" way of doing this (which I'm frankly kind of proud of), which is essentially akin to the following:

old_getattribute = obj.__getattribute__
def new_getattribute(self, attr):
    # various things...
    return old_getattribute(attr)
methods = {"__getattribute__" : new_getattribute}
obj.__class__ = type(              # Dynamically create a new class
    "%s" % obj.__class__.__name__, # whose name is _CLASS
    (obj.__class__, ),             # which subclasses CLASS
    methods)                       # and uses the new __getattribute__

Essentially, I dynamically create a new type that subclasses the original type, using a new __getattribute__ method, and then reassign the object's internal __class__ to this new dynamic type.

As amazed and happy as I am that this works (in Python >=3.6 at least), I'm still on the fence about using it. In my past python programming, any kind of modification of an object's magic methods from outside the object's class definition was pretty much forbidden, and yet here I am doing it twice in really messed up ways.

I would like to use it, as it would save me a ton of time instead of keeping track of individual objects in some "appropriate" manner and then continually checking them for changes, but I want to be sure that I'm not going to create any potential danger for the rest of the program.

So that's what I'm here to ask about. From a software architecture standpoint, how potentially dangerous is this method, and what pitfalls I may encounter getting this to work? I'm sure there are more than a few OOP/general programming principles I'm severely violating with this, but I'm willing to let those slide if this can be "managed".

Answer 1

When you're using __getattribute__ you are deep in the bowel's of Python's object model. Things might work, but require a solid understanding. Also, some aspects might be implementation details of CPython and might fail on other implementations.

Things to consider:

• __getattribute__ is insufficient to observe object modifications.
• __getattribute__ will not be called when accessing special methods and possibly not when the object is accessed from within the Python interpreter.
• consider how this interacts with class member access.
• __class__ might not be assignable.
• Changing the class breaks type(x) is Foo style checks.
• Metaclasses might prevent you from extending the original class.
• Specifying a __getattribute__ implies a noticeable performance overhead. This might be a bit better if you implement it in C (provided you're using CPython).

From an OOP perspective, you're no longer working within the OO paradigm: instead of sending messages to other objects, you're intercepting the language's message dispatch implementation. That's arguable fine, and even has a name: Aspect-Oriented Programming. Python is flexible enough to pull this off implicitly, as you have shown. However, it might be vastly desirable to not instrument existing objects like this, but offer an API that makes it easier to make objects observable. Such an API would perhaps implement properties that can trigger a callback on access.

If you need to observe objects not during their normal execution but for some analysis, it might actually be easier to patch the CPython runtime with an extra hook. You might find some overlap with existing debugger hooks.