Python __iter__ and for loops

Question

As I understand it, I can use the for loop construction on an object with a __iter__ method that returns an iterator. I have an object for which I implement the following __getattribute__ method:

def __getattribute__(self,name):
    if name in ["read","readlines","readline","seek","__iter__","closed","fileno","flush","mode","tell","truncate","write","writelines","xreadlines"]:
        return getattr(self.file,name)
    return object.__getattribute__(self,name)

I have an object of this class, a for which the following happens:

>>> hasattr(a,"__iter__")
True
>>> for l in a: print l
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'TmpFile' object is not iterable
>>> for l in a.file: print l
...
>>>

So python sees that a has an __iter__ method, but doesn't think it is iterable. What have I done wrong? This is with python 2.6.4.

Answer 1

There's a subtle implementation detail getting in your way: __iter__ isn't actually an instance method, but a class method. That is, obj.__class__.__iter__(obj) is called, rather than obj.__iter__().

This is due to slots optimizations under the hood, allowing the Python runtime to set up iterators faster. This is needed since it's very important that iterators be as fast as possible.

It's not possible to define __getattribute__ for the underlying class type, so it's not possible to return this method dynamically. This applies to most __metamethods__; you'll need to write an actual wrapper.

Answer 2

Some of the special methods are optimised when a class is created and cannot be added later or overridden by assignment. See the documentation for __getattribute__ which says:

This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions.

What you need to do in this case is provide a direct implementation of __iter__ that forwards the call:

def __iter__(self):
    return self.file.__iter__()