Python bug? Lazy objects have hidden state

Question

Consider this (Python 3.3):

a=enumerate([2,3,5])
print(list(a))
print(list(a))

Do you really expect two print calls to print different things? Neither did I.

The same thing happens if you replace list with set, tuple or dict. It also happens if you replace enumerate object with map or filter, but curiously, not if you replace it with range.

Maybe it is a feature. But it's very surprising, not documented (at least I haven't been able to find anything about it), and not consistent (range works differently). What do you think?

Answer 1

The behaviour is documented at http://docs.python.org/3/glossary.html#term-iterator

One notable exception is code which attempts multiple iteration passes. ... Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

Answer 2

enumerate() returns an iterator, as do the other calls.. You can only loop through an iterator once; it is then exhausted.

You can create such an iterator yourself with a generator function:

def somelist_generator():
    somelist = [1, 2, 3]
    while somelist:
        yield somelist.pop()

If you were to loop over somelist_generator(), the list somelist would be emptied. You can only do that once, since .pop() removes elements:

>>> it = somelist_generator()
>>> for i in it:
...     print(i)
... 
3
2
1
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

The next() call tries to get another value from the it iterator; it is already empty so the StopIteration exception is raised. That exception signals there are no more elements to be gotten, and that's why you end up with an empty list the second time you try and get anything from an iterator:

>>> list(it)
[]

range() does not return an iterator. It returns a range object instead, which represents a memory-efficient series of numbers; only the start, end and stride need to be stored and everything else can be derived from those 3 points.