Why do list and tuple change the output of enumerate?

Question

I was surprised at the following behavior:

>>> a=['a','b','c','d','e','f','g']
>>> en=enumerate(a)
>>> list(en)
[(0, 'a'), (1, 'b'), (2, 'c'), (3, 'd'), (4, 'e'), (5, 'f'), (6, 'g')]
>>> list(en)
[]
>>> # en changed?   let's try that again!
>>> a=['a','b','c','d','e','f','g']
>>> en=enumerate(a)
>>> tuple(en)
((0, 'a'), (1, 'b'), (2, 'c'), (3, 'd'), (4, 'e'), (5, 'f'), (6, 'g'))
>>> tuple(en)
()
>>> # en changes again?

I was presuming that list() would just construct a list from the data in en, and would not change it in any way. The same with tuple(). But they both affect en in a "profound" way. Am I misunderstanding something about list() and tuple(), or am I misunderstanding enumerate()? Thanks.

Answer 1

There's nothing special about list and tuple, either. This will happen however you consume en, including via your own hand-written for loop:

>>> a=['a','b','c','d','e','f','g']
>>> en=enumerate(a)
>>> en
<enumerate object at 0x21172d0>
>>> for index, elem in en:
...   print index, elem
... 
0 a
1 b
2 c
3 d
4 e
5 f
6 g
>>> for index, elem in en:
...   print index, elem
... 
>>> 

Note that en is an "enumerate object", not a list or tuple, and that printing it didn't attempt to print its contents. This is because enumerate doesn't actually produce a data structure containing all the same data as a plus the index values. Instead it returns a small object that remembers internally which container it was iterating over, where it was up to, and what count it had reached. That's all you need to produce the "next" value, and so it can be iterated over even though it's not a container as such.

The reason for this is that people almost never store the result of enumerate, it's usually called to immediately iterate over it with a for loop. For that purpose, it would be wasteful of time and memory to go to the effort of building a copy of all the data and hold all the indexes in memory at once. Producing them as you go is enough.

If you do need to store the resulting data from enumerate to use more than once (or to use somewhere else than where you generated it), then you will need that copy. The easiest way to get that is actually to do something like en = list(enumerate(a)). Then you can use en exactly as you were expecting.

Answer 2

I think you are misunderstanding enumerate().

>>> en=enumerate(a)

Is a generator/iterator, not a data structure. Once you run through 'en' once, by creating the list, the generator has run and there's nothing left. Trying list(en) again is trying to iterate through an iterator that has reached it's end, so there's nothing there.

Answer 3

enumerate returns an iterator, not an iterable like a list. This means that list and tuple can only get all of the elements of enumerate's return value by repeatedly calling its next() method until the iterator is exhausted, after which the iterator, like all other iterators in Python, will not be able to produce any more elements.

Answer 4

You may not be completely understanding enumerate() (or, I could be misunderstanding the question). Check out the help(enumerate) docs (specifically the part about yielding pairs of index and value:

Help on class enumerate in module __builtin__:

class enumerate(object)
 |  enumerate(iterable[, start]) -> iterator for index, value of iterable
 |
 |  Return an enumerate object.  iterable must be another object that supports
 |  iteration.  The enumerate object yields pairs containing a count (from
 |  start, which defaults to zero) and a value yielded by the iterable argument.
 |  enumerate is useful for obtaining an indexed list:
 |      (0, seq[0]), (1, seq[1]), (2, seq[2]), ...
 |
 |  Methods defined here:
 |
 |  __getattribute__(...)
 |      x.__getattribute__('name') <==> x.name
 |
 |  __iter__(...)
 |      x.__iter__() <==> iter(x)
 |
 |  next(...)
 |      x.next() -> the next value, or raise StopIteration
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  __new__ = <built-in method __new__ of type object>
 |      T.__new__(S, ...) -> a new object with type S, a subtype of T