Python: next() function

Question

I'm learning Python from a book, and I came across this example:

M = [[1,2,3],
     [4,5,6],
     [7,8,9]]

G = (sum(row) for row in M) # create a generator of row sums
next(G) # Run the iteration protocol

Since I'm an absolute beginner, and the author hasn't provided any explanation of the example or the next() function, I don't understand what the code is doing.

Answer 1

The expression (sum(row) for row in M) creates what's called a generator. This generator will evaluate the expression (sum(row)) once for each row in M. However, the generator doesn't do anything yet, we've just set it up.

The statement next(G) actually runs the generator on M. So, if you run next(G) once, you'll get the sum of the first row. If you run it again, you'll get the sum of the second row, and so on.

>>> M = [[1,2,3],
...      [4,5,6],
...      [7,8,9]]
>>> 
>>> G = (sum(row) for row in M) # create a generator of row sums
>>> next(G) # Run the iteration protocol
6
>>> next(G)
15
>>> next(G)
24

See also:

• Documentation on generators
• Documentation on yield expressions (with some info about generators)

Answer 2

If you've come that far, then you should already know how a common for-in statement works.

The following statement:

for row in M: print row

would see M as a sequence of 3 rows (sub sequences) consisting of 3 items each, and iterate through M, outputting each row on the matrix:

[1, 2, 3]
[4, 5, 6]
[7, 8, 9]

You knew that, well...

You can see Generators just as some syntactic sugar around for-in loops. Forget about the sum() call, and type something like this on IDLE:

G = (row for row in M)
print G
for a in G: print a

You see, the Generator cannot be directly represented as text, not just as a sequence can be. But, you can iterate through a Generator as if it were a sequence.

You'll find some big differences then, but the basics are that you can use a generator not to return just the value of each item in the sequence, but the result of any expression. In the tutorial's example, the expression is sum(row).

Try the following and see what happens:

G = ("("+str(row[2])+";"+str(row[1])+";"+str(row[0])+")" for row in M)
G.next()
G.next()
G.next()

Answer 3

To Understand how does this code work,you need to understand:

1)iterable object?

2)iterator?

3)Iteration protocol

4)generator?

5)How does generator work?

Let me give glimpse of each of these :

Iterable : An iterable is any object in Python which has an iter or a getitem method defined which returns an iterator or can take indexes.Basically object on which we can run loop. eg:

>>> for i in [1,2,3]:
print(i)

1 2 3

here we have list as an iterable object,whose items can be indexed and retrieved using index.

>>> for i in {x:1,y:2}:
print(x)

x y

here we have dictionary as iterable object, it loops over its keys.

Iterator:An iterator is any object in Python which has a next (Python2) or next method defined. That’s it. That’s an iterator.

Iteration protocol:The built-in function iter takes an iterable object and returns an iterator.

>>> x = iter([1, 2, 3])
>>> x
<listiterator object at 0x1004ca850>
>>> x.next()
1
>>> x.next()
2
>>> x.next()
3
>>> x.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration

Generator:Generators are iterators, but you can only iterate over them once. It’s because they do not store all the values in memory, they generate the values on the fly.

for eg:

def yrange(n):
i = 0
while i < n:
    yield i
    i += 1

Each time the yield statement is executed the function generates a new value.

>>> y = yrange(3)
>>> y
<generator object yrange at 0x401f30>
>>> y.next()
0
>>> y.next()
1
>>> y.next()
2
>>> y.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration

So a generator is also an iterator.

Note: Each time next() is called ,it resumes where it left off(it remembers all the data values and which statement was last executed). Another key feature is that the local variables and execution state are automatically saved between calls.

Let's understand this with example:

>>> def foo():
     print "begin"
     for i in range(3):
         print "before yield", i
         yield i
         print "after yield", i
         print "end"

>>> f = foo()
>>> f.next()
begin
before yield 0
0
>>> f.next()
after yield 0
before yield 1
1
>>> f.next()
after yield 1
before yield 2
2
>>> f.next()
after yield 2
end
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration

Now let's come to your problem:

M = [[1,2,3],       #M is iterable object 
 [4,5,6],
 [7,8,9]]

G = (sum(row) for row in M) # creates a generator of row sums 
next(G) # Run the iteration protocol