Creating a list of functions in python (python function closure bug?)

Question

I understand functional programming well. I want to create a list of functions that each selects a different element of a list. I have reduced my problem to a simple example. Surely this is a Python bug:

fun_list = []
for i in range(5):
    def fun(e):
        return e[i]
    fun_list.append(fun)

mylist = range(10)
print([f(mylist) for f in fun_list])

"Obviously" it should return [0,1,2,3,4]. It however returns [4, 4, 4, 4, 4]. How can I coerce Python to do the right thing? (Hasn't this been noticed before? Or am I just being thick?)

This is Python 3.4.0 (default, Mar 25 2014, 11:07:05)

Thanks, David

Answer 1

How can I coerce Python to do the right thing?

Here is one approach:

fun_list = []
for i in range(5):
    def fun(e, _ndx=i):
        return e[_ndx]
    fun_list.append(fun)

mylist = range(10)
print([f(mylist) for f in fun_list])

This works because the default value for _ndx is evaluated and saved when the def statement for fun is executed. (In python, def statements are executed.)

Answer 2

Surely this is a Python bug...

This is a misunderstanding of scopes. Since all five instances of fun() are defined in the same scope, they will all have the same value of all names in that scope, including i. In order to fix this you need to separate the value used from the scope containing the loop itself. This can be done by defining the function within a completely different scope.

fun_list = []

def retfun(i):
  def fun(e):
    return e[i]
  return fun

for i in range(5):
  fun_list.append(retfun(i))

mylist = range(10)
print([f(mylist) for f in fun_list])

Answer 3

You can do this using the itemgetter:

from operator import itemgetter
fun_list = []
for i in range(5):
    fun_list.append(itemgetter(i))

mylist = range(10)
print([f(mylist) for f in fun_list])

In your case, you are assigning a function to all elements referencing a global i with the value of i at the time of the call, which is 4 for all calls. You need some sort of currying.

Same without the itemgetter:

def indexer(i): return lambda y: y[i]

fun_list = []
for i in range(5):
    fun_list.append(indexer(i))

mylist = range(10)
print([f(mylist) for f in fun_list])