Why aren't python nested functions called closures?

Question

I have seen and used nested functions in Python, and they match the definition of a closure. So why are they called nested functions instead of closures?

Are nested functions not closures because they are not used by the external world?

UPDATE: I was reading about closures and it got me thinking about this concept with respect to Python. I searched and found the article mentioned by someone in a comment below, but I couldn't completely understand the explanation in that article, so that is why I am asking this question.

Answer 1

A closure occurs when a function has access to a local variable from an enclosing scope that has finished its execution.

def make_printer(msg):
    def printer():
        print msg
    return printer

printer = make_printer('Foo!')
printer()

When make_printer is called, a new frame is put on the stack with the compiled code for the printer function as a constant and the value of msg as a local. It then creates and returns the function. Because the function printer references the msg variable, it is kept alive after the make_printer function has returned.

So, if your nested functions don't

1. access variables that are local to enclosing scopes,
2. do so when they are executed outside of that scope,

then they are not closures.

Here's an example of a nested function which is not a closure.

def make_printer(msg):
    def printer(msg=msg):
        print msg
    return printer

printer = make_printer("Foo!")
printer()  #Output: Foo!

Here, we are binding the value to the default value of a parameter. This occurs when the function printer is created and so no reference to the value of msg external to printer needs to be maintained after make_printer returns. msg is just a normal local variable of the function printer in this context.

Answer 2

The question has already been answered by aaronasterling

However, someone might be interested in how the variables are stored under the hood.

Before coming to the snippet:

Closures are functions that inherit variables from their enclosing environment. When you pass a function callback as an argument to another function that will do I/O, this callback function will be invoked later, and this function will — almost magically — remember the context in which it was declared, along with all the variables available in that context.

• If a function does not use free variables it doesn't form a closure.

• If there is another inner level which uses free variables -- all previous levels save the lexical environment ( example at the end )

• function attributes func_closure in python < 3.X or __closure__ in python > 3.X save the free variables.

• Every function in python has this closure attributes, but it doesn't save any content if there is no free variables.

example: of closure attributes but no content inside as there is no free variable.

>>> def foo():
...     def fii():
...         pass
...     return fii
...
>>> f = foo()
>>> f.func_closure
>>> 'func_closure' in dir(f)
True
>>>

NB: FREE VARIABLE IS MUST TO CREATE A CLOSURE.

I will explain using the same snippet as above:

>>> def make_printer(msg):
...     def printer():
...         print msg
...     return printer
...
>>> printer = make_printer('Foo!')
>>> printer()  #Output: Foo!

And all Python functions have a closure attribute so let's examine the enclosing variables associated with a closure function.

Here is the attribute func_closure for the function printer

>>> 'func_closure' in dir(printer)
True
>>> printer.func_closure
(<cell at 0x108154c90: str object at 0x108151de0>,)
>>>

The closure attribute returns a tuple of cell objects which contain details of the variables defined in the enclosing scope.

The first element in the func_closure which could be None or a tuple of cells that contain bindings for the function’s free variables and it is read-only.

>>> dir(printer.func_closure[0])
['__class__', '__cmp__', '__delattr__', '__doc__', '__format__', '__getattribute__',
 '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', 
 '__setattr__',  '__sizeof__', '__str__', '__subclasshook__', 'cell_contents']
>>>

Here in the above output you can see cell_contents, let's see what it stores:

>>> printer.func_closure[0].cell_contents
'Foo!'    
>>> type(printer.func_closure[0].cell_contents)
<type 'str'>
>>>

So, when we called the function printer(), it accesses the value stored inside the cell_contents. This is how we got the output as 'Foo!'

Again I will explain using the above snippet with some changes:

 >>> def make_printer(msg):
 ...     def printer():
 ...         pass
 ...     return printer
 ...
 >>> printer = make_printer('Foo!')
 >>> printer.func_closure
 >>>

In the above snippet, I din't print msg inside the printer function, so it doesn't create any free variable. As there is no free variable, there will be no content inside the closure. Thats exactly what we see above.

Now I will explain another different snippet to clear out everything Free Variable with Closure:

>>> def outer(x):
...     def intermediate(y):
...         free = 'free'
...         def inner(z):
...             return '%s %s %s %s' %  (x, y, free, z)
...         return inner
...     return intermediate
...
>>> outer('I')('am')('variable')
'I am free variable'
>>>
>>> inter = outer('I')
>>> inter.func_closure
(<cell at 0x10c989130: str object at 0x10c831b98>,)
>>> inter.func_closure[0].cell_contents
'I'
>>> inn = inter('am')

So, we see that a func_closure property is a tuple of closure cells, we can refer them and their contents explicitly -- a cell has property "cell_contents"

>>> inn.func_closure
(<cell at 0x10c9807c0: str object at 0x10c9b0990>, 
 <cell at 0x10c980f68: str object at   0x10c9eaf30>, 
 <cell at 0x10c989130: str object at 0x10c831b98>)
>>> for i in inn.func_closure:
...     print i.cell_contents
...
free
am 
I
>>>

Here when we called inn, it will refer all the save free variables so we get I am free variable

>>> inn('variable')
'I am free variable'
>>>

Answer 3

Python has a weak support for closure. To see what I mean take the following example of a counter using closure with JavaScript:

function initCounter(){
    var x = 0;
    function counter  () {
        x += 1;
        console.log(x);
    };
    return counter;
}

count = initCounter();

count(); //Prints 1
count(); //Prints 2
count(); //Prints 3

Closure is quite elegant since it gives functions written like this the ability to have "internal memory". As of Python 2.7 this is not possible. If you try

def initCounter():
    x = 0;
    def counter ():
        x += 1 ##Error, x not defined
        print x
    return counter

count = initCounter();

count(); ##Error
count();
count();

You'll get an error saying that x is not defined. But how can that be if it has been shown by others that you can print it? This is because of how Python it manages the functions variable scope. While the inner function can read the outer function's variables, it cannot write them.

This is a shame really. But with just read-only closure you can at least implement the function decorator pattern for which Python offers syntactic sugar.

Update

As its been pointed out, there are ways to deal with python's scope limitations and I'll expose some.

1. Use the global keyword (in general not recommended).

2. In Python 3.x, use the nonlocal keyword (suggested by @unutbu and @leewz)

3. Define a simple modifiable class Object

class Object(object):
    pass

and create an Object scope within initCounter to store the variables

def initCounter ():
    scope = Object()
    scope.x = 0
    def counter():
        scope.x += 1
        print scope.x

    return counter

Since scope is really just a reference, actions taken with its fields do not really modify scope itself, so no error arises.

4. An alternative way, as @unutbu pointed out, would be to define each variable as an array (x = [0]) and modify it's first element (x[0] += 1). Again no error arises because x itself is not modified.

5. As suggested by @raxacoricofallapatorius, you could make x a property of counter

def initCounter ():

    def counter():
        counter.x += 1
        print counter.x

    counter.x = 0
    return counter

Answer 4

Python 2 didn't have closures - it had workarounds that resembled closures.

There are plenty of examples in answers already given - copying in variables to the inner function, modifying an object on the inner function, etc.

In Python 3, support is more explicit - and succinct:

def closure():
    count = 0
    def inner():
        nonlocal count
        count += 1
        print(count)
    return inner

Usage:

start = closure()
start() # prints 1
start() # prints 2
start() # prints 3

The nonlocal keyword binds the inner function to the outer variable explicitly mentioned, in effect enclosing it. Hence more explicitly a 'closure'.

Answer 5

I had a situation where I needed a separate but persistent name space. I used classes. I don't otherwise. Segregated but persistent names are closures.

>>> class f2:
...     def __init__(self):
...         self.a = 0
...     def __call__(self, arg):
...         self.a += arg
...         return(self.a)
...
>>> f=f2()
>>> f(2)
2
>>> f(2)
4
>>> f(4)
8
>>> f(8)
16

# **OR**
>>> f=f2() # **re-initialize**
>>> f(f(f(f(2)))) # **nested**
16

# handy in list comprehensions to accumulate values
>>> [f(i) for f in [f2()] for i in [2,2,4,8]][-1] 
16

Answer 6

def nested1(num1): 
    print "nested1 has",num1
    def nested2(num2):
        print "nested2 has",num2,"and it can reach to",num1
        return num1+num2    #num1 referenced for reading here
    return nested2

Gives:

In [17]: my_func=nested1(8)
nested1 has 8

In [21]: my_func(5)
nested2 has 5 and it can reach to 8
Out[21]: 13

This is an example of what a closure is and how it can be used.

Answer 7

I'd like to offer another simple comparison between python and JS example, if this helps make things clearer.

JS:

function make () {
  var cl = 1;
  function gett () {
    console.log(cl);
  }
  function sett (val) {
    cl = val;
  }
  return [gett, sett]
}

and executing:

a = make(); g = a[0]; s = a[1];
s(2); g(); // 2
s(3); g(); // 3

Python:

def make (): 
  cl = 1
  def gett ():
    print(cl);
  def sett (val):
    cl = val
  return gett, sett

and executing:

g, s = make()
g() #1
s(2); g() #1
s(3); g() #1

Reason: As many others said above, in python, if there is an assignment in the inner scope to a variable with the same name, a new reference in the inner scope is created. Not so with JS, unless you explicitly declare one with the var keyword.