Writing (and not) to global variable in Python

Question

Coming from much less dynamic C++, I have some trouble understanding the behaviour of this Python (2.7) code.

Note: I am aware that this is bad programming style / evil, but I would like to understand it non the less.

vals = [1,2,3]

def f():
    vals[0] = 5
    print 'inside', vals

print 'outside', vals
f()
print 'outside', vals

This code runs without error, and f manipulates the (seemingly) global list. This is contrary to my prior understanding that global variables that are to be manipulated (and not only read) in a function must be declared as global ....

On the other hand, if I replace vals[0] = 5 with vals += [5,6], execution fails with an UnboundLocalError unless I add a global vals to f. This is what I would have expected to happen in the first case as well.

Could you explain this behaviour?

Why can I manipulate vals in the first case? Why does the second type of manipulation fail while the first does not?

Update: It was remarked in a comment that vals.extend(...) works without global. This adds to my confusion - why is += treated differently from a call to extend?

Answer 1

global is only needed when you are trying to change the object which the variable references. Because vals[0] = 5 changes the actual object rather than the reference, no error is raised. However, with vals += [5, 6], the interpreter tries to find a local variable because it can't change the global variable.

The confusing thing is that using the += operator with list modifies the original list, like vals[0] = 5. And whereas vals += [5, 6] fails, vals.extend([5, 6]) works. We can enlist the help of dis.dis to lend us some clues.

>>> def a(): v[0] = 1
>>> def b(): v += [1]
>>> def c(): v.extend([1])
>>> import dis
>>> dis.dis(a)
  1           0 LOAD_CONST               1 (1)
              3 LOAD_GLOBAL              0 (v)
              6 LOAD_CONST               2 (0)
              9 STORE_SUBSCR        
             10 LOAD_CONST               0 (None)
             13 RETURN_VALUE        
>>> dis.dis(b)
  1           0 LOAD_FAST                0 (v)
              3 LOAD_CONST               1 (1)
              6 BUILD_LIST               1
              9 INPLACE_ADD         
             10 STORE_FAST               0 (v)
             13 LOAD_CONST               0 (None)
             16 RETURN_VALUE        
d
>>> dis.dis(c)
  1           0 LOAD_GLOBAL              0 (v)
              3 LOAD_ATTR                1 (extend)
              6 LOAD_CONST               1 (1)
              9 BUILD_LIST               1
             12 CALL_FUNCTION            1
             15 POP_TOP             
             16 LOAD_CONST               0 (None)
             19 RETURN_VALUE

We can see that functions a and c use LOAD_GLOBAL, whereas b tries to use LOAD_FAST. We can see now why using += won't work - the interpreter tries to load v as a local variable because of it's default behaviour with in-place addition. Because it can't know whether v is a list or not, it essentially assumes that the line means the same as v = v + [1].

Answer 2

global is needed when you want to assign to a variable in the outer scope. If you don't use global, Python will consider vals as a local variable when doing assignments.

+= is an assignment (an augmented assignment) and vals += [5, 6] is equivalent to reading vals, then append [5, 6] to that value and assign the resulting list back to the original vals. Because vals += [5,6] has no global statement, Python sees the assignment and treats vals as local. You didn't create a local variable called vals but you try to append to it and from here the UnboundLocalError.

But for reading it is not necessary to use global. The variable will be looked up locally first then, if it's not found in the local scope, it's looked up in the outer scope and so on. And since you are dealing with a reference type you get back a reference when you do the read. You can change the content of the object trough that reference.

That's why .extend() works (because it's called on the reference and acts on the object itself) while vals += [5, 6] fails (because vals is neither local nor marked global).

Here is a modified example to try out (using a local vals clears the UnboundLocalError):

vals = [1, 2, 3]

def f():
    vals = []
    vals += [5,6]
    print 'inside', vals

print 'outside', vals
f()
print 'outside', vals

Answer 3

As long as you do not change object reference, Python will preserve global object. Compare

In [114]: vals = [1,2,3]

In [116]: id(vals)
Out[116]: 144255596

In [118]: def func():
    vals[0] = 5
    return id(vals)
   .....: 

In [119]: func()
Out[119]: 144255596

In [120]: def func_update():
    vals = vals
    return id(vals)
   .....: 

In [121]: func_update()
---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
/homes/markg/<ipython-input-121-f1149c600a85> in <module>()
----> 1 func_update()

/homes/markg/<ipython-input-120-257ba6ff792a> in func_update()
      1 def func_update():
----> 2     vals = vals
      3     return id(vals)

UnboundLocalError: local variable 'vals' referenced before assignment

The moment you try assignment, Python regards vals as local variable - and (oops) it's not there!