Python object initialization bug. Or am I misunderstanding how objects work?

Question

  1 import sys
  2 
  3 class dummy(object):
  4     def __init__(self, val):
  5         self.val = val
  6 
  7 class myobj(object):
  8     def __init__(self, resources):
  9         self._resources = resources
 10 
 11 class ext(myobj):
 12     def __init__(self, resources=[]):
 13         #myobj.__init__(self, resources)
 14         self._resources = resources
 15 
 16 one = ext()
 17 one._resources.append(1)
 18 two = ext()
 19 
 20 print one._resources
 21 print two._resources
 22 
 23 sys.exit(0)

This will print the reference to the object assigned to one._resources for both one and two objects. I would think that two would be an empty array as it is clearly setting it as such if it's not defined when creating the object. Uncommenting myobj.__init__(self, resources) does the same thing. Using super(ext, self).__init__(resources) also does the same thing.

The only way I can get it to work is if I use the following:

two = ext(dummy(2))

I shouldn't have to manually set the default value when creating the object to make this work. Or maybe I do. Any thoughts?

I tried this using Python 2.5 and 2.6.

Answer 1

You should change

def __init__(self, resources=[]):
    self._resources = resources

to

def __init__(self, resources=None):
    if resources is None:
        resources = []
    self._resources = resources

and all will be better. This is a detail in the way default arguments are handled if they're mutable. There's some more information in the discussion section of this page.

Answer 2

Your problem is that the default value is evaluated at function definition time. This means that the same list object is shared between instances. See the answer to this question for more discussion.

Answer 3

Please read this answer for a discussion of how to setup a class from __init__(). You have encountered a well-known quirk of Python: you are trying to set up a mutable, and your mutable is being evaluated once when __init__() is compiled. The standard workaround is:

class ext(myobj):
    def __init__(self, resources=None):
        if resources is None:
            resources = []
        #myobj.__init__(self, resources)
        self._resources = resources

Answer 4

From http://docs.python.org/3.1/tutorial/controlflow.html:

The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes.

Answer 5

This is a known Python gotcha.

You have to avoid using a mutable object on the call of a function/method.

The objects that provide the default values are not created at the time that the function/method is called. They are created at the time that the statement that defines the function is executed. (See the discussion at Default arguments in Python: two easy blunders: "Expressions in default arguments are calculated when the function is defined, not when it’s called.")

This behavior is not a wart in the Python language. It really is a feature, not a bug. There are times when you really do want to use mutable default arguments. One thing they can do (for example) is retain a list of results from previous invocations, something that might be very handy.

But for most programmers — especially beginning Pythonistas — this behavior is a gotcha. So for most cases we adopt the following rules.

• Never use a mutable object — that is: a list, a dictionary, or a class instance — as the default value of an argument.
• Ignore rule 1 only if you really, really, REALLY know what you're doing.

So... we plan always to follow rule #1. Now, the question is how to do it... how to code functionF in order to get the behavior that we want.

Fortunately, the solution is straightforward. The mutable objects used as defaults are replaced by None, and then the arguments are tested for None.

---

So how can one do it correctly? One solution is avoid using mutable default values for arguments. But this is hardly satisfactory, as from time to time a new list is a useful default. There are some complex solutions like defining a decorator for functions that deep-copies all arguments. This is an overkill, and the problem can be solved easily as follows:

class ext(myobj):
    def __init__(self, resources=None):
        if not resources:
            resources = []
        self._resources = resources