Keyword argument performance (python)

Question

I am trying to optimise some python code, via testing (timing) various functions using timeit.

I have found that I am getting different speeds depending on whether a variable is a keyword argument or within the function.

That is:

def test_function(A = value()):
    #rest of function....

Is returning a different result than:

def test_function():
    A = value()
    #rest of function ...

I would have figured they would have very similar results - I am guessing I am not understanding / missing something here...

(doing a 10,000 loops for the tests too)

Answer 1

Keyword arguments are evaluated once at function definition time. So in your first example value() is called exactly once, no matter how often you call the test function. If value() is expensive-ish this explains the difference in runtime between the two versions.

Answer 2

There's no reason your two functions should be expected to do the same thing, let alone have the same performance characteristics.

def test_function(A = value()):
    #rest of function....

This function doesn't have a "keyword argument"; there's no such thing. It has an argument with a default value. Any parameter for any function (some recalcitrant built-ins aside) can be passed by keyword or by position, but arguments are not intrinsically "keyword arguments".

The only association between keyword arguments and default values is that when you have multiple arguments with default values, the only way to supply an explicit value to a later argument while accepting the default for an earlier one is to pass the later argument by keyword.

The huge difference between the two functions is that when you declare a default value for A, it's a default value, not some code that will regenerate the value each time if an explicit value isn't provided. When you say this:

def test_function(A = value()):
    #rest of function....

You're setting a default value for A. As in any other context, when you provide a complex expression where Python needs a value, Python will evaluate that expression and then use the resulting value. So when you set the default value for A, at function definition time, it gets set to whatever value() returns at that time. Then that one single value is the default value for A.

def test_function():
    A = value()
    #rest of function ...

In this function, value() is evaluated every time the function is called. So if value() is expensive, then this version will take much longer than the first version. But if value() returns an object which you later mutate, then the default-argument version will be always using the one single object, in whatever state it was in at the time the function was called, while the second version will be constructing a new value every time. Which version you use should be determined by the semantics you want your program to have.

Answer 3

There are discussions as to why this method isn't the best to determine efficacy of approaches, but if you use dis to inspect the bytecode of the functions, you can see that they are structured in different ways, namely that t1 evaluates its default argument at the time it is defined, and therefore does not require it to be redefined on subsequent function calls:

>>> import dis
>>> def t1(A=1):
...   pass
>>> def t2():
....  A=1
>>> dis.dis(t1)
  2           0 LOAD_CONST               0 (None)
              3 RETURN_VALUE        
>>> dis.dis(t2)
  2           0 LOAD_CONST               1 (1)
              3 STORE_FAST               0 (A)
              6 LOAD_CONST               0 (None)
              9 RETURN_VALUE