- 5.1 Function Definitions
- 5.2 Default Arguments
- 5.3 Variadic Arguments
- 5.4 Keyword Arguments
- 5.5 Variadic Keyword Arguments
- 5.6 Functions Accepting All Inputs
- 5.7 Positional-Only Arguments
- 5.8 Names, Documentation Strings, and Type Hints
- 5.9 Function Application and Parameter Passing
- 5.10 Return Values
- 5.11 Error Handling
- 5.12 Scoping Rules
- 5.13 Recursion
- 5.14 The lambda Expression
- 5.15 Higher-Order Functions
- 5.16 Argument Passing in Callback Functions
- 5.17 Returning Results from Callbacks
- 5.18 Decorators
- 5.19 Map, Filter, and Reduce
- 5.20 Function Introspection, Attributes, and Signatures
- 5.21 Environment Inspection
- 5.22 Dynamic Code Execution and Creation
- 5.23 Asynchronous Functions and await
- 5.24 Final Words: Thoughts on Functions and Composition
This chapter is from the book
This chapter is from the book
This chapter is from the book
5.16 Argument Passing in Callback Functions
One challenging problem with callback functions is that of passing arguments to the supplied function. Consider the after() function written earlier:
import time
def after(seconds, func):
time.sleep(seconds)
func()
In this code, func() is hardwired to be called with no arguments. If you want to pass extra arguments, you’re out of luck. For example, you might try this:
def add(x, y):
print(f'{x} + {y} -> {x+y}')
return x + y
after(10, add(2, 3)) # Fails: add() called immediately
In this example, the add(2, 3) function runs immediately, returning 5. The after() function then crashes 10 seconds later as it tries to execute 5(). That is definitely not what you intended. Yet there seems to be no obvious way to make it work if add() is called with its desired arguments.
This problem hints towards a greater design issue concerning the use of functions and functional programming in general—function composition. When functions are mixed together in various ways, you need to think about how function inputs and outputs connect together. It is not always simple.
In this case, one solution is to package up computation into a zero-argument function using lambda. For example:
after(10, lambda: add(2, 3))
A small zero-argument function like this is sometimes known as a thunk. Basically, it’s an expression that will be evaluated later when it’s eventually called as a zero-argument function. This can be a general-purpose way to delay the evaluation of any expression to a later point in time: put the expression in a lambda and call the function when you actually need the value.
As an alternative to using lambda, you could use functools.partial() to create a partially evaluated function like this:
from functools import partial after(10, partial(add, 2, 3))
partial() creates a callable where one or more of the arguments have already been specified and are cached. It can be a useful way to make nonconforming functions match expected calling signatures in callbacks and other applications. Here are a few more examples of using partial():
def func(a, b, c, d):
print(a, b, c, d)
f = partial(func, 1, 2) # Fix a=1, b=2
f(3, 4) # func(1, 2, 3, 4)
f(10, 20) # func(1, 2, 10, 20)
g = partial(func, 1, 2, d=4) # Fix a=1, b=2, d=4
g(3) # func(1, 2, 3, 4)
g(10) # func(1, 2, 10, 4)
partial() and lambda can be used for similar purposes, but there is an important semantic distinction between the two techniques. With partial(), the arguments are evaluated and bound at the time the partial function is first defined. With a zero-argument lambda, the arguments are evaluated and bound when the lambda function actually executes later (the evaluation of everything is delayed). To illustrate:
>>> def func(x, y): ... return x + y ... >>> a = 2 >>> b = 3 >>> f = lambda: func(a, b) >>> g = partial(func, a, b) >>> a = 10 >>> b = 20 >>> f() # Uses current values of a, b 30 >>> g() # Uses initial values of a, b 5 >>>
Since partials are fully evaluated, the callables created by partial() are objects that can be serialized into bytes, saved in files, and even transmitted across network connections (for example, using the pickle standard library module). This is not possible with a lambda function. Thus, in applications where functions are passed around, possibly to Python interpreters running in different processes or on different machines, you’ll find partial() to be a bit more adaptable.
As an aside, partial function application is closely related to a concept known as currying. Currying is a functional programming technique where a multiple-argument function is expressed as a chain of nested single-argument functions. Here is an example:
# Three-argument function
def f(x, y, z):
return x + y + z
# Curried version
def fc(x):
return lambda y: (lambda z: x + y + z)
# Example use
a = f(2, 3, 4) # 3-argument function
b = fc(2)(3)(4) # Curried version
This is not a common Python programming style and there are few practical reasons for doing it. However, sometimes you’ll hear the word “currying” thrown about in conversations with coders who’ve spent too much time warping their brains with things like lambda calculus. This technique of handling multiple arguments is named in honor of the famous logician Haskell Curry. Knowing what it is might be useful—should you stumble into a group of functional programmers having a heated flamewar at a social event.
Getting back to the original problem of argument passing, another option for passing arguments to a callback function is to accept them separately as arguments to the outer calling function. Consider this version of the after() function:
def after(seconds, func, *args):
time.sleep(seconds)
func(*args)
after(10, add, 2, 3) # Calls add(2, 3) after 10 seconds
You will notice that passing keyword arguments to func() is not supported. This is by design. One issue with keyword arguments is that the argument names of the given function might clash with argument names already in use (that is, seconds and func). Keyword arguments might also be reserved for specifying options to the after() function itself. For example:
def after(seconds, func, *args, debug=False):
time.sleep(seconds)
if debug:
print('About to call', func, args)
func(*args)
All is not lost, however. If you need to specify keyword arguments to func(), you can still do it using partial(). For example:
after(10, partial(add, y=3), 2)
If you wanted the after() function to accept keyword arguments, a safe way to do it might be to use positional-only arguments. For example:
def after(seconds, func, debug=False, /, *args, **kwargs):
time.sleep(seconds)
if debug:
print('About to call', func, args, kwargs)
func(*args, **kwargs)
after(10, add, 2, y=3)
Another possibly unsettling insight is that after() actually represents two different function calls merged together. Perhaps the problem of passing arguments can be decomposed into two functions like this:
def after(seconds, func, debug=False):
def call(*args, **kwargs):
time.sleep(seconds)
if debug:
print('About to call', func, args, kwargs)
func(*args, **kwargs)
return call
after(10, add)(2, y=3)
Now, there are no conflicts whatsoever between the arguments to after() and the arguments to func. However, there is a chance that doing this will introduce a conflict between you and your coworkers.