- 5.1 Function Definitions
- 5.2 Default Arguments
- 5.3 Variadic Arguments
- 5.4 Keyword Arguments
- 5.5 Variadic Keyword Arguments
- 5.6 Functions Accepting All Inputs
- 5.7 Positional-Only Arguments
- 5.8 Names, Documentation Strings, and Type Hints
- 5.9 Function Application and Parameter Passing
- 5.10 Return Values
- 5.11 Error Handling
- 5.12 Scoping Rules
- 5.13 Recursion
- 5.14 The lambda Expression
- 5.15 Higher-Order Functions
- 5.16 Argument Passing in Callback Functions
- 5.17 Returning Results from Callbacks
- 5.18 Decorators
- 5.19 Map, Filter, and Reduce
- 5.20 Function Introspection, Attributes, and Signatures
- 5.21 Environment Inspection
- 5.22 Dynamic Code Execution and Creation
- 5.23 Asynchronous Functions and await
- 5.24 Final Words: Thoughts on Functions and Composition
This chapter is from the book
This chapter is from the book
This chapter is from the book
5.15 Higher-Order Functions
Python supports the concept of higher-order functions. This means that functions can be passed as arguments to other functions, placed in data structures, and returned by a function as a result. Functions are said to be first-class objects, meaning there is no difference between how you might handle a function and any other kind of data. Here is an example of a function that accepts another function as input and calls it after a time delay—for example, to emulate the performance of a microservice in the cloud:
import time
def after(seconds, func):
time.sleep(seconds)
func()
# Example usage
def greeting():
print('Hello World')
after(10, greeting) # Prints 'Hello World' after 10 seconds
Here, the func argument to after() is an example of what’s known as a callback function. This refers to the fact that the after() function “calls back” to the function supplied as an argument.
When a function is passed as data, it implicitly carries information related to the environment in which the function was defined. For example, suppose the greeting() function makes use of a variable like this:
def main():
name = 'Guido'
def greeting():
print('Hello', name)
after(10, greeting) # Produces: 'Hello Guido'
main()
In this example, the variable name is used by greeting(), but it’s a local variable of the outer main() function. When greeting is passed to after(), the function remembers its environment and uses the value of the required name variable. This relies on a feature known as a closure. A closure is a function along with an environment containing all of the variables needed to execute the function body.
Closures and nested functions are useful when you write code based on the concept of lazy or delayed evaluation. The after() function, shown above, is an illustration of this concept. It receives a function that is not evaluated right away—that only happens at some later point in time. This is a common programming pattern that arises in other contexts. For example, a program might have functions that only execute in response to events— key presses, mouse movement, arrival of network packets, and so on. In all of these cases, function evaluation is deferred until something interesting happens. When the function finally executes, a closure ensures that the function gets everything that it needs.
You can also write functions that create and return other functions. For example:
def make_greeting(name):
def greeting():
print('Hello', name)
return greeting
f = make_greeting('Guido')
g = make_greeting('Ada')
f() # Produces: 'Hello Guido'
g() # Produces: 'Hello Ada'
In this example, the make_greeting() function doesn’t carry out any interesting computations. Instead, it creates and returns a function greeting() that does the actual work. That only happens when that function gets evaluated later.
In this example, the two variables f and g hold two different versions of the greeting() function. Even though the make_greeting() function that created those functions is no longer executing, the greeting() functions still remember the name variable that was defined—it’s part of each function’s closure.
One caution about closures is that binding to variable names is not a “snapshot” but a dynamic process—meaning the closure points to the name variable and the value that it was most recently assigned. This is subtle, but here’s an example that illustrates where trouble can arise:
def make_greetings(names):
funcs = []
for name in names:
func.append(lambda: print('Hello', name))
return funcs
# Try it
a, b, c = make_greetings(['Guido', 'Ada', 'Margaret'])
a() # Prints 'Hello Margaret'
b() # Prints 'Hello Margaret'
c() # Prints 'Hello Margaret'
In this example, a list of different functions is made (using lambda). It may appear as if they are all using a unique value of name, as it changes on each iteration of a for loop. This is not the case. All functions end up using the same value of name —the value it has when the outer make_greetings() function returns.
This is probably unexpected and not what you want. If you want to capture a copy of a variable, capture it as a default argument, as previously described:
def make_greetings(names):
funcs = []
for name in names:
funcs.append(lambda name=name: print('Hello', name))
return funcs
# Try it
a, b, c = make_greetings(['Guido', 'Ada', 'Margaret'])
a() # Prints 'Hello Guido'
b() # Prints 'Hello Ada'
c() # Prints 'Hello Margaret'
In the last two examples, functions have been defined using lambda. This is often used as a shortcut for creating small callback functions. However, it’s not a strict requirement. You could have rewritten it like this:
def make_greetings(names):
funcs = []
for name in names:
def greeting(name=name):
print('Hello', name)
funcs.append(greeting)
return funcs
The choice of when and where to use lambda is one of personal preference and a matter of code clarity. If it makes code harder to read, perhaps it should be avoided.