- Item 30: Know That Function Arguments Can Be Mutated
- Item 31: Return Dedicated Result Objects Instead of Requiring Function Callers to Unpack More Than Three Variables
- Item 32: Prefer Raising Exceptions to Returning None
- Item 33: Know How Closures Interact with Variable Scope and nonlocal
- Item 34: Reduce Visual Noise with Variable Positional Arguments
- Item 35: Provide Optional Behavior with Keyword Arguments
- Item 36: Use None and Docstrings to Specify Dynamic Default Arguments
- Item 37: Enforce Clarity with Keyword-Only and Positional-Only Arguments
- Item 38: Define Function Decorators with functools.wraps
- Item 39: Prefer functools.partial over lambda Expressions for Glue Functions
This chapter is from the book
This chapter is from the book
This chapter is from the book
Item 39: Prefer functools.partial over lambda Expressions for Glue Functions
Many APIs in Python accept simple functions as part of their interface (see Item 100: “Sort by Complex Criteria Using the key Parameter,” Item 27: “Prefer defaultdict over setdefault to Handle Missing Items in Internal State,” and Item 24: “Consider itertools for Working with Iterators and Generators”). However, these interfaces can cause friction because they might fall short of your needs.
For example, the reduce function from the functools built-in module allows you to calculate one result from a near-limitless iterable of values. Here, I use reduce to calculate the sum of many log-scaled numbers (which effectively multiplies them):
def log_sum(log_total, value):
log_value = math.log(value)
return log_total + log_value
result = functools.reduce(log_sum, [10, 20, 40], 0)
print(math.exp(result))
>>>
8000.0
The problem is that you don’t always have a function like log_sum that exactly matches the function signature required by reduce. For example, imagine that you simply had the parameters reversed—since it’s an arbitrary choice anyway—with value first and log_total second. How could you easily fit this function to the required interface?
def log_sum_alt(value, log_total): # Changed
...
One solution is to define a lambda function in an expression to reorder the input arguments to match what’s required by reduce:
result = functools.reduce(
lambda total, value: log_sum_alt(value, total), # Reordered
[10, 20, 40],
0,
)
For one-offs, creating a lambda like this is fine. But if you find yourself doing this repeatedly and copying code, it’s worth defining another helper function with reordered arguments that you can call multiple times:
def log_sum_for_reduce(total, value):
return log_sum_alt(value, total)
Another situation where function interfaces are mismatched is when you need to pass along some additional information for use in processing. For example, say I want to choose the base for the logarithm instead of always using natural log:
def logn_sum(base, logn_total, value): # New first parameter
logn_value = math.log(value, base)
return logn_total + logn_value
In order to pass this function to reduce, I need to somehow provide the base argument for every call. But reduce doesn’t give me a way to do this easily. Again, lambda can help here by allowing me to specify one parameter and pass through the rest. Here, I always provide 10 as the first argument to logn_sum in order to calculate a base-10 logarithm:
result = functools.reduce(
lambda total, value: logn_sum(10, total, value), # Changed
[10, 20, 40],
0,
)
print(math.pow(10, result))
>>>
8000.000000000004
This pattern of pinning some arguments to specific values while allowing the rest of them to be passed normally is quite common with functional-style code. This technique is often called Currying or partial application. The functools built-in module provides the partial function to make this easy and more readable. It takes the function to partially apply as the first argument followed by the pinned positional arguments:
result = functools.reduce(
functools.partial(logn_sum, 10), # Changed
[10, 20, 40],
0,
)
partial also allows you to easily pin keyword arguments (see Item 35: “Provide Optional Behavior with Keyword Arguments” and Item 37: “Enforce Clarity with Keyword-Only and Positional-Only Arguments” for background). For example, imagine that the logn_sum function accepts base as a keyword-only argument, like this:
def logn_sum_last(logn_total, value, *, base=10): # New kwarg
logn_value = math.log(value, base)
return logn_total + logn_value
Here, I use partial to pin the value of base to Euler’s number:
import math
log_sum_e = functools.partial(logn_sum_last,
base=math.e) # Pinned `base`
print(log_sum_e(3, math.e**10))
>>>
13.0
Achieving the same behavior is possible with a lambda expression, but it’s verbose and error prone:
log_sum_e_alt = lambda *a, base=math.e, **kw: logn_sum_last(*a, base=base, **kw)
partial also allows you to inspect which arguments have already been supplied, and the function being wrapped, which can be helpful for debugging:
print(log_sum_e.args, log_sum_e.keywords, log_sum_e.func)
>>>
() {'base': 2.718281828459045} <function logn_sum_last at
➥0x1033534c0>
In general, you should prefer using partial when it satisfies your use case because of these extra niceties. However, partial can’t be used to reorder the parameters altogether, so that’s one situation where lambda is preferable.
In many cases, a lambda or partial instance is still not enough, especially if you need to access or modify state as part of a simple function interface. Luckily, Python provides additional facilities, including closures, to make this possible (see Item 33: “Know How Closures Interact with Variable Scope and nonlocal” and Item 48: “Accept Functions Instead of Classes for Simple Interfaces”).
Things to Remember
lambda expressions can succinctly make two function interfaces compatible by reordering arguments or pinning certain parameter values.
The partial function from the functools built-in is a general tool for creating functions with pinned positional and keyword arguments.
Use lambda instead of partial if you need to reorder the arguments of a wrapped function.