- 5.1 Introduction
- 5.2 Lists
- 5.3 Tuples
- 5.4 Unpacking Sequences
- 5.5 Sequence Slicing
- 5.6 del Statement
- 5.7 Passing Lists to Functions
- 5.8 Sorting Lists
- 5.9 Searching Sequences
- 5.10 Other List Methods
- 5.11 Simulating Stacks with Lists
- 5.12 List Comprehensions
- 5.13 Generator Expressions
- 5.14 Filter, Map and Reduce
- 5.15 Other Sequence Processing Functions
- 5.16 Two-Dimensional Lists
- 5.17 Intro to Data Science: Simulation and Static Visualizations
- 5.18 Wrap-Up
This chapter is from the book
This chapter is from the book
This chapter is from the book
5.14 Filter, Map and Reduce
The preceding section introduced several functional-style features—list comprehensions, filtering and mapping. Here we demonstrate the built-in filter and map functions for filtering and mapping, respectively. We continue discussing reductions in which you process a collection of elements into a single value, such as their count, total, product, average, minimum or maximum.
Filtering a Sequence’s Values with the Built-In filter Function
Let’s use built-in function filter to obtain the odd values in numbers:
In [1]: numbers = [10, 3, 7, 1, 9, 4, 2, 8, 5, 6] In [2]: def is_odd(x): ...: """Returns True only if x is odd.""" ...: return x % 2 != 0 ...: In [3]: list(filter(is_odd, numbers)) Out[3]: [3, 7, 1, 9, 5]
Like data, Python functions are objects that you can assign to variables, pass to other functions and return from functions. Functions that receive other functions as arguments are a functional-style capability called higher-order functions. For example, filter’s first argument must be a function that receives one argument and returns True if the value should be included in the result. The function is_odd returns True if its argument is odd. The filter function calls is_odd once for each value in its second argument’s iterable (numbers). Higher-order functions may also return a function as a result.
Function filter returns an iterator, so filter’s results are not produced until you iterate through them. This is another example of lazy evaluation. In snippet [3], function list iterates through the results and creates a list containing them. We can obtain the same results as above by using a list comprehension with an if clause:
In [4]: [item for item in numbers if is_odd(item)] Out[4]: [3, 7, 1, 9, 5]
Using a lambda Rather than a Function
For simple functions like is_odd that return only a single expression’s value, you can use a lambda expression (or simply a lambda) to define the function inline where it’s needed—typically as it’s passed to another function:
In [5]: list(filter(lambda x: x % 2 != 0, numbers)) Out[5]: [3, 7, 1, 9, 5]
We pass filter’s return value (an iterator) to function list here to convert the results to a list and display them.
A lambda expression is an anonymous function—that is, a function without a name. In the filter call
filter(lambda x: x % 2 != 0, numbers)
the first argument is the lambda
lambda x: x % 2 != 0
A lambda begins with the lambda keyword followed by a comma-separated parameter list, a colon (:) and an expression. In this case, the parameter list has one parameter named x. A lambda implicitly returns its expression’s value. So any simple function of the form
def function_name(parameter_list):
return expression
may be expressed as a more concise lambda of the form
lambda parameter_list: expression
Mapping a Sequence’s Values to New Values
Let’s use built-in function map with a lambda to square each value in numbers:
In [6]: numbers Out[6]: [10, 3, 7, 1, 9, 4, 2, 8, 5, 6] In [7]: list(map(lambda x: x ** 2, numbers)) Out[7]: [100, 9, 49, 1, 81, 16, 4, 64, 25, 36]
Function map’s first argument is a function that receives one value and returns a new value—in this case, a lambda that squares its argument. The second argument is an iterable of values to map. Function map uses lazy evaluation. So, we pass to the list function the iterator that map returns. This enables us to iterate through and create a list of the mapped values. Here’s an equivalent list comprehension:
In [8]: [item ** 2 for item in numbers] Out[8]: [100, 9, 49, 1, 81, 16, 4, 64, 25, 36]
Combining filter and map
You can combine the preceding filter and map operations as follows:
In [9]: list(map(lambda x: x ** 2,
...: filter(lambda x: x % 2 != 0, numbers)))
...:
Out[9]: [9, 49, 1, 81, 25]
There is a lot going on in snippet [9], so let’s take a closer look at it. First, filter returns an iterable representing only the odd values of numbers. Then map returns an iterable representing the squares of the filtered values. Finally, list uses map’s iterable to create the list. You might prefer the following list comprehension to the preceding snippet:
In [10]: [x ** 2 for x in numbers if x % 2 != 0] Out[10]: [9, 49, 1, 81, 25]
For each value of x in numbers, the expression x ** 2 is performed only if the condition x % 2 != 0 is True.
Reduction: Totaling the Elements of a Sequence with sum
As you know reductions process a sequence’s elements into a single value. You’ve performed reductions with the built-in functions len, sum, min and max. You also can create custom reductions using the functools module’s reduce function. See https://docs.python.org/3/library/functools.html for a code example. When we investigate big data and Hadoop in Chapter 16, we’ll demonstrate MapReduce programming, which is based on the filter, map and reduce operations in functional-style programming.