- 5.1 Introduction
- 5.2 Lists
- 5.3 Tuples
- 5.4 Unpacking Sequences
- 5.5 Sequence Slicing
- 5.6 del Statement
- 5.7 Passing Lists to Functions
- 5.8 Sorting Lists
- 5.9 Searching Sequences
- 5.10 Other List Methods
- 5.11 Simulating Stacks with Lists
- 5.12 List Comprehensions
- 5.13 Generator Expressions
- 5.14 Filter, Map and Reduce
- 5.15 Other Sequence Processing Functions
- 5.16 Two-Dimensional Lists
- 5.17 Intro to Data Science: Simulation and Static Visualizations
- 5.18 Wrap-Up
This chapter is from the book
This chapter is from the book
This chapter is from the book
5.17 Intro to Data Science: Simulation and Static Visualizations
The last few chapters’ Intro to Data Science sections discussed basic descriptive statistics. Here, we focus on visualizations, which help you “get to know” your data. Visualizations give you a powerful way to understand data that goes beyond simply looking at raw data.
We use two open-source visualization libraries—Seaborn and Matplotlib—to display static bar charts showing the final results of a six-sided-die-rolling simulation. The Seaborn visualization library is built over the Matplotlib visualization library and simplifies many Matplotlib operations. We’ll use aspects of both libraries, because some of the Seaborn operations return objects from the Matplotlib library. In the next chapter’s Intro to Data Science section, we’ll make things “come alive” with dynamic visualizations.
5.17.1 Sample Graphs for 600, 60,000 and 6,000,000 Die Rolls
The screen capture below shows a vertical bar chart that for 600 die rolls summarizes the frequencies with which each of the six faces appear, and their percentages of the total. Seaborn refers to this type of graph as a bar plot:
)
Here we expect about 100 occurrences of each die face. However, with such a small number of rolls, none of the frequencies is exactly 100 (though several are close) and most of the percentages are not close to 16.667% (about 1/6th). As we run the simulation for 60,000 die rolls, the bars will become much closer in size. At 6,000,000 die rolls, they’ll appear to be exactly the same size. This is the “law of large numbers” at work. The next chapter will show the lengths of the bars changing dynamically.
We’ll discuss how to control the plot’s appearance and contents, including:
the graph title inside the window (Rolling a Six-Sided Die 600 Times),
the descriptive labels Die Value for the x-axis and Frequency for the y-axis,
the text displayed above each bar, representing the frequency and percentage of the total rolls, and
the bar colors.
We’ll use various Seaborn default options. For example, Seaborn determines the text labels along the x-axis from the die face values 1–6 and the text labels along the y-axis from the actual die frequencies. Behind the scenes, Matplotlib determines the positions and sizes of the bars, based on the window size and the magnitudes of the values the bars represent. It also positions the Frequency axis’s numeric labels based on the actual die frequencies that the bars represent. There are many more features you can customize. You should tweak these attributes to your personal preferences.
The first screen capture below shows the results for 60,000 die rolls—imagine trying to do this by hand. In this case, we expect about 10,000 of each face. The second screen capture below shows the results for 6,000,000 rolls—surely something you’d never do by hand! In this case, we expect about 1,000,000 of each face, and the frequency bars appear to be identical in length (they’re close but not exactly the same length). Note that with more die rolls, the frequency percentages are much closer to the expected 16.667%.
)
5.17.2 Visualizing Die-Roll Frequencies and Percentages
In this section, you’ll interactively develop the bar plots shown in the preceding section.
Launching IPython for Interactive Matplotlib Development
IPython has built-in support for interactively developing Matplotlib graphs, which you also need to develop Seaborn graphs. Simply launch IPython with the command:
ipython --matplotlib
Importing the Libraries
First, let’s import the libraries we’ll use:
In [1]: import matplotlib.pyplot as plt In [2]: import numpy as np In [3]: import random In [4]: import seaborn as sns
The matplotlib.pyplot module contains the Matplotlib library’s graphing capabilities that we use. This module typically is imported with the name plt.
The NumPy (Numerical Python) library includes the function unique that we’ll use to summarize the die rolls. The numpy module typically is imported as np.
The random module contains Python’s random-number- generation functions.
The seaborn module contains the Seaborn library’s graphing capabilities we use. This module typically is imported with the name sns. Search for why this curious abbreviation was chosen.
Rolling the Die and Calculating Die Frequencies
Next, let’s use a list comprehension to create a list of 600 random die values, then use NumPy’s unique function to determine the unique roll values (most likely all six possible face values) and their frequencies:
In [5]: rolls = [random.randrange(1, 7) for i in range(600)] In [6]: values, frequencies = np.unique(rolls, return_counts=True)
The NumPy library provides the high-performance ndarray collection, which is typically much faster than lists.1 Though we do not use ndarray directly here, the NumPy unique function expects an ndarray argument and returns an ndarray. If you pass a list (like rolls), NumPy converts it to an ndarray for better performance. The ndarray that unique returns we’ll simply assign to a variable for use by a Seaborn plotting function.
Specifying the keyword argument return_counts=True tells unique to count each unique value’s number of occurrences. In this case, unique returns a tuple of two one-dimensional ndarrays containing the sorted unique values and the corresponding frequencies, respectively. We unpack the tuple’s ndarrays into the variables values and frequencies. If return_counts is False, only the list of unique values is returned.
Creating the Initial Bar Plot
Let’s create the bar plot’s title, set its style, then graph the die faces and frequencies:
In [7]: title = f'Rolling a Six-Sided Die {len(rolls):,} Times'
In [8]: sns.set_style('whitegrid')
In [9]: axes = sns.barplot(x=values, y=frequencies, palette='bright')
Snippet [7]’s f-string includes the number of die rolls in the bar plot’s title. The comma (,) format specifier in
{len(rolls):,}
displays the number with thousands separators—so, 60000 would be displayed as 60,000.
By default, Seaborn plots graphs on a plain white background, but it provides several styles to choose from ('darkgrid', 'whitegrid', 'dark', 'white' and 'ticks'). Snippet [8] specifies the 'whitegrid' style, which displays light-gray horizontal lines in the vertical bar plot. These help you see more easily how each bar’s height corresponds to the numeric frequency labels at the bar plot’s left side.
Snippet [9] graphs the die frequencies using Seaborn’s barplot function. When you execute this snippet, the following window appears (because you launched IPython with the --matplotlib option):
)
Seaborn interacts with Matplotlib to display the bars by creating a Matplotlib Axes object, which manages the content that appears in the window. Behind the scenes, Seaborn uses a Matplotlib Figure object to manage the window in which the Axes will appear. Function barplot’s first two arguments are ndarrays containing the x-axis and y-axis values, respectively. We used the optional palette keyword argument to choose Seaborn’s predefined color palette 'bright'. You can view the palette options at:
https://seaborn.pydata.org/tutorial/color_palettes.html
Function barplot returns the Axes object that it configured. We assign this to the variable axes so we can use it to configure other aspects of our final plot. Any changes you make to the bar plot after this point will appear immediately when you execute the corresponding snippet.
Setting the Window Title and Labeling the x- and y-Axes
The next two snippets add some descriptive text to the bar plot:
In [10]: axes.set_title(title) Out[10]: Text(0.5,1,'Rolling a Six-Sided Die 600 Times') In [11]: axes.set(xlabel='Die Value', ylabel='Frequency') Out[11]: [Text(92.6667,0.5,'Frequency'), Text(0.5,58.7667,'Die Value')]
Snippet [10] uses the axes object’s set_title method to display the title string centered above the plot. This method returns a Text object containing the title and its location in the window, which IPython simply displays as output for confirmation. You can ignore the Out[]s in the snippets above.
Snippet [11] add labels to each axis. The set method receives keyword arguments for the Axes object’s properties to set. The method displays the xlabel text along the x-axis, and the ylabel- text along the y-axis, and returns a list of Text objects containing the labels and their locations. The bar plot now appears as follows:
)
Finalizing the Bar Plot
The next two snippets complete the graph by making room for the text above each bar, then displaying it:
In [12]: axes.set_ylim(top=max(frequencies) * 1.10)
Out[12]: (0.0, 122.10000000000001)
In [13]: for bar, frequency in zip(axes.patches, frequencies):
...: text_x = bar.get_x() + bar.get_width() / 2.0
...: text_y = bar.get_height()
...: text = f'{frequency:,}\n{frequency / len(rolls):.3%}'
...: axes.text(text_x, text_y, text,
...: fontsize=11, ha='center', va='bottom')
...:
To make room for the text above the bars, snippet [12] scales the y-axis by 10%. We chose this value via experimentation. The Axes object’s set_ylim method has many optional keyword arguments. Here, we use only top to change the maximum value represented by the y-axis. We multiplied the largest frequency by 1.10 to ensure that the y-axis is 10% taller than the tallest bar.
Finally, snippet [13] displays each bar’s frequency value and percentage of the total rolls. The axes object’s patches collection contains two-dimensional colored shapes that represent the plot’s bars. The for statement uses zip to iterate through the patches and their corresponding frequency values. Each iteration unpacks into bar and frequency one of the tuples zip returns. The for statement’s suite operates as follows:
The first statement calculates the center x-coordinate where the text will appear. We calculate this as the sum of the bar’s left-edge x-coordinate (bar.get_x()) and half of the bar’s width (bar.get_width() / 2.0).
The second statement gets the y-coordinate where the text will appear—bar.get_y() represents the bar’s top.
The third statement creates a two-line string containing that bar’s frequency and the corresponding percentage of the total die rolls.
The last statement calls the Axes object’s text method to display the text above the bar. This method’s first two arguments specify the text’s x–y position, and the third argument is the text to display. The keyword argument ha specifies the horizontal alignment—we centered text horizontally around the x-coordinate. The keyword argument va specifies the vertical alignment—we aligned the bottom of the text with at the y-coordinate. The final bar plot is shown below:
)
Rolling Again and Updating the Bar Plot—Introducing IPython Magics
Now that you’ve created a nice bar plot, you probably want to try a different number of die rolls. First, clear the existing graph by calling Matplotlib’s cla (clear axes) function:
In [14]: plt.cla()
IPython provides special commands called magics for conveniently performing various tasks. Let’s use the %recall magic to get snippet [5], which created the rolls list, and place the code at the next In [] prompt:
In [15]: %recall 5 In [16]: rolls = [random.randrange(1, 7) for i in range(600)]
You can now edit the snippet to change the number of rolls to 60000, then press Enter to create a new list:
In [16]: rolls = [random.randrange(1, 7) for i in range(60000)]
Next, recall snippets [6] through [13]. This displays all the snippets in the specified range in the next In [] prompt. Press Enter to re-execute these snippets:
In [17]: %recall 6-13
In [18]: values, frequencies = np.unique(rolls, return_counts=True)
...: title = f'Rolling a Six-Sided Die {len(rolls):,} Times'
...: sns.set_style('whitegrid')
...: axes = sns.barplot(x=values, y=frequencies, palette='bright')
...: axes.set_title(title)
...: axes.set(xlabel='Die Value', ylabel='Frequency')
...: axes.set_ylim(top=max(frequencies) * 1.10)
...: for bar, frequency in zip(axes.patches, frequencies):
...: text_x = bar.get_x() + bar.get_width() / 2.0
...: text_y = bar.get_height()
...: text = f'{frequency:,}\n{frequency / len(rolls):.3%}'
...: axes.text(text_x, text_y, text,
...: fontsize=11, ha='center', va='bottom')
...:
The updated bar plot is shown below:
)
Saving Snippets to a File with the %save Magic
Once you’ve interactively created a plot, you may want to save the code to a file so you can turn it into a script and run it in the future. Let’s use the %save magic to save snippets 1 through 13 to a file named RollDie.py. IPython indicates the file to which the lines were written, then displays the lines that it saved:
In [19]: %save RollDie.py 1-13
The following commands were written to file `RollDie.py`:
import matplotlib.pyplot as plt
import numpy as np
import random
import seaborn as sns
rolls = [random.randrange(1, 7) for i in range(600)]
values, frequencies = np.unique(rolls, return_counts=True)
title = f'Rolling a Six-Sided Die {len(rolls):,} Times'
sns.set_style("whitegrid")
axes = sns.barplot(values, frequencies, palette='bright')
axes.set_title(title)
axes.set(xlabel='Die Value', ylabel='Frequency')
axes.set_ylim(top=max(frequencies) * 1.10)
for bar, frequency in zip(axes.patches, frequencies):
text_x = bar.get_x() + bar.get_width() / 2.0
text_y = bar.get_height()
text = f'{frequency:,}\n{frequency / len(rolls):.3%}'
axes.text(text_x, text_y, text,
fontsize=11, ha='center', va='bottom')
Command-Line Arguments; Displaying a Plot from a Script
Provided with this chapter’s examples is an edited version of the RollDie.py file you saved above. We added comments and a two modifications so you can run the script with an argument that specifies the number of die rolls, as in:
ipython RollDie.py 600
The Python Standard Library’s sys module enables a script to receive command-line arguments that are passed into the program. These include the script’s name and any values that appear to the right of it when you execute the script. The sys module’s argv list contains the arguments. In the command above, argv[0] is the string 'RollDie.py' and argv[1] is the string '600'. To control the number of die rolls with the command-line argument’s value, we modified the statement that creates the rolls list as follows:
rolls = [random.randrange(1, 7) for i in range(int(sys.argv[1]))]
Note that we converted the argv[1] string to an int.
Matplotlib and Seaborn do not automatically display the plot for you when you create it in a script. So at the end of the script we added the following call to Matplotlib’s show function, which displays the window containing the graph:
plt.show()