- Python Shortcuts, Commands, and Packages
- 4.2 Twenty-Two Programming Shortcuts
- 4.3 Running Python from the Command Line
- 4.4 Writing and Using Doc Strings
- 4.5 Importing Packages
- 4.6 A Guided Tour of Python Packages
- 4.7 Functions as First-Class Objects
- 4.8 Variable-Length Argument Lists
- 4.9 Decorators and Function Profilers
- 4.10 Generators
- 4.11 Accessing Command-Line Arguments
- Chapter 4 Summary
- Chapter 4 Questions for Review
- Chapter 4 Suggested Problems
This chapter is from the book
This chapter is from the book
This chapter is from the book
4.9 Decorators and Function Profilers
When you start refining your Python programs, one of the most useful things to do is to time how fast individual functions run. You might want to know how many seconds and fractions of a second elapse while your program executes a function generating a thousand random numbers.
Decorated functions can profile the speed of your code, as well as provide other information, because functions are first-class objects. Central to the concept of decoration is a wrapper function, which does everything the original function does but also adds other statements to be executed.
Here’s an example, illustrated by Figure 4.3. The decorator takes a function F1 as input and returns another function, F2, as output. This second function, F2, is produced by including a call to F1 but adding other statements as well. F2 is a wrapper function.
)
Figure 4.3. How decorators work (high-level view)
Here’s an example of a decorator function that takes a function as argument and wraps it by adding calls to the time.time function. Note that time is a package, and it must be imported before time.time is called.
import time
def make_timer(func):
def wrapper():
t1 = time.time()
ret_val = func()
t2 = time.time()
print('Time elapsed was', t2 - t1)
return ret_val
return wrapper
There are several functions involved with this simple example (which, by the way, is not yet complete!), so let’s review.
There is a function to be given as input; let’s call this the original function (F1 in this case). We’d like to be able to input any function we want, and have it decorated—that is, acquire some additional statements.
The wrapper function is the result of adding these additional statements to the original function. In this case, these added statements report the number of seconds the original function took to execute.
The decorator is the function that performs the work of creating the wrapper function and returning it. The decorator is able to do this because it internally uses the def keyword to define a new function.
Ultimately, the wrapped version is intended to replace the original version, as you’ll see in this section. This is done by reassigning the function name.
If you look at this decorator function, you should notice it has an important omission: The arguments to the original function, func, are ignored. The wrapper function, as a result, will not correctly call func if arguments are involved.
The solution involves the *args and **kwargs language features, introduced in the previous section. Here’s the full decorator:
import time
def make_timer(func):
def wrapper(*args, **kwargs):
t1 = time.time()
ret_val = func(*args, **kwargs)
t2 = time.time()
print('Time elapsed was', t2 - t1)
return ret_val
return wrapper
The new function, remember, will be wrapper. It is wrapper (or rather, the function temporarily named wrapper) that will eventually be called in place of func; this wrapper function therefore must be able to take any number of arguments, including any number of keyword arguments. The correct action is to pass along all these arguments to the original function, func. Here’s how:
ret_val = func(*args, **kwargs)
Returning a value is also handled here; the wrapper returns the same value as func, as it should. What if func returns no value? That’s not a problem, because Python functions return None by default. So the value None, in that case, is simply passed along. (You don’t have to test for the existence of a return value; there always is one!)
Having defined this decorator, make_timer, we can take any function and produce a wrapped version of it. Then—and this is almost the final trick—we reassign the function name so that it refers to the wrapped version of the function.
def count_nums(n):
for i in range(n):
for j in range(1000):
pass
count_nums = make_timer(count_nums)
The wrapper function produced by make_timer is defined as follows (except that the identifier func will be reassigned, as you’ll see in a moment).
def wrapper(*args, **kwargs):
t1 = time.time()
ret_val = func(*args, **kwargs)
t2 = time.time()
print('Time elapsed was', t2 - t1)
return ret_val
We now reassign the name count_nums so that it refers to this function—wrapper—which will call the original count_nums function but also does other things.
Confused yet? Admittedly, it’s a brain twister at first. But all that’s going on is that (1) a more elaborate version of the original function is being created at run time, and (2) this more elaborate version is what the name, count_nums, will hereafter refer to. Python symbols can refer to any object, including functions (callable objects). Therefore, we can reassign function names all we want.
count_nums = wrapper
Or, more accurately,
count_nums = make_timer(count_nums)
So now, when you run count_nums (which now refers to the wrapped version of the function), you’ll get output like this, reporting execution time in seconds.
>>> count_nums(33000) Time elapsed was 1.063697338104248
The original version of count_nums did nothing except do some counting; this wrapped version reports the passage of time in addition to calling the original version of count_nums.
As a final step, Python provides a small but convenient bit of syntax to automate the reassignment of the function name.
@decorator
def func(args):
statements
This syntax is translated into the following:
def func(args):
statements
func = decorator(func)
In either case, it’s assumed that decorator is a function that has already been defined. This decorator must take a function as its argument and return a wrapped version of the function. Assuming all this has been done correctly, here’s a complete example utilizing the @ sign.
@make_timer
def count_nums(n):
for i in range(n):
for j in range(1000):
pass
After this definition is executed by Python, count_num can then be called, and it will execute count_num as defined, but it will also add (as part of the wrapper) a print statement telling the number of elapsed seconds.
Remember that this part of the trick (the final trick, actually) is to get the name count_nums to refer to the new version of count_nums, after the new statements have been added through the process of decoration.