Home > Articles

Functions

This chapter is from the book

Item 36: Use None and Docstrings to Specify Dynamic Default Arguments

Sometimes it can be helpful to use a function call, a newly created object, or a container type (like an empty list) as a keyword argument’s default value. For example, say that I want to print logging messages that are marked with the time of the logged event. In the default case, I want the message to include the time when the function was called. I might try the following approach, which assumes that the default value for the when keyword argument is reevaluated each time the function is called:

from time import sleep
from datetime import datetime

def log(message, when=datetime.now()):
    print(f"{when}: {message}")

log("Hi there!")
sleep(0.1)
log("Hello again!")

>>>
2024-06-28 22:44:32.157132: Hi there!
2024-06-28 22:44:32.157132: Hello again!

This doesn’t work as expected. The timestamps are the same because datetime.now is executed only a single time: when the function is defined at module import time. A default argument value is evaluated only once per module load, which usually happens when a program starts up (see Item 98: “Lazy-Load Modules with Dynamic Imports to Reduce Startup Time” for details). After the module containing this code is loaded, the datetime.now() default argument expression will never be evaluated again.

The convention for achieving the desired result in Python is to provide a default value of None and to document the actual behavior in the docstring (see Item 118: “Write Docstrings for Every Function, Class, and Module” for background). When your code sees that the argument value is None, you allocate the default value accordingly:

def log(message, when=None):
    """Log a message with a timestamp.

    Args:
        message: Message to print.
        when: datetime of when the message occurred.
            Defaults to the present time.
    """
    if when is None:
        when = datetime.now()
    print(f"{when}: {message}")

Now the timestamps will be different:

log("Hi there!")
sleep(0.1)
log("Hello again!")

>>>
2024-06-28 22:44:32.446842: Hi there!
2024-06-28 22:44:32.551912: Hello again!

Using None for default argument values is especially important when the arguments are mutable. For example, say that I want to load a value that’s encoded as JSON data; if decoding the data fails, I want an empty dictionary to be returned by default:

import json

def decode(data, default={}):
    try:
        return json.loads(data)
    except ValueError:
        return default

The problem here is similar to the problem in the datetime.now example above. The dictionary specified for default will be shared by all calls to decode because default argument values are evaluated only once (at module load time). This can cause extremely surprising behavior:

foo = decode("bad data")
foo["stuff"] = 5
bar = decode("also bad")
bar["meep"] = 1
print("Foo:", foo)
print("Bar:", bar)

>>>
Foo: {'stuff': 5, 'meep': 1}
Bar: {'stuff': 5, 'meep': 1}

You might expect two different dictionaries, each with a single key and value. But modifying one seems to also modify the other. The culprit is that foo and bar are both equal to the default parameter to the decode function. They are the same dictionary object:

assert foo is bar

The fix is to set the keyword argument default value to None, document the actual default value in the function’s docstring, and act accordingly in the function body when the argument has the value None:

def decode(data, default=None):
    """Load JSON data from a string.

    Args:
        data: JSON data to decode.
        default: Value to return if decoding fails.
            Defaults to an empty dictionary.
    """
    try:
        return json.loads(data)
    except ValueError:
        if default is None:  # Check here
            default = {}
        return default

Now, running the same test code as before produces the expected result:

foo = decode("bad data")
foo["stuff"] = 5
bar = decode("also bad")
bar["meep"] = 1
print("Foo:", foo)
print("Bar:", bar)
assert foo is not bar

>>>
Foo: {'stuff': 5}
Bar: {'meep': 1}

This approach also works with type annotations (see Item 124: “Consider Static Analysis via typing to Obviate Bugs”). Here, the when argument is marked as having an optional value that is a datetime. Thus, the only two valid choices for when are None or a datetime object:

def log_typed(message: str, when: datetime | None = None) ->
              None:
    """Log a message with a timestamp.

    Args:
        message: Message to print.
        when: datetime of when the message occurred.
            Defaults to the present time.
    """
    if when is None:
        when = datetime.now()
    print(f"{when}: {message}")

Things to Remember

  • A default argument value is evaluated only once: during function definition at module load time. This can cause odd behaviors for dynamic values (like function calls, newly created objects, and container types).

  • Use None as a placeholder default value for a keyword argument that must have its actual default value initialized dynamically. Document the intended default for the argument in the function’s docstring. Check for the None argument value in the function body to trigger the correct default behavior.

  • Using None to represent keyword argument default values also works correctly with type annotations.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.