Pragmatic Paranoia
Dave Thomas and Andy Hunt teach how to accept that you (and no one ever) will write a perfect piece of software.
This chapter is from the book
This chapter is from the book
This chapter is from the book
Did that hurt? It shouldn’t. Accept it as an axiom of life. Embrace it. Celebrate it. Because perfect software doesn’t exist. No one in the brief history of computing has ever written a piece of perfect software. It’s unlikely that you’ll be the first. And unless you accept this as a fact, you’ll end up wasting time and energy chasing an impossible dream.
So, given this depressing reality, how does a Pragmatic Programmer turn it into an advantage? That’s the topic of this chapter.
Everyone knows that they personally are the only good driver on Earth. The rest of the world is out there to get them, blowing through stop signs, weaving between lanes, not indicating turns, texting on the phone, and just generally not living up to our standards. So we drive defensively. We look out for trouble before it happens, anticipate the unexpected, and never put ourselves into a position from which we can’t extricate ourselves.
The analogy with coding is pretty obvious. We are constantly interfacing with other people’s code—code that might not live up to our high standards—and dealing with inputs that may or may not be valid. So we are taught to code defensively. If there’s any doubt, we validate all information we’re given. We use assertions to detect bad data, and distrust data from potential attackers or trolls. We check for consistency, put constraints on database columns, and generally feel pretty good about ourselves.
But Pragmatic Programmers take this a step further. They don’t trust themselves, either. Knowing that no one writes perfect code, including themselves, Pragmatic Programmers build in defenses against their own mistakes. We describe the first defensive measure in Topic 23, Design by Contract: clients and suppliers must agree on rights and responsibilities.
In Topic 24, Dead Programs Tell No Lies, we want to ensure that we do no damage while we’re working the bugs out. So we try to check things often and terminate the program if things go awry.
Topic 25, Assertive Programming describes an easy method of checking along the way—write code that actively verifies your assumptions.
As your programs get more dynamic, you’ll find yourself juggling system resources—memory, files, devices, and the like. In Topic 26, How to Balance Resources, we’ll suggest ways of ensuring that you don’t drop any of the balls.
And most importantly, we stick to small steps always, as described in Topic 27, Don’t Outrun Your Headlights, so we don’t fall off the edge of the cliff.
In a world of imperfect systems, ridiculous time scales, laughable tools, and impossible requirements, let’s play it safe. As Woody Allen said, “When everybody actually is out to get you, paranoia is just good thinking.”
Topic 23: Design by Contract
Nothing astonishes men so much as common sense and plain dealing.
Ralph Waldo Emerson, Essays
Dealing with computer systems is hard. Dealing with people is even harder. But as a species, we’ve had longer to figure out issues of human interactions. Some of the solutions we’ve come up with during the last few millennia can be applied to writing software as well. One of the best solutions for ensuring plain dealing is the contract.
A contract defines your rights and responsibilities, as well as those of the other party. In addition, there is an agreement concerning repercussions if either party fails to abide by the contract.
Maybe you have an employment contract that specifies the hours you’ll work and the rules of conduct you must follow. In return, the company pays you a salary and other perks. Each party meets its obligations and everyone benefits.
It’s an idea used the world over—both formally and informally—to help humans interact. Can we use the same concept to help software modules interact? The answer is “yes.”
DBC
Bertrand Meyer (Object-Oriented Software Construction [Mey97]) developed the concept of Design by Contract for the language Eiffel.1 It is a simple yet powerful technique that focuses on documenting (and agreeing to) the rights and responsibilities of software modules to ensure program correctness. What is a correct program? One that does no more and no less than it claims to do. Documenting and verifying that claim is the heart of Design by Contract (DBC, for short).
Every function and method in a software system does something. Before it starts that something, the function may have some expectation of the state of the world, and it may be able to make a statement about the state of the world when it concludes. Meyer describes these expectations and claims as follows:
Preconditions
What must be true in order for the routine to be called; the routine’s requirements. A routine should never get called when its preconditions would be violated. It is the caller’s responsibility to pass good data (see the box on page 110).
Postconditions
What the routine is guaranteed to do; the state of the world when the routine is done. The fact that the routine has a postcondition implies that it will conclude: infinite loops aren’t allowed.
Class invariants
A class ensures that this condition is always true from the perspective of a caller. During internal processing of a routine, the invariant may not hold, but by the time the routine exits and control returns to the caller, the invariant must be true. (Note that a class cannot give unrestricted write-access to any data member that participates in the invariant.)
The contract between a routine and any potential caller can thus be read as
If all the routine’s preconditions are met by the caller, the routine shall guarantee that all postconditions and invariants will be true when it completes.
If either party fails to live up to the terms of the contract, then a remedy (which was previously agreed to) is invoked—maybe an exception is raised, or the program terminates. Whatever happens, make no mistake that failure to live up to the contract is a bug. It is not something that should ever happen, which is why preconditions should not be used to perform things such as user-input validation.
Some languages have better support for these concepts than others. Clojure, for example, supports pre- and post-conditions as well as the more comprehensive instrumentation provided by specs. Here’s an example of a banking function to make a deposit using simple pre- and post-conditions:
(defn accept-deposit [account-id amount]
{ :pre [ (> amount 0.00)
(account-open? account-id) ]
:post [ (contains? (account-transactions account-id) %) ] }
"Accept a deposit and return the new transaction id"
;; Some other processing goes here...
;; Return the newly created transaction:
(create-transaction account-id :deposit amount))
There are two preconditions for the accept-deposit function. The first is that the amount is greater than zero, and the second is that the account is open and valid, as determined by some function named account-open?. There is also a postcondition: the function guarantees that the new transaction (the return value of this function, represented here by ‘%’) can be found among the transactions for this account.
If you call accept-deposit with a positive amount for the deposit and a valid account, it will proceed to create a transaction of the appropriate type and do whatever other processing it does. However, if there’s a bug in the program and you somehow passed in a negative amount for the deposit, you’ll get a runtime exception:
Exception in thread "main"... Caused by: java.lang.AssertionError: Assert failed: (> amount 0.0)
Similarly, this function requires that the specified account is open and valid. If it’s not, you’ll see that exception instead:
Exception in thread "main"... Caused by: java.lang.AssertionError: Assert failed: (account-open? account-id)
Other languages have features that, while not DBC-specific, can still be used to good effect. For example, Elixir uses guard clauses to dispatch function calls against several available bodies:
defmodule Deposits do
def accept_deposit(account_id, amount) when (amount > 100000) do
# Call the manager!
end
def accept_deposit(account_id, amount) when (amount > 10000) do
# Extra Federal requirements for reporting
# Some processing...
end
def accept_deposit(account_id, amount) when (amount > 0) do
# Some processing...
end
end
In this case, calling accept_deposit with a large enough amount may trigger additional steps and processing. Try to call it with an amount less than or equal to zero, however, and you’ll get an exception informing you that you can’t:
** (FunctionClauseError) no function clause matching in Deposits.accept_deposit/2
This is a better approach than simply checking your inputs; in this case, you simply can not call this function if your arguments are out of range.
In Topic 10, Orthogonality, we recommended writing “shy” code. Here, the emphasis is on “lazy” code: be strict in what you will accept before you begin, and promise as little as possible in return. Remember, if your contract indicates that you’ll accept anything and promise the world in return, then you’ve got a lot of code to write!
In any programming language, whether it’s functional, object-oriented, or procedural, DBC forces you to think.
Class Invariants and Functional Languages
It’s a naming thing. Eiffel is an object-oriented language, so Meyer named this idea “class invariant.” But, really, it’s more general than that. What this idea really refers to is state. In an object-oriented language, the state is associated with instances of classes. But other languages have state, too.
In a functional language, you typically pass state to functions and receive updated state as a result. The concepts of invariants is just as useful in these circumstances.
Implementing DBC
Simply enumerating what the input domain range is, what the boundary conditions are, and what the routine promises to deliver—or, more importantly, what it doesn’t promise to deliver—before you write the code is a huge leap forward in writing better software. By not stating these things, you are back to programming by coincidence (see the discussion on page 197), which is where many projects start, finish, and fail.
In languages that do not support DBC in the code, this might be as far as you can go—and that’s not too bad. DBC is, after all, a design technique. Even without automatic checking, you can put the contract in the code as comments or in the unit tests and still get a very real benefit.
Assertions
While documenting these assumptions is a great start, you can get much greater benefit by having the compiler check your contract for you. You can partially emulate this in some languages by using assertions: runtime checks of logical conditions (see Topic 25, Assertive Programming, on page 115). Why only partially? Can’t you use assertions to do everything DBC can do?
Unfortunately, the answer is no. To begin with, in object-oriented languages there probably is no support for propagating assertions down an inheritance hierarchy. This means that if you override a base class method that has a contract, the assertions that implement that contract will not be called correctly (unless you duplicate them manually in the new code). You must remember to call the class invariant (and all base class invariants) manually before you exit every method. The basic problem is that the contract is not automatically enforced.
In other environments, the exceptions generated from DBC-style assertions might be turned off globally or ignored in the code.
Also, there is no built-in concept of “old” values; that is, values as they existed at the entry to a method. If you’re using assertions to enforce contracts, you must add code to the precondition to save any information you’ll want to use in the postcondition, if the language will even allow that. In the Eiffel language, where DBC was born, you can just use old expression.
Finally, conventional runtime systems and libraries are not designed to support contracts, so these calls are not checked. This is a big loss, because it is often at the boundary between your code and the libraries it uses that the most problems are detected (see Topic 24, Dead Programs Tell No Lies, on page 112 for a more detailed discussion).
DBC and Crashing Early
DBC fits in nicely with our concept of crashing early (see Topic 24, Dead Programs Tell No Lies). By using an assert or DBC mechanism to validate the preconditions, postconditions, and invariants, you can crash early and report more accurate information about the problem.
For example, suppose you have a method that calculates square roots. It needs a DBC precondition that restricts the domain to positive numbers. In languages that support DBC, if you pass sqrt a negative parameter, you’ll get an informative error such as sqrt_arg_must_be_positive, along with a stack trace.
This is better than the alternative in other languages such as Java, C, and C++ where passing a negative number to sqrt returns the special value NaN (Not a Number). It may be some time later in the program that you attempt to do some math on NaN, with surprising results.
It’s much easier to find and diagnose the problem by crashing early, at the site of the problem.
Semantic Invariants
You can use semantic invariants to express inviolate requirements, a kind of “philosophical contract.”
We once wrote a debit card transaction switch. A major requirement was that the user of a debit card should never have the same transaction applied to their account twice. In other words, no matter what sort of failure mode might happen, the error should be on the side of not processing a transaction rather than processing a duplicate transaction.
This simple law, driven directly from the requirements, proved to be very helpful in sorting out complex error recovery scenarios, and guided the detailed design and implementation in many areas.
Be sure not to confuse requirements that are fixed, inviolate laws with those that are merely policies that might change with a new management regime. That’s why we use the term semantic invariants—it must be central to the very meaning of a thing, and not subject to the whims of policy (which is what more dynamic business rules are for).
When you find a requirement that qualifies, make sure it becomes a well-known part of whatever documentation you are producing—whether it is a bulleted list in the requirements document that gets signed in triplicate or just a big note on the common whiteboard that everyone sees. Try to state it clearly and unambiguously. For example, in the debit card example, we might write
Err in favor of the consumer.
This is a clear, concise, unambiguous statement that’s applicable in many different areas of the system. It is our contract with all users of the system, our guarantee of behavior.
Dynamic Contracts and Agents
Until now, we have talked about contracts as fixed, immutable specifications. But in the landscape of autonomous agents, this doesn’t need to be the case. By the definition of “autonomous,” agents are free to reject requests that they do not want to honor. They are free to renegotiate the contract—“I can’t provide that, but if you give me this, then I might provide something else.”
Certainly any system that relies on agent technology has a critical dependence on contractual arrangements—even if they are dynamically generated.
Imagine: with enough components and agents that can negotiate their own contracts among themselves to achieve a goal, we might just solve the software productivity crisis by letting software solve it for us.
But if we can’t use contracts by hand, we won’t be able to use them automatically. So next time you design a piece of software, design its contract as well.
Related Sections Include
Topic 24, Dead Programs Tell No Lies
Topic 25, Assertive Programming
Topic 38, Programming by Coincidence
Topic 42, Property-Based Testing
Topic 43, Stay Safe Out There
Topic 45, The Requirements Pit
Challenges
Points to ponder: If DBC is so powerful, why isn’t it used more widely? Is it hard to come up with the contract? Does it make you think about issues you’d rather ignore for now? Does it force you to THINK!? Clearly, this is a dangerous tool!
Exercises
Exercise 14 (possible answer)
Design an interface to a kitchen blender. It will eventually be a web-based, IoT-enabled blender, but for now we just need the interface to control it. It has ten speed settings (0 means off). You can’t operate it empty, and you can change the speed only one unit at a time (that is, from 0 to 1, and from 1 to 2, not from 0 to 2).
Here are the methods. Add appropriate pre- and postconditions and an invariant.
int getSpeed() void setSpeed(int x) boolean isFull() void fill() void empty()
Exercise 15 (possible answer)
How many numbers are in the series 0, 5, 10, 15, …, 100?