Imperfect Plans, Flawed Execution, and Unpredictable Results
- Dealing with the "Fog of Beforehand" One Step at a Time
- Facing the Unknown in Software Development
- How the Agile Manifesto Covers Common Anti-Patterns for Dealing with Friction
- Key Takeaways
This chapter is from the book
This chapter is from the book
This chapter is from the book
“The enemy of a good plan is the dream of a perfect plan.”
—General Carl von Clausewitz
In 1995, when I was 12 years old, I found myself sitting blindfolded in an old military truck with five classmates. The vehicle transported us to an unknown destination on the Dutch island of Vlieland in the middle of the night. After what seemed like forever, the truck stopped. Suddenly we were thrown out and left to our own devices.
Our assignment was to figure out how to get back to the farm where we were staying on our own—without any adult supervision or interference. All we had was a flashlight, our wits, and young legs. We had no map, mobile phone, water, or food. We were clueless about our location other than it was somewhere on the island.
Vlieland is a small island of roughly 40 km2 (10.000 acres) with a population of approximately 1000 people. To give you an idea of how small Vlieland is, you can drive from one end of the island to the other in about 15 minutes. When our whole class, including teachers and supporting staff, stayed there, we temporarily increased the island’s population by 4%.
At my primary school, it was customary for all kids in their last year to visit Vlieland before going off to their first year of high school. The trip’s highlight was a ritual we called “De Dropping.” It meant we were dropped off at a random location and challenged to find our way back to the farmhouse camp as quickly as possible.
None of the kids in my group had been to Vlieland before. Given our unfamiliarity with the island, it wouldn’t have made a difference where they dropped us—we would have felt lost anywhere. The group that found their way back the first would win a prize. “De Dropping” felt like a rite of passage to us. We had to face something scary and unknown together to prove we were mature enough to discover our own path.
When the truck dropped us off, it was pitch dark and our visibility was minimal, even with all the beams of our flashlights shining in the same direction. The only visible landmark to indicate the direction we should be heading was a lighthouse. We stood still for a moment, thinking, “What should we do now?”
Instead of debating, deliberating, or analyzing what best to do, we started walking together in a random direction. While walking, we noticed a higher vantage point and decided to make the ascent to establish our bearings. I remember hiking at the front because I had the fastest walking pace. This led to disagreement as others wanted to have a chance to take the lead, too. I slowed down to give them the opportunity to guide the group.
When we arrived at the top of the vantage point, we saw a lighthouse in the distance and a few roads we could follow. We decided to walk along the biggest road we could see. We surmised a larger road would be more likely to bring us to the town center. We reasoned that finding the town center would be our best bet to find our way back to camp, as we had visited it before. While following the road, we noticed a familiar church and used it to find the farmhouse sooner than we anticipated.
I do not remember how long it took exactly, but we found our way back swiftly. We did not arrive first, but did come in second place. Reflecting on our journey, returning to camp was a challenge in dealing with uncertainty and complexity. We were in the dark and unfamiliar with our surroundings. We did not have enough information to devise a definitive plan or strategy to find our way home.
Let’s examine what we can learn from the approach we followed to find our way back to the farmhouse.
Dealing with the “Fog of Beforehand” One Step at a Time
The red tower pictured on the cover of this book is the lighthouse on Vlieland that I saw as a kid in the distance. You can see part of the path toward the lighthouse, but not the entire path. Imagine we would want to go toward the lighthouse. Only by starting to walk the path can we discover all the steps we must take, with the lighthouse visible to help guide us every step of the way. If our assignment in Vlieland as kids would have been to reach the lighthouse, our journey would have been far easier.
Except we didn’t have to reach the lighthouse—we had to reach the farmhouse where we were staying. Only by taking a first step in any direction did we start to gain a better understanding and a mental picture of how we could find our way home. We could only start forming a rudimentary plan to find our way back after we started walking, observing, and reflecting on what we observed. Every step provided us with feedback on the best next step to take.
Before moving, we were limited by the fog of beforehand. Our knowledge was constrained to what we could know before starting. Our unfamiliarity with Vlieland meant we had insufficient information to find our way home. We knew too little to make a good plan or invent a definitive strategy. With each step, we gained more information and better understanding, slowly reducing the fog of beforehand.
Our natural response was not to discuss, deliberate, plan, or analyze, but rather to start walking in a random direction. We first made a few steps, then adjusted our plan and strategy based on what we saw and learned. Our plan emerged during and as a result of our walk. We were doing and learning simultaneously. We started with a humble plan, which we then adjusted and adapted as we discovered what was necessary as our understanding of the situation increased.
Unfortunately, in my experience, all too often adults take the opposite approach when dealing with the unknown. We are aware of the complexity we face but mistakenly believe we can eliminate it with sufficient planning, elaboration, and rational thought. We do the opposite of humble planning—namely, overconfident planning. We believe that we know more than we actually do, and that what we don’t know matters less than it does. And that’s the beginning of our downfall: Much like in a Greek tragedy, we’re suffering from planning hubris.
This line of thinking aligns perfectly with what we are taught at school: By spending enough time studying and trying to understand something, you can fully grasp it and pass the test. You have all the information at your disposal to potentially ace the test and score 100%.
The real world is vastly different, more dynamic, and far messier than passing an exam. There is no test to pass and no convenient textbook you can study to find all the answers you need. What to do and why are often unclear. You don’t have all the information available to make the best decisions and get a perfect grade. In such an uncertain and complex situation, the academic approach of discussing, analyzing, communicating, and planning everything before acting does not work.
When you lack enough information and understanding, an over-contemplative approach produces poor results. The fog of beforehand will limit your plans, which, due to excessive analysis before starting, will be anchored in fantasy—that is, your plans will be clouded by the fog of speculation. As a result, those plans will provide a false sense of comfort and prove more difficult to adjust when they inevitably turn out to be wrong.
The story from my childhood is easy to relate to but may leave you wondering how it is connected to software development. The fog of beforehand we experienced when walking in the dark in Vlieland also exists in software development. We often make the mistake of acting as if the fog doesn’t exist or believing we can think and analyze it away. Relying too heavily on what we believe to know injects the fog of speculation into our plans, which already suffer from the fog of beforehand. Consequently, our plans become even more disconnected from reality.
Let’s explore how you can notice the effects of the fog of beforehand and the fog of speculation in software development further by examining concrete examples.