This chapter is from the book
This chapter is from the book
This chapter is from the book
Examples of Technical Debt
To illustrate our definition, we offer a few stories about technical debt in software development projects. You will see organizations struggling with their technical debt and software development teams failing to strategize about it.
Quick-and-Dirty if-then-else
A company in Canada developed a good product for its local customers. Based on local success, the company decided to extend the market to the rest of Canada and immediately faced a new challenge: addressing the 20% of Canada that uses the French language in most aspects of life. The developers labored for a week to produce a French version of the product, planting a global flag for French = Yes or No as well as hundreds of if-then-else statements all over the code. A product demo went smoothly, and they got the sale!
Then, a month later, on a trip to Japan, a salesperson proudly boasted that the software was multilingual, returned to Canada with a potential order, and assumed that a Japanese version was only one week of work away. Now the decision not to use a more sophisticated strategy—such as externalizing all the text strings and using an internationalization package—was badly hurting the developers. They would not only have to select and implement a scalable and maintainable strategy but also have to undo all the quick-and-dirty if-then-else statements.
For the Canadian company, the decision to use if-then-else statements spread the change throughout the code, but it was a necessary quick-and-dirty solution from a business perspective to get a quick sale. Doing the right thing at that stage would have postponed the delivery of the system and likely lost them the deal. So even though the resulting code was ugly—as well as hard to modify and evolve—it was the right decision. Now, would you continue down that path and add another layer of if-then-else for each language? Or would you rethink the strategy and decide to repay the original technical debt? Inserting the Japanese version of the quick fix, with its issues of character sets and vertical text, would be too much of a burden and a subsequent maintenance issue. You may argue that a good designer would have set up provisions for internationalization and localization right at the outset, but this is easy to say in hindsight; the demands and constraints at the beginning of development for this small venture were quite different, focused on the main features, and didn’t foresee the need for a multilingual feature.
Hitting the Wall
Two large global financial institutions merged. As a result, two IT systems essential to their business had to merge. The management of the new company determined that a duct-tape and rubber-band system, mixing the two systems in some kind of chimera, would not work. They decided to build a support system from scratch, using more recent technologies and, in some ways, walking away from years of accumulated technical debt in the original systems.
The company organized a team to build the new replacement system. They progressed rapidly because the first major release was to provide an exact replacement of the existing systems. In a few months, they accumulated a lot of code that performed well in demos for each one week “sprint” (or iteration). But nobody thought about the architecture of the system; everyone focused on creating more and more features for the demo. Finally, some harder issues of scalability, data management, distribution of the system, and security began to surface, and the team discovered that refactoring the mass of code already produced to address these issues was rapidly leading them to a complete stop. They hit the wall, as marathon runners would say. They had lots of code but no explicit architecture. In six months, the organization had accumulated a massive amount of technical debt that brought them to a standstill.
The situation here is very different from the first case. This was not an issue of code quality. It was an issue of foresight. The development team neglected to consider architectural and technology selection issues or learn from the two existing systems at appropriate times during development; the team did not need to do all of that up front, but it needed to do it early enough not to burden the project downstream. Refactoring is valuable, but it has limits. The development team had to throw away large portions of the existing code weeks after its original production. Although the organization hoped to eliminate technical debt when it decided to implement a brand-new system after the merger, it failed to incorporate eliminating technical debt into the project management strategy for the new system. Ignorance is bliss—but only for a while.
Crumbling Under the Load
A successful company in the maritime equipment industry successfully evolved its products for 16 years, in the process amassing 3 million lines of code. Over these 16 years, the company launched many different products, all under warranty or maintenance contracts; new technologies evolved; staff turned over; and new competitors entered the industry.
The company’s products were hard to evolve. Small changes or additions led to large amounts of work in regression testing with the existing products, and much of the testing had to be done manually, over several days per release. Small changes often broke the code, for reasons unsuspected by the new members of the development team, because many of the design and program choices were not documented.
In the case of the maritime equipment company, there was no single cause of technical debt. There were hundreds of causes: code imperfections, tricks, and workarounds, compounded by no usable documentation and little automated testing. While the development team dreams of a complete rewrite, the economic situation does not allow delaying new releases or new products or abandoning support for older products. Some intermediate strategy must be implemented.
Death by a Thousand Cuts
One IT-service organization landed several major contracts. Some of this new business allowed the organization to grow its offshore development businesses and enter emerging software development markets. For several years, the organization experienced a hiring boom.
The IT-service projects were similar in nature, and the organization assumed that its new developers were interchangeable across projects. The project managers thought, “The task is customization of the same or similar software, so how different could it be?” But in some cases, the new employees lacked the right skills or knowledge about the packages used. In other cases, time and revenue-growth pressures pushed them to skip testing the code thoroughly or fail to think through their designs. They also did not put in the time to create common application programming interfaces (APIs). The hiring boom created unstable teams, with new members introduced almost every month. It even became an internal joke: “Get a bunch of online Java and Microsoft certifications, and you are a senior developer here.” In no time, the project managers lost control of the schedule as well as the number of defects introduced into the system.
This IT-service organization provides another example in which there is no single source of technical debt. We call this “death by a thousand cuts” because a pervasive lack of competence can result in many small, avoidable coding issues that are never caught. Lack of organizational competency—as in the case of this IT-service organization—easily activates a number of cascading effects. The unplanned and unmanaged hiring boom, the missed opportunity to enforce commonality across the products, and the limited testing all contributed to the accumulating technical debt.
Tactical Investment
A five-person company developed a web application in the urban transportation domain, targeted at users of buses and trains. In this relatively new and rapidly evolving domain, the targeted users could not really tell the company what they would need. “I’ll know it when I see it” was the general response. So, the company developed a “minimum viable product” (MVP) with some core functionality and little underlying sophistication. Members of the company beta-tested it with about 100 users in one city. They had to “pivot” several times until they found their niche, at which point they invested heavily in building the right infrastructure for a product that would be able to support millions of simultaneous users and adapt to dozens of situations and cities.
The initial shortcuts that members of this small company took and the high-level rudimentary infrastructure they initially developed are examples of technical debt wisely assumed. The company borrowed the time it would have spent on the complete definition and implementation of the infrastructure to deliver early. This allowed it to complete an MVP months earlier than traditional development practices, which put the infrastructure first, would have allowed. Moreover, the company learned useful lessons about the key issues (which did not necessarily match its initial assumptions) of reliability, fault tolerance, adaptability, and portability. Building in these quality attributes up front would have created massive rework once the developers understood more completely what their users needed.
All along, members of this company were aware of the deliberate shortcuts they were taking and their consequences on future development. From the perspective of their angel investors, these were good strategies for risk management; if the company found no traction in the market, the developers could stop development early and minimize cost before the company made massive financial investments. Management also made it very clear to everyone, internal and external, that the shortcuts were temporary solutions so that no one would be tempted to keep them, painfully patched, as part of the permanent solution. In this manner, taking on technical debt was a wise investment that paid off. The company repaid the “borrowed time,” but it could also have walked away from the project.
In all these examples, the current state of the software carries code that works, but it makes further evolutions harder. The debt was induced by lack of foresight, time constraints, significant changes in requirements, or changes in the business context.