This chapter is from the book
This chapter is from the book
This chapter is from the book
Technical Debt
The term technical debt has gained a lot of traction in the software industry. It is a metaphor that addresses the challenge caused by several short-term decisions resulting in long-term challenges. It draws comparison with how financial debt works. Technical debt is not always bad—it is sometimes beneficial (e.g., quick solutions to get a product to market). The concept was first introduced by Ward Cunningham:
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. . . . The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.17
Although the term is used widely in the industry, it is not clearly defined. It is similar to how the term use case gained wide usage—but lost its original intent and clear definition. In their book Managing Technical Debt, Kruchten, Nord, and Ozkaya address this ambiguity and provide a comprehensive overview of the concept of technical debt and how to manage it. Their definition of technical debt is as follows:
In software-intensive systems, technical debt consists of design or implementation constructs that are expedient in the short term but that set up a technical context that can make future change more costly or impossible. Technical debt is a contingent liability whose impact is limited to internal system qualities—primarily, but not only, maintainability and evolvability.18
This is a good definition because it focuses more on the impact of technical debt and does not strictly follow the financial debt metaphor—which, though useful, is not a fully accurate way to represent the topic. The focus on maintainability and evolvability is key to how to think about technical debt. It implies that if your system is not expected to evolve, the focus on technical debt should be minimal. For example, software written for the Voyager spacecraft should have very limited focus on technical debt19 because it is not expected to evolve and has limited maintenance opportunities.
As shown in Figure 2.6, technical debt can be divided into three categories:
)
Figure 2.6 Technical debt landscape. (Source: Kruchten, P., R. Nord & I. Ozkaya, Managing Technical Debt, SEI Series in Software Engineering, Addison-Wesley, 2019.)
Code: This category includes expediently written code that is difficult to maintain and evolve (i.e., introduce new features). The Object Management Group (OMG)’s Automated Technical Debt Measure specification20 can be used by source code analysis tools to measure this aspect. You can view the specification as standardized best practices for common topics such as managing loops, initialization of variables, and so on. Because this book is more about architecture than implementation, we do not discuss this aspect of technical debt any further.
Architecture: Debt in this category is the result of architectural decisions made during the software development process. This type of technical debt is difficult to measure via tools but usually has a more significant impact on the system than other types of debt. For example, the decision to use a database technology that cannot provide the quality attributes required (e.g., using a relational database when a basic key–value database would do) has a significant impact on the scalability and maintainability of a system.
Production infrastructure: This category of technical debt deals with decisions focused on the infrastructure and code that are used to build, test, and deploy a software system. Build-test-deploy is becoming increasingly integral to software development and is the main focus of DevOps. Continuous Architecture sees the build-test-deploy environment as part of the overall architecture, as stated by principle 5, Architect for build, test, deploy, and operate.
We refer readers to Managing Technical Debt and other books for more in-depth information on this important topic. In the next sections, we focus on recommendations of incorporating practices to identify and manage technical debt from the perspective of the architecture of a product.
Capturing Technical Debt
We recommend creating a technical debt registry as a key artifact for managing the architecture of a system. In terms of visibility and linkage to product backlogs, it should be managed in a similar manner to the architectural decision backlog.
For each technical item, it is important to capture the following relevant information:
Consequences of not addressing the technical debt item. The consequences can be articulated in terms of inability to meet future business requirements or limitations to the quality attributes of the product. These should be defined in a business-friendly manner because, at the end of the day, addressing technical debt will be prioritized against meeting immediate business demand.
Remediation approach for addressing the technical debt item. The clearer this approach can be defined, the easier it is to make decisions on prioritization of a technical debt item against other features.
Just like the architectural decision backlog, the technical debt registry should be viewable separately. However, it does not need to be managed as a separate item.24 One effective approach we have observed is to have product backlog items tagged as technical debt. When required, you can easily pull in all technical debt items from the individual project backlogs, as shown in Figure 2.7.
)
Figure 2.7 Technical debt registry and backlogs
How to Manage Technical Debt
Once you have the technical debt registry in place, it is also important to agree on a process for the prioritization of the technical debt items. We recommend basing prioritization on the consequences of the technical debt items and not worrying too much about “technical purity.” For example, converting a file-based batch interface to an API-based real-time interface might seem like a good thing to do, but if there is limited impact on the system’s business value, it should not be prioritized.
We see two main drivers for the architectural focus on technical debt: to make appropriate architectural decisions and to influence prioritization of future releases.
While making an architectural decision, it is important to understand if we are alleviating any existing technical debt items or introducing new technical debt. This ensures that we keep the perspective of the long-term conceptual integrity of the product at each step.
Now, let us look at how prioritization of backlog items works. In an agile model, it is the product owner who decides what items should be prioritized. Even if you do not operate in a fully agile model, you still have conversations about prioritization and budget with your business stakeholders. If technical debt and its impact is not visible to the business stakeholders, it will always take a back seat to new features. Technical debt items are, by their nature, not clearly visible as features, and they have an impact predominantly on quality attributes.25 This is where an architectural focus comes in.26 The objective is to articulate the impact of delaying addressing technical debt items. If we delay addressing technical debt for too long, the software system can hit the technical debt singularity.
Another tactic for making sure technical debt is not lost in the rush for new features is to carve out a proportion of each release to address technical debt. How to categorize your backlog items is a wide area that is not in the scope of this book; however, a compelling view is offered by Mik Kersten.27 He states that there are four types of items (i.e., flow items) to be considered in the backlog: features, defects, technical debt, and risk (e.g., security, regulatory).
To limit our scope to an achievable size, we decided not to discuss technical debt in the rest of this book. However, we believe that it is an important area for architects to actively manage and refer you to the references provided.