This chapter is from the book
This chapter is from the book
This chapter is from the book
Why Test?
Much has been written about the need for automated unit and acceptance tests as part of agile software development. Writing good test code is hard, and maintaining obtuse test code is even harder. Because test code is optional (i.e., it is not what the customer is paying for), there is a strong temptation to abandon testing when the tests become difficult or expensive to maintain. Once we have given up on the principle of "keep the bar green to keep the code clean," much of the value of the automated tests is lost.
Over a series of projects, the teams I have worked with have faced a number of challenges to automated testing. The cost of writing and maintaining test suites has been a particular challenge, especially on projects with thousands of tests. Fortunately, as the cliché says, "Necessity is the mother of invention." My teams, and others, have developed a number of solutions to address these challenges. I have since spent a lot of time reflecting on these solutions to ask why they are good solutions. Along the way, I have divided the components of successful solutions into goals (things to achieve) and principles (ways to achieve them). Adherence to these goals and principles will result in automated tests that are easier to write, read, and maintain.
Economics of Test Automation
Of course, there is always a cost incurred in building and maintaining an automated test suite. Ardent test automation advocates will argue that it is worth spending more to have the ability to change the software later. Unfortunately, this "pay me now so you don't have to pay me later" argument doesn't go very far in a tough economic climate.1
Our goal should be to make the decision to do test automation a "no-brainer" by ensuring that it does not increase the cost of software development. Thus the additional cost of building and maintaining automated tests must be offset by savings through reduced manual unit testing and debugging/troubleshooting as well as the remediation cost of the defects that would have gone undetected until the formal test phase of the project or early production usage of the application. Figure 3.1 shows how the cost of automation is offset by the savings received from automation.
)
Figure 3.1 An automated unit test project with a good return on investment. The cost-benefit trade-off when the total cost is reduced by good test practices.
Initially, the cost of learning the new technology and practices takes additional effort. Once we get over this "hump," however, we should settle down to a steady state where the added cost (the part above the line) is fully offset by the savings (the part below the line). If tests are difficult to write, are difficult to understand, and require frequent, expensive maintenance, the total cost of software development (the heights of the vertical arrows) goes up as illustrated in Figure 3.2.
)
Figure 3.2 An automated unit test project with a poor return on investment. The cost-benefit trade-off when the total cost is increased by poor test practices.
Note how the added work above the line in Figure 3.2 is more than that seen in Figure 3.1 and continues to increase over time. Also, the saved effort below the line is reduced. This reflects the increase in overall effort, which exceeds the original effort without test automation.