This chapter is from the book
This chapter is from the book
This chapter is from the book
When Should Software Be Tested?
The previous section gave away the answer to the preceding question to some extent—the earlier a part of the product can be tested, the cheaper it will be to find any problems that exist. If the parts of the application available at one stage of the process are known to work well and reliably, fewer problems will occur with integrating them or adding to them at later stages than if all the testing is done at the end. However, it was also shown in that section that software products are traditionally only tested at the end: An explicit QA phase follows the development, then the software is released to beta testers before finally being opened up for general release.
Modern approaches to software project management recognize that this is deficient and aim to continually test all parts of the product at all times. This is the main difference between “agile” projects and traditionally managed projects. Agile projects are organized in short stints called iterations (sometimes sprints). At every iteration, the requirements are reviewed; anything obsolete is dropped and any changes or necessary additions are made. The most important requirement is designed, implemented, and tested in that iteration. At the end of the iteration, the progress is reviewed and a decision made as to whether to add the newly developed feature to the product, or add requirements to make changes in future iterations. Crucially, because the agile manifesto (http://agilemanifesto.org/) values “individuals and interactions over processes and tools,” the customer or a representative is included in all the important decisions. There’s no need to sweat over perfecting a lengthy functional specification document if you can just ask the user how the app should work—and to confirm that the app does indeed work that way.
In agile projects then, all aspects of the software project are being tested all the time. The customers are asked at every implementation what their most important requirements are, and developers, analysts, and testers all work together on software that meets those requirements. One framework for agile software projects called Extreme Programming (or XP) goes as far as to require that developers unit test their code and work in pairs, with one “driving” the keyboard while the other suggests changes, improvements, and potential pitfalls.
So the real answer is that software should be tested all the time. You can’t completely remove the chance that users will use your product in unexpected ways and uncover bugs you didn’t address internally—not within reasonable time and budget constraints, anyway. But you can automatically test the basic stuff yourself, leaving your QA team or beta testers free to try out the experimental use cases and attempt to break your app in new and ingenious ways. And you can ask at every turn whether what you’re about to do will add something valuable to your product and increase the likelihood that your customers will be satisfied that your product does what the marketing text said it would.