This chapter is from the book
This chapter is from the book
This chapter is from the book
How Much Automation Is Enough?
This is the question that has been asked since the inception of automated test tools. The tools vendors have presented us with one point of view, and industry experts have given us varied viewpoints. Vendors began with basic capture/playback tools that have evolved into some very sophisticated and highly integrated tool suites. They seem to have left it to the practitioner to determine what lies beyond the basic capture/playback model. The experts in test automation have written many articles and books. They have cited case studies of both successful and failed automation attempts. In the end there has been a modicum of agreement about what we must do, but not how we have to do it. In this text we will give you our point of view on how to do test automation. We believe the industry has debated what to do long enough. Until the tool suites reach a new plateau and until they possess even more sophistication, we have a working archetype for an automation framework that we can use.
To find out how much test automation is enough, we have to look at the areas of the software testing process that can be automated followed by the areas that should be automated. There is a difference between a tool and a process. Tools are used to facilitate processes. A tool can be used to implement a process and to enforce process standards. In many instances, the built-in procedures that tools bring with them can be construed as processes.
They are, however, frequently incomplete and ill-thought-out processes. The best software testing tools are the ones you can mold to your testing needs. They are the ones that offer a high degree of customizability with respect to workflow and to their tracking and reporting capabilities.
What are the types of tests that can be automated? They include unit, integration, and system testing. Subcategories of automated system tests include security, configuration, and load testing. Automated regression testing should be implemented at the unit, integration, and system levels during development and for use against major and minor system production releases.
What are the regions of the testing process that we should consider? They include the general spheres of test planning, test design, test construction, test execution, test results capture and analysis, test results verification, and test reporting. There are other activities that are closely associated with the testing activities proper. They include problem (defect) tracking and resolution, software configuration management, and software test metrics. Overall, the activities of the testing process are glued together, as is the software development process, by good project management techniques.
All of these areas should be automated to the degree that it makes sense for your organization in terms of time and costs. The more automation you can implement, the better and more effective your process will be. Realize that this statement holds true only if the tools are the appropriate ones and if they have been properly implemented. By implemented we mean that an integrated test automation framework has been constructed and is being used.