The Automated Testing Lifecycle Methodology (ATLM)
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Introducing the ATLM
Software project managers and software developers building today's applications face the challenge of doing so within an ever-shrinking schedule and with minimal resources. As part of their attempt to do more with less, organizations want to test software adequately, but as quickly and thoroughly as possible. To accomplish this goal, organizations are turning to automated testing.
Faced with this reality and realizing that many tests cannot be executed manually, such as simulating 1,000 virtual users for volume testing, software professionals are introducing automated testing to their projects. But these software professionals may not know what's involved in introducing an automated test tool to a software project, and they may be unfamiliar with the breadth of application that automated test tools have today. The Automated Testing Lifecycle Methodology (ATLM), depicted in Figure 1, provides guidance in these areas.
){kind=link}
Figure 1 The Automated Test Lifecycle Methodology (ATLM).
By using the systematic approach outlined within the ATLM, organizations can organize and execute test activities in such a way as to maximize test coverage within the limits of testing resources. This structured test methodology involves a multi-stage process, supporting the detailed and interrelated activities that are required to introduce and utilize an automated test tool:
Develop test design.
Develop and execute test cases.
Develop and manage test data and the test environment.
Document, track, and obtain closure on issue/trouble reports.
Clearly, the emphasis on automated testing represents a paradigm change for the software industry. This change doesn't simply involve the application of tools and the performance of test automation. Rather, it blankets the entire test lifecycle and the system development lifecycle. The ATLM implementation takes place in parallel with the system development lifecycle. For software professionals to make a successful leap to automated testing, they must embrace structured approaches to testing. The ATLM is revolutionary in the fact that it promulgates a new structured, building-block approach to the entire test lifecycle, which enables software test professionals to approach software testing in a methodical and repeatable fashion.
The growth of automated test capability has stemmed in large part from the growing popularity of the iterative and incremental development lifecycle, a software development methodology that focuses on minimizing the development schedule while providing frequent, incremental software builds. The objective of this incremental and iterative development is to engage the user and the test team early throughout the design and development of each build in order to refine the software, thereby ensuring that it more closely reflects the needs and preferences of the user and thus addressing the riskiest aspects of development in early builds.
In this environment of continual changes and additions to the software through each software build, software testing itself takes on an iterative nature. Each new build is accompanied by a considerable number of new tests as well as rework to existing test scripts, just as there is rework on previously released software modules. Given the continual changes and additions to software applications, especially Web applications, automated software testing becomes an important control mechanism to ensure accuracy and stability of the software through each build.
The ATLM, invoked to support testing efforts involving automated test tools, incorporates a multi-stage process. The methodology supports the detailed and interrelated activities that are required to decide whether to acquire an automated testing tool. The methodology includes the process of how to introduce and utilize an automated test tool, covers test development and test design, and addresses test execution and management. The methodology also supports the development and management of test data and the test environment, and addresses test documentation to include problem reports.
The ATLM methodology represents a structured approach, which depicts a process with which to approach and execute testing. This structured approach is necessary to help steer the test team away from these common test program mistakes:
Implementing the use of an automated test tool without a testing process in place, resulting in an ad hoc, non-repeatable, non-measurable test program.
Implementing a test design without following any design standards, resulting in the creation of test scripts that are not repeatable and therefore not reusable for incremental software builds.
Attempting to automate 100% of test requirements, when tools or in-house–developed automated test harnesses do not support automation of all tests required.
Using the wrong tool or developing a too-elaborate in-house test harness.
Initiating test tool implementation too late in the application-development lifecycle—not allowing sufficient time for tool setup and test tool introduction process (learning curve).
Initiating test engineer involvement too late in the application-development lifecycle, resulting in poor understanding of the application and system design, which results in incomplete testing.
The Automated Test Lifecycle Methodology (ATLM) comprises six primary processes or components:
Decision to Automate Testing
Test Tool Acquisition
Automated Testing Introduction Process
Test Planning, Design, and Development
Execution and Management of Tests
Test Program Review and Assessment
The following sections describe each primary process, as well as the subordinate processes contained within each primary process.