Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Continuous Build, Test, and Regression
To help alleviate errors caused by manual changes and tests, one critical technique is to automate the process of building and testing software. Today, the tools exist to implement the following procedures:
- Detect a change in the source code that a software developer has provided.
- Automatically pull the change from the repository.
- Compile, link, and produce the deployable artifacts (executable files, databases, HTML files, configuration, and so on).
- Automatically deploy the artifacts to a testing environment.
- Automatically run unit tests against the code.
- If the unit tests pass, run regression tests against the code.
- Alert QA that a "regression tested and unit tested" build is ready for inspection.
- Upon passing manual inspection, allow QA to alert that the change is ready for production.
- Allow the appropriate administrator to promote the change to a staging environment, which can then be moved automatically to production.
This idea of continuously testing changes to make sure that they work (and alerting people when they don't) is known as continuous integration. It's a sharp departure from what happened in many IT departments even 10 years ago—and in many places, still happens today!—where you only know once every couple of weeks whether the software will even compile, much less have a chance of passing through quality assurance. Once a continuous-integration environment is up and running, the cost of doing basic regression testing is substantially reduced, which has the following primary outcomes:
- Testing frequency increases because you can do it more often at far lower people cost.
- Quality of testing increases because people in quality assurance spend more time testing complex edge cases, rather than more repetitive routine cases.
In a model like this, especially one with a robust suite of unit and regression tests, the time is reduced between making a change and knowing whether it had some unintended downstream effect. Another secondary effect it reducing the "It works on my machine" effect we hear from developers, as the deployment automation tends to standardize how the QA image is produced, which can limit or entirely eliminate environment variability.