Four Tips for Integrating Quality Assurance with Scrum
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
While Scrum is a great Agile framework for managing complex software development efforts, it does not have any intrinsic guidance on integrating Quality Assurance (QA) practices into the process. The traditional approaches to software testing that often rely on writing tests for comprehensive specifications, and testing the entire application at the end of the development cycle, fly in the face of Agile principles of delivering working software frequently in small iterations. My clients who have been practicing Scrum for some time as well as those that are newly transitioning struggle to incorporate adequate testing approaches in every Sprint.
Here are four tips for integrating quality assurance processes with Scrum that I have collected through years of struggling with multiple companies of various sizes.
Tip 1: Know the Underlying Agile/Scrum Principles Regarding Testing
While any team may adopt different mechanisms, practices, and tools for testing, the underlying values and principles of Agile and Scrum are the same. If you blindly adopt a process without understanding the concepts behind them it is all too easy to get caught up in the mechanics and stray from those values. The Scrum Guide states the following about the Product Increment:
“At the end of a Sprint, the new Increment must be ‘Done,’ which means it must be in useable condition and meet the Scrum Team’s definition of ‘Done.’ It must be in useable condition regardless of whether the Product Owner decides to actually release it.”
This is commonly referred to as being “release ready” at the end of a Sprint. As the guide points out, while you may not actually release the product increment to your customers, each deliverable resulting from the Sprint must be in a useable condition. Useable is a very relative word, and can vary per Scrum Team, as can their definition of “Done.” Generally this means the software has been coded, tested, reviewed by the Product Owner, and is ready to be deployed to production.
One of the twelve principles of the Agile Manifesto says:
“Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”
This is one of the underlying concepts behind delivering a new product increment each Sprint. The phrase “working software” is used several times in the manifesto as the primary measure of progress as well as one of the four core values. This is also a fairly nebulous term but the implication is that the resulting software has been tested and has the appropriate level of quality to be deployed to production.
The Scrum Guide also says:
“Each Increment is additive to all prior Increments and thoroughly tested, ensuring that all Increments work together.”
This speaks not only to testing the items you deliver in each Sprint, but regression testing them against everything you have delivered previously.
When you embark on adopting Scrum and its values and principles, then the approach you take with your testing processes should reflect them. Testing deliverables from one Sprint in the next Sprint does not adhere to these values. A Sprint dedicated to fixing bugs and regression testing the entire application is also not reflective of these principles. When crafting your testing approach weigh each practice against these values to ensure you are truly striving for working software in each Sprint.
Tip 2: Find and Remove Impediments to Adopting Agile Values Before Modifying Your Process
When you first try to adopt the principles discussed above you will likely run into issues that make testing in each Sprint difficult if not downright impossible. It is way too easy to modify your process around these issues and stray away from the values you are striving for.
One common example is not having enough time in the Sprint to test everything the team committed to. When this happens frequently, the idea of making your Sprint length longer inevitably comes up. While there may be valid reasons for longer Sprints, remember one of the values we are aiming for is to “Deliver working software frequently […] with a preference to the shorter timescale.” So before you make any decision, do some root-cause analysis.
I like to pretend I am my 5 year old daughter and use the “5 Whys” technique. The conversation may go something like this:
Scrum Team: “We need to go to three-week Sprints.”Me: “Why?”
Scrum Team: “QA doesn’t have enough time to test all the stories.”
Me: “Why?”
Scrum Team: “We have to wait until we get the finished feature before we can start writing test cases for it.”
Me: “Why?”
Scrum Team: “We don’t have enough detail about the feature to start with and there are changes made that we don’t know about.”
Me: “Why?”
Scrum Team: “We don’t interact with the developers enough to hear about these changes.”
Me: “Why?”
Scrum Team: “Because they sit in another building and we rarely see them during the Sprint. And stopping asking me ‘Why?’ you big jerk.”
At this point I would get the team together and discuss the Agile values of “individuals and interactions” as well as “The most efficient and effective method of conveying […] is face-to-face conversation.” The team and I talked about how the lack of regular interaction between developers and QA is making it hard to complete all the necessary testing in the Sprint, and then brainstorm on how to resolve that issue. Normally we arrive at everyone sitting together and involving QA in more design meetings, but I have had this conversation many times with various root causes being exposed.
Scrum is meant to expose underlying flaws in your processes, but all too often teams navigate around their problems instead of addressing the root causes. It’s a bit like buying a louder stereo because your car is making a funny sound. The goal is to dig deeper under the covers to find out what the problem really is and fix it rather than change your process to accommodate it.
Tip 3: Learn and Test as You Go
If I had a dollar for every time a QA analyst said to me “I can’t test until the application is done” or “I can’t write test cases without a finished specification,” I would be able to buy that new guitar I have my eye on. QA has been indoctrinated through waterfall processes that you have to test at the end of the project and must write your test cases from a comprehensive specification document. Trying to do this in a Sprint of a two to four weeks can be very painful, leading to QA scrambling at the end of the Sprint to finish test cases and perform some level of testing (as with the example in the previous section).
With Agile’s preference for “working software over comprehensive documentation” and the trend to get requirements at the last possible moment, teams won’t have all the details about a feature until close to the time they start to develop it. These details may also be fairly loose with the idea that the team can get clarification at any time because “Business people and developers must work together daily throughout the project.” Also keep in mind that when Agile people say “developer” that means anyone on the Scrum Team who helps construct the Product Increment, and that includes QA. The entire Scrum Team (including QA) assists the Product Owner with refining the Product Backlog. According to the Scrum Guide this can include up to 10% of the team’s capacity in a Sprint. In a two week Sprint that would be around eight hours. It is important that QA be a part of this so that they have as much knowledge about the expectations of a User Story as the developers. That way they can start creating high-level test cases very early in the process and evolve them over time.
Agile is a big proponent of short feedback loops inside a Sprint, and a team with tight collaboration between software developers and QA analysts can test elements of a feature before it is completed. A developer might be writing a web page to capture customer information that includes their name, address, contact information, and tax number. If they are developing the feature iteratively, they might get just the name and address working all the way first and have QA take a look at it while they work on the contact information section. If QA finds an issue, they can discuss it with the developer who can add a task to the Sprint Backlog to address the issue and the test case will be updated to not allow the issue to escape the Sprint.
This way QA can be involved throughout the development process and evolve their test cases the entire time. If there are changes made during the course of the development, they are right there to hear them and adjust their tests accordingly. When the User Story is finally complete the last round of testing takes less time because QA is now very familiar with the feature and has been constantly running versions of the current test cases all along the way. This has the side effect of exposing the software developers to the testing efforts, which gives them more perspective on the size of those efforts. I have personally seen developers’ methods for testing their code improve as a result of observing how QA approaches testing.
Tip 4: Accept That It’s a Journey
It is called an “Agile transition” for a reason. Scrum is a simple framework to learn, but difficult to master. It takes time and plenty of trial and error. There are some companies that can rip the band aid off and discard all their old waterfall ways, but not many. For most it takes time to integrate the mechanics of Scrum into their processes and often even longer for the Agile values and principles behind them to seep into their culture.
Early in your transition you may have to implement less-than-optimal practices until such a time when you can truly address the root causes. While some will say “That’s not Scrum!” part of being in a transition is to recognize that you may be addressing a symptom and not the disease, acknowledge you are temporarily compromising the Agile values you are striving for, and have a plan to eventually resolve the underlying issue.
Previously I stated that having a Sprint dedicated to regression testing would not be reflective of Scrum’s underlying Agile values. This is sometimes referred to as a “Hardening Sprint” and is widely considered to be an anti-pattern. One client who was adopting Scrum struggled with testing everything in a Sprint because they had a large amount of legacy code that had no automated tests and a hardware element that would always require some amount of manual testing efforts.
We implemented a window of all-hands, targeted regression testing at the end of each Sprint and reserved the last Sprint before each release for a more thorough, all-hands complete regression. Everyone understood that the lack of test automation was one of the root causes of our issue, but it was a significant effort to automate the huge library of test suites for this very complex application. An entire other team was brought in to start chipping away at automating these tests and we aggressively trained our entire team in test automation techniques and tools so that they could start creating these automated test scripts during the Sprints.
Over time, the test suites were automated, the dedicated test team was no longer needed, and the current Scrum teams could handle the automation of the current testing efforts each Sprint. The time needed for the targeted regression testing at the end of each Sprint was cut in half and only really remained for the manual hardware testing that could not be totally automated. The hardening Sprints were also drastically reduced.
This is a prime example of a company trying to adopt a testing approach that adhered to the principles of Scrum, but having to accept a less than optimal approach temporarily while they addressed the root causes of their issues. Remember that moving to Scrum is a journey and there are many steps (and quite a few missteps) in that process.
I cover some of these topics and more in my Scrum Fundamentals video on Safari Books Online and InformIT.