- UAT Theory Versus Reality
- Profiling Testers
- Preparing Test Materials
- Conducting UAT Sessions
- Preventing Common UAT Failure Points
- Summary
Preventing Common UAT Failure Points
It may sound pessimistic, but whenever I'm involved with or responsible for running UAT, I've heard the same three comments enough times from UAT testers that I now plan ahead for these potential roadblocks.
Potential roadblock 1: "I was too busy to complete the testing."
Often the testers for UAT are truly busy employees whose primary role isn't testing. These testers frequently put the UAT cycle so low on their to-do lists that they don't complete the testing. I try to avoid this roadblock by discussing time limitations ahead of the testing, but then I also plan accordingly. One tactic that has worked several times for me has been to assign each UAT tester only a single test case, making sure that each case will be executed by at least one person. If I suspect that time will still be an issue, I try to call each tester during the test period, nudging them to complete their individual test case.
Another tactic is to use the testers on my team to complete the same test cases in parallel—but I don't let the UAT testers know about this strategy. I don't keep the duplicate effort a secret in order to be manipulative; the point is to ensure that someone who is nearly always more technical has executed the same test case. We can compare the results later to see if there is any difference in findings.
A second reason is fail-safe planning. While UAT is supposed to be conducted by users, I see the gaps between reality and theory often enough to know better. Sometimes the most sincerely dedicated UAT testers don't complete what is needed. When I arrive at the final session with completed test cases from the test team, we may conclude that although some users didn't execute their testing, the team is still comfortable enough to sign off.
Potential roadblock 2: "I'm here, now what do you want me to do?"
At one UAT session where the users were expected to spend the afternoon testing, a tester asked to speak to me privately. This senior director for the company whispered, "I don't know what to do. I don't know what to test and I don't know how to test." Initially I was stunned. Here was a highly skilled company director who had experience with the product—more experience than I had at this point—and she didn't know what to test or what to do?
What I learned by sitting down and talking with her is obvious to me now in hindsight: This executive was not a tester. She certainly knew the product and knew what she needed the new features to provide, but she didn't know what to test for, beyond a couple of "happy path" ideas. In this particular UAT session, no materials had been planned. A training room had been allocated for one day of UAT, so hardware and software access wasn't an issue, but the executive didn't know how to spend her time.
We took two different approaches that day. One was for me to sit with the executive and help her to execute her testing, which solved our immediate shared problem of completing the UAT. The second approach was for the executive to spend time with me exchanging ideas and knowledge. I explained some testing concepts to her, and she explained her department's most pressing needs for this product. If she could explain her needs to me, I would be able to bring her ideas back to the test team I was managing. My team and I would be incorporate ideas earlier in the testing—a concept that we both liked. The session also helped us to get to know each other and forge a lasting relationship.
Potential roadblock 3: "It's so late in the product development cycle, it doesn't matter what issues I find—the team will never incorporate our feedback."
Sometimes UAT testing is more about politics than about testing. That's an ugly reality in some situations. The concept of UAT sounds great, but the reality is that sometimes the users really are not involved, and their testing comes so late in the cycle that only a show-stopping defect will hold up the release. If that's your scenario, you can see why the UAT testers won't be particularly enthusiastic.
What do you do in this sticky situation? Don't think of product development as a one-time event. A specific product release may be a singular event, but the product probably will continue to be upgraded. For this particular UAT, consider the immediate need. Does a document need to be signed? A helpful approach in situations like this is to offer a detailed discussion of what testing has been done by the test team. Can the team conclude for this release that testing has been sufficient? With the immediate UAT need resolved, work to prevent the same predicament from occurring for the next release.