- Experience 1: Early Success
- Experience 2: Challenges
- Lessons Learned
- Integrating Conventional Software Testing on an XP Team
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Experience 2: Challenges
While my first XP testing experience was a success, another project posed many more challenges. In this case, the project sponsors required a "QA person" on the project, but the developers didn't buy into that requirement. I knew how to work within the methodology, but the developers made it very clear that I wasn't welcome, with comments like this: "There is no testing role in the 'white book' [Extreme Programming Explained]." Finally, after pressure from the customer, the developers decided that it might be okay if I could write automated unit tests and do some user acceptance testing, but they really didn't think that I could add value. When I explained that my conventional testing activities had provided value on a previous XP project, they weren't interested. They spent some time telling me how the last project I was on "wasn't really XP"; if it had been, I wouldn't have been on the team. They thought that I should just come in to test at the end of the project, rather than being involved throughout the process, because that was how they had dealt with QA people in the past. I responded that that wasn't the way I worked, and the customer had asked for me to be on the team. I told them I'd work with them to see where testing could fit and help out.
My initial feedback was met with some suspicion from the developers. They even discouraged me from testing a functioning build when it was available: "We've already tested it with automated unit tests. Your retesting it is going to be a waste of time." When I explained that I could do testing that complemented their testing, they grudgingly agreed to let me work. When I found tricky problems that were due to system configuration issues that couldn't be taken into account by automated tests, I was told not to log that kind of bug. They also discouraged me from doing any manual exploratory testing. Again, I stuck to my guns and kept logging important bugs. After a few days, the developers started to realize that they were missing whole classes of tests that a willing conventional tester could execute. Slowly, they started to like having testers around because of the feedback we could provide.
Once we started getting along well, the developers decided that I should do work in the source code. They hoped that I could take over some of their unit test development, but I didn't really have their level of architectural and programming expertise; I couldn't just read the code and see the problem areas. I also felt that code-level testing was an important part of development work because a testable design is a good design. This bothered us at first, but we worked together on finding common ground. From my viewpoint, they were looking for a senior developer who was seasoned in design and test-driven development. What I offered was conventional testing activities to complement their work, within the XP developmental process. My knowledge and experience in XP carried a lot of weight, so the developers decided to support my doing conventional testing activities, especially since the feedback I provided was different and useful.
On this project, I felt a bit isolated from the developers at first, so I sought them out with clarifying questions on designs with stories. Sometimes this approach helped us to find problems with stories that were somewhat incoherent, so the designs firmed up a bit. I was logging bug reports as stories, but in general they were being ignored. The developers were ambivalent toward these bugs, but when the time came to demo the software to the project stakeholders at the end of an iteration, things started to change. The developers started asking for more conventional testing feedback, and they started taking bug reports more seriously. I stuck to my service role, focusing on providing feedback when they needed it, but being firm on bugs that would have a significant impact on customers. With time, they started asking for more feedback beyond bug reports, and took advantage of the availability of a dedicated tester to help them investigate bugs.
By project end, the developers had gone from saying "We don't need testers!" to "Please test this and give me feedback." Instead of dreading an unproductive period at the end of a project or iteration in which the QA team would begin to test the software and deem it worthy to be released at their discretion, the developers had testers whose job was to serve the team. They were enthusiastic about the feedback they got, particularly in the form of bug reports. They agreed that if the testers found problems before the users did, it was a good thing, so the more bugs we found, the happier they were. They said they were able to deliver the software with a great deal more confidence, and were surprised at how many bugs a conventional tester could find, even with all the automated unit tests the developers had written. In the end, they embraced the diversity of testing activities and were pleased to work with like-minded conventional testers.