- Documentation as Discipline
- Documentation Gone Wrong
- A Clear Departure from the Past
- Finding the Right Balance
- Staying Focused on Discipline
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Documentation Gone Wrong
I recently worked with a software development team that functioned in an almost completely ad hoc way. Testing was inconsistent from release to release. Mistakes in one area weren't captured in any way to ensure that they weren't made in another area. Defects were entered, worked, and managed in different ways depending on the people involved. New people who were brought into the testing team were on their own in coming up to speed.
When I came on board, the engineering team was preparing for its first external audit for PCI compliance. At the same time, the team was trying to get a major platform release out the door, and simultaneously changing the software development methodology to Scrum (the team's first formal adoption of a methodology). As a testing team, this situation created three interesting (and somewhat conflicting) short-term goals:
- Document the existing testing process for the PCI audit. Try to remain honest, but paint things in the best possible light.
- Figure out a testing methodology that would integrate into the Scrum process being implemented by the programming team.
- Try to assess current testing risk and coverage for the upcoming platform release, so the team would know when the release was ready to be deployed.
Asking around to start leveraging what we had, I received many comments that either we didn't have something documented yet or that it was "documented on the wiki." Now, don't get me wrong—I love Wikipedia as much as the next guy, but they put a lot of effort in managing the structure of their website. If you work on a team that doesn't manage information according to some standard, your wiki is likely to be just as useful as a network drive where people drop things into folders at will. Just because people can search a little bit easier, that doesn't mean that the content is more accessible.
Upon reviewing what documentation I could find, I discovered that most of it was out of date, conflicting, and poorly written (bad grammar, bad formatting, and the like)—not something I could share with an auditor.
At the same time, I worked with the team to understand what testing had been done for the upcoming release. I got a mix of answers; it wasn't clear who was working on what. It wasn't clear what had been tested, or sometimes even what new functionality was in the release. There was no clear concept of when testing would be finished, or what still needed to be tested. When I reviewed bug reports, I couldn't tell how to reproduce a bug (or even if the bug was reproducible), what version of the code contained the bug, or even what environment was involved. At one point, I had found a suite of regression tests documented on the wiki, but when I asked if the tests had ever been run, no one could tell me.