- Automated Functional Tests
- Outside-In Development
- Where Do We Start?
- References
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Automated Functional Tests
One side effect of ATDD is automated functional tests. Most teams work with automated tests nowadays. Unfortunately, this means different things in different companies—and even to different teams in the same company this might have completely different meanings. While one team might indeed mean capture-and-replay tests that usually become too brittle for Agile development teams; others might talk about developed tests using some kind of unit or functional testing framework.
In my experience, functional test automation becomes nearly impossible without some prerequisites, especially when working with a legacy code base. A few years ago, Lisa Crispin mentioned to me in a personal conversation that she considered continuous integration and a fully automated build an industry-wide accepted standard. At that point, I was working in a project that didn't have a common way to build their application, so programmers new to the project were unable to set up a development environment and start being productive immediately. Likewise, with most teams new to Agile software development, right from the start I work on practices such as test-first programming and a build that takes at most 10 minutes to compile and execute all the automated unit tests.
I don't consider these practices to be mandatory before we start automating the first functional tests in a framework that customers can also understand. (See my article "Getting Started with ATDD: Overcoming the Biggest Mistakes Right from the Start" [7] for a discussion.) But I see too many teams with automated build problems, unstable and brittle continuous integration cycles, and long build times taking these issues more seriously than building their functional test automation. As long as these teams suffer from problems underlying their build systems and continuous integration cycle—even their design and architecture—they'll spend too little time on their automated functional tests. Therefore they'll be brittle and fragile, and they'll fail often. In the end, these teams will face so many problems that it will be hard for them to focus, or they'll focus on the wrong things—they might even consider Agile methodologies a failure for them.
For all these reasons, I wouldn't start with ATDD and functional test automation right from the beginning for a team that's new to Agile development. But I would work hard with the team to overcome their primary problems—the build system, the continuous integration system, and their architecture and design—in order to prepare them to work as closely as possible with the examples derived from customers in their automated functional tests.
Regarding Kent Beck's corollary practices, I feel confident putting ATDD into place once the team has managed to put code and tests together into their shared codebase. Ideally, I would like to have collective code ownership first, but I can achieve collective code (and test) ownership alongside with implementing an ATDD approach in the team. Once every team member is able to read and understand the examples from other team members, the team will be ready to get a commonly shared codebase as a side effect.
Regarding automated functional tests, I consider ATDD to be a second-level practice, as compared to Beck's XP practices. Although some teams might not be ready to deal with a fully automated build and at least one daily submission of each programmer to the codebase, the burden to implement ATDD for the team's functional tests should be pretty low in this regard.