Acceptance Test Driven Development (ATDD) 20 Sep 10

You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Acceptance test driven development is an agile software development method that uses the Acceptance tests to establish when a requirement meets the client’s expectation. It is an advanced practice of the Test Driven Development method.

Acceptance tests or Customer tests are written by the client together with the developers for every requirement that is part of the specification. Whereas a requirement describes specific behaviour the client wants in the finished software, the Acceptance tests make sure that the requirement was implemented in its completeness and according to the client’s input.
As an example we can use the tried and tested password requirement:

Requirement “When a user is creating an account, he/she is required to choose an outlandishly secure password”
Acceptance Test1 Test that the password contains at least 8 characters with at least one upper case letter, a number and a symbol”
Acceptance Test2 Test that the password does not contain a word from the english dictionary”
Acceptance Test3 Test that the user receives different error messages when the password is too short, contains no upper case letter, no number or no symbol. “
In this case the Acceptance tests define now what an “outlandishly secure password” is and increase the shared understanding for this requirement. And after the tenth iteration I can make sure that these Acceptance tests still pass so I have regression testing built right in.

The many benefits of ATDD

• Acceptance Tests increase the shared understanding of a requirement because they are a product of direct interaction between the client and the developers.
• Clarifies requirements and keeps the developers focused on what the client really wants.
• Drives out ambiguity if certain behaviour is ultimately a bug (behaviour violates either requirement or tests or both) or a change request (behaviour is acceptable within the scope of requirement and acceptance tests but ultimately undesirable).
• Software delivery is now dependent on all Acceptance tests passing and with that defines when a project is done.
• Percentage of passed acceptance tests/all acceptance tests acts as a project progress indicator.
• Free regression tests for the future iterations (especially if automated).

Some ATDD drawbacks

• Client interaction is needed which can prove difficult due to time constraints
• More work for the developer if tests are automated.
• Project progress might be slower due to additional effort.

Automating ATDD
There are many tools and frameworks for acceptance test automation, even some that can be used for User Interface testing. Many of them are free and open source, so the framework can be adapted to suit the project. Automating acceptance tests is highly desirable because regression testing is a hugely important step for iterative agile development methods. The immediate drawback is that this also means the developer has even more work to do because test automation is a development task in itself.

Are Unit tests not needed anymore?
Acceptance tests are no replacement for Unit tests. Unit tests must still be written by the developers to make sure that the code they are writing is performing as it should. Unit tests are written to test individual units of source code, whereas Acceptance tests are written to test several units of source code connected by some type of workflow defined by the client.

ATDD and outsourcing
ATDD is very much suited for outsourcing because its previously listed advantages address many of the issues encountered in outsourced software development. But it is important to notice that it will only work if the project is using an agile development method and can’t be shoehorned into a waterfall based development scenario of design first, development second, testing third. ATDD needs commitment from both the project owner and also the agile supplier because the real emphasis is not necessarily on the testing itself but on the increased shared understanding that is a product of the increased contact between the two parties. Including ATDD will also most likely increase the cost of the project at the start as more time is needed by the developers to write the additional tests. The real gain comes over time as the functionality of the delivered software should with every iteration be closer to what the client envisaged in the first place and therefore should reduce the need to add or refine functionality. The Acceptance tests also act as a baseline that can be consulted if a dispute about the scope should arise.