Acceptance Test Driven Development and the cost of (not) testing 20 Jul 09
All requirements that were created in the last step have one issue, they describe in (hopefully) simple words the business requirement that was created using the insight of end user and the developer. But this is a description of functionality as it should happen in the best case. So there will be a good bit of ambiguity in a requirement.
This is where acceptance tests are coming in. At the start they are simply notes about what should be tested for every requirement. Acceptance tests are later often used as the criteria to verify whether a requirement was fully implemented and this makes a lot of sense. Because the initial tests are described again in plain english and outside of a technical framework, the end users should define the bulk of them. Also new tests should be added over time to cover all eventualities.
Acceptance tests should be defined before a line of code is written as they provide a lot of information for the developer that could be interpreted differently due to requirement ambiguity. They also create boundaries for the requirement and limit the amount of work that is needed for a given requirement. Tests should be integrated into a test automation framework so they can be reused for every iteration and with that become automated regression tests that make sure any new iteration did not break what was developed in an earlier iteration. There are many frameworks out there like Fit, Fitnesse and Selenium that can help with this automation.
But besides Acceptance Tests, it is important to request that the developers also write lower level unit tests during development. When combining both lower level unit tests and higher level acceptance tests quite a good amount of test coverage is achievable.
Unfortunately testing comes with a steep price tag. Most outsource companies slapped on between 25-50% for extended repetitive manual testing (poor testers). It has to be said that requesting proper automated unit testing and acceptance testing is a development step in itself because tests have to be written in much he same way as the rest of the software. But as a project goes over several iterations, regression tests will help finding issues early rather then at the last test run. In our opinion paying for testing should mean that the tests are automated and can be reused in later iterations or even releases. Manual testing is error prone and one of the least motivating pieces of work and should be best avoided if possible.