What is Agile Testing?
Agile is a derivative of iterative development.
Iterative development is often compared to what is called ‘traditional’ development. Traditional development refers to projects where the requirements for a system are fully specified and documented up front. The system is then built and tested to these specifications. Systems being developed to fulfil legislative or safety related systems for instance, may benefit from this approach.
Iterative development is used where systems are likely to change at pace; thus attempting to define the system fully at the start may be difficult or even wasteful in time and money. Iterative development is usually part of an incremental development approach. This means that the system is built in chunks, with each chunk being tested and approved before moving on to next chunk. This is called iterative-incremental development. This way of working allows time for the users or their representatives to check that the look and feel of the system meets their needs.
Testing of the system for code quality and system functionality remains the domain of the developers and those with dedicated testing skills.
Edwards Deming was a pupil of his, honoured by the Japanese and then their Emperor for his work in transforming Japan into a manufacturing super power in the 1950s.
In the late 1980s, Deming’s System of Profound Knowledge espoused 14 points with the ultimate aim of the win:win:win. These points included:
- Reducing costs by reducing waste
- Realizing the power of intrinsic motivation
- Cultivating the value of collaboration.
Agile development is supported by the Agile Alliance – www.agilealliance.org . It came about in 2001. The Alliance expresses Agile as an umbrella term for processes following the Agile Manifesto and Principles. These include:
- Simplicity–the art of maximizing the amount of work not done–is essential
- Build projects around motivated individuals
- Customer collaborationover contract negotiation
Note the similarities here with Deming’s system. The Agile Alliance makes no claim of newness of ideas, but a coming together of them to create a way forward for the fast pace today’s world.
Scrum in software development follows similar ideas. However, it prescribes neither how to create the code nor how to test it.
Over the years that Agile has become mainstream, several processes have emerged, now considered common-place. These include:
Test Driven Development (TDD)
This does not relate to testing the code after it has been created, but rather to use tests to drive the creation of code. This is a developer domain. He or she writes tests that the code must pass in order to be deemed satisfactory. The code is written in an iterative fashion. The first test will fail, since the code will not have been written to pass it yet. The code is then written and run against the test in an iterative fashion. It will be deemed satisfactory once it passes the test. This process is repeated for other tests.
Behaviour Driven Development (BDD)
This is a derivative of TDD. As its name suggests, it focuses on creating code to meet the desired behaviour of the system. It is referred to as an ‘outside-in’ approach. This approach requires the business to define the requirements using specific constructs, such as the ‘Given a context, When an event occurs, Then execute an outcome’ style of writing.
There are other variants of the X driven development, such as Acceptance Test Driven Development etc.
Agile Testing Processes
A key tenet of Agile development is that it aims to deliver working software in small increments, frequently. This gives us a sense of the testing required. The developers must test the individual pieces of code as part of unit testing, and the end user representative will check largely the look and feel of the functionality being created in the iteration.
Integration and system testing are of course also important and necessary. The former is assisted by a process of continuous integration. This means that new pieces of code are uploaded to the main system and tested immediately. This provides a means of adding significant quality to the iterative process – experienced testers may recall that the more changes you make to a system, the more error prone it becomes. This continual checking assists in making the system regression-averse.
System testing is supported by a testing methodology
Agile Testing Methodology
The general test process broadly comprises the following:
- Test Planning
- Test Analysis and Design
- Test Execution
- Test Recording
- Test Completion.
The methods used for test planning in Agile incorporate two levels – Release Planning and Iteration Planning. The former typically describes the system to be delivered; whilst the latter shows the number of iterations required.
The testing is conducted against user stories (often following the Given; When; Then construct – e.g. ‘as a user I would like some functionality so that I can achieve some aim’) for which acceptance criteria are written. Testing contributes to both of these specifically by checking that acceptance criteria are clear and that they are testable.
Test analysis serves to identify the features to be tested. In Agile these are provided by the user stories and the acceptance criteria.
Test design techniques can be used to provide some assurance that the testing has been carried in a systematic and methodical way. The black-box test design techniques that can be used for system testing in Agile development include those used in traditional projects:
- Classification Trees – these can be used to aid in clarifying the user story and its required acceptance criteria.
- Equivalence Partitioning – this focuses testing on groupings that should exhibit similar behaviour. The classes can be identified for both input and output domains.
- Boundary Value Analysis – when a partition includes numerical boundaries, there is an additional risk that the boundaries may not be properly handled. Boundary Value Analysis checks for this.
- Decision Tables – theses show permutations of logically true and false conditions and their associated actions to be tested. These can also be derived using Cause-Effect Graphs.
- Process Cycle Test – this involves the tester working through a process from start to finish, in any order, until all identified paths have been covered.
- State Transition Testing – testing that a system changes state as defined, in response to events. These states can be physical or logical.
The extent to which the techniques above can be used will depend on the level of documentation provided in the user story and its acceptance criteria. If the documentation is lightweight, then the tester may not be able to trace the tests back to a specific requirement.
Since Agile development lifecycles bring the end user in earlier than would have been the case for traditional projects, there is an emphasis on Experience-based testing. This relies on the experience of the tester to derive the tests. They may know of previous and perhaps likely faults (as for Error Guessing), or they may be quite clear on what they want the system to do (as for Exploratory Testing). This testing can include using the black-box techniques outlined above.
Automation in Agile
- Application Lifecycle Management tools – these capture user stories, task boards and provide status updates.
- Software Build and Distribution Tools – these facilitate daily build and deployment of software
- Test Design Tools – such as mind maps to develop tests
- Test Data and Preparation Tools – These allow quick updating of data when changes occur
- Test Execution Tools – in addition to the usual test running tools, these are also used to support TDD and BDD. In addition, capture-replay tools can be used to capture experience-based tests.
- Configuration Management Tools – these store source code and both manual and automated tests to provide traceability.
- Communication tools – these supplement (or sometimes replace) the face-to-face communications. They can include desktop sharing and instant messaging.