Better, Stronger, Faster Integration Tests
Users don't use the software one object at a time. They don't deliberately try to break it. They just want to get their work done. But to do that work, a user typically has long, complex interactions with the software. It's these complex interactions that generate the "off-the-wall" bugs that always seem to catch the programmers off guard. It's common for a programmer to be surprised when he finds out how the user actually uses the software, and it's not uncommon for that use to differ significantly from what the programmer intended. That's why it's important to test the software in a way that gets all of its components interacting together. And that's the goal of this practice for integration testing.
This method of integration testing gets the programmers working together in a way that resembles a scaled-down version of a full test cycle. Before it starts, the team should be relatively confident that they have completed their programming, and that the code is ready to roll. This practice requires roughly a day of work for the entire team. To make sure that it only takes about a day (or two), every single developer should take part in this test. A larger team will expend more effort without needing more calendar time. Note that if the project is very large, it usually makes sense to break it into multiple chunks, and have a separate integration test at the end of each one. (A good rule of thumb for sizing each chunk is that if it seems like the integration tests will take far more than two days to do, then the project should probably be broken down further.)
Table 1 shows the steps involved in running the integration tests.
Preparation
1) Each programmer on the team (including the team lead) writes down a list of features that they worked on, and comes up with two to five test cases for each feature.
2) The team meets to combine their results. The first half of the meeting is used to review everyone's work. The team uses the rest of the meeting to brainstorm new test cases that were missed.
3) The test cases are divided into packets for each team member to test, and distributed to the team. No team member executes a test case that they wrote " everyone executes tests written by others.
4) The lead creates an empty spreadsheet for storing test results, which is placed on a shared drive for everyone to update. (If there's already a bug tracking database like Bugzilla installed, use that instead!)
Execution and review
1)The team cuts a build and distributes it to the developers.
2) Each team member runs through one packet of test cases.
3) When a team member finds any behavior that does not match his or her expectations, a bug is added to the spreadsheet (or bug tracker).
4) The team meets to talk about which bugs to fix.
|
Table 1: Running a better, stronger, faster integration test.
The test is divided into two stages. The goal of the preparation stage is to create a packet of test cases for each team member. A typical test case includes these sections:
- A unique name and number.
- Preconditions that describe the state of the software before the test case (which is often a predecessor test case that must always be run first).
- The specific steps that make up the interaction.
- Expected results that describe the state that the software should end up in after the test case is executed.
Table 2 is an example of a test case that would exercise one specific behavior in a word processing application. In this example, the programmers intended the word processor's search-and-replace feature to work in such a way that if the original text had been all lowercase, then the replacement text had to be inserted in all lowercase.
Name |
Test #7: Verify that lowercase data entry results in lowercase insert |
| Preconditions | The test document TESTDOC.DOC in the share drive is loaded. (Line #38 has the text "this is the search term" in it.) |
| Steps | 1. Click on the "Search and Replace " button. |
| 2. Click in the "Search Term " field. | |
| 3. Enter This is the Search Term | |
| 4. Click in the "Replacement Text" field. | |
| 5. Enter This IS THE Replacement TeRM | |
| 6. Verify that the "Case Sensitivity" checkbox is unchecked | |
| 7. Click the OK button | |
| Expected Results | The search-and-replace window is dismissed, and the text this is the search term has been replaced by this is the replacement term in all lowercase. |
Table 2: Example of an integration test case
