Wherein tests are written by everyone, even a monkey

"So let's stop here and prioritize the test cases we have so far" Hazim suggested.

Hazim had been taking notes throughout the test case brainstorming. He switched the shared display back to his laptop. "Here's everything you all said."

What Happens When I...

• Open every coloring book
  • By browsing from the first book on the shelf to the desired book
  • By browsing from the last book on the shelf to the desired book
  • By browsing from somewhere in the middle of the shelf to the desired book
• Navigate to each page in each coloring book
  • By browsing from the first page in the book to the desired page
  • By browsing from the last page in the book to the desired page
  • By browsing from somewhere in the middle of the book to the desired page
• Name a drawing using each of the standard string and filename test values
• Name a drawing the same name as another page in its coloring book
• Name a drawing the same name as a page in a different coloring book
• Name the drawing the same as a read-only page
• Save to
  • Hard drive
  • Floppy drive
  • Zip drive
  • CD-ROM
  • CD-RW
  • DVD-ROM
  • DVD-RW
  • USB drive
  • Network share
• Print to loads of different printers
• Switch to every color from every color (full cross-product)
• Different screen resolutions
• Changes in screen resolution
• Switch from every tool to every other tool
• Use each tool exactly at and one pixel away from each corner of the page (e.g., 0,0; 1,0; 1,1; 0,1; -1,1; -1,0; -1,-1; 0,-1; 1,-1)
• Use each tool in each quadrant of the page
• Use each tool in the middle of the page
• Fill an area that is a different color from what it will be filled with
• Fill an area that is the same color as what it will be filled with
• Fill an area that has never been filled
• Fill a line
• Fill a filled area
• Fill a stamped shape
• Use every stamp shape
• Monkey tests for interaction testing

"Shall we start by defining our BVT?" he asked.

"Let's start at the very beginning, a very good place to start" Daphne sung.

Lucas grinned. "Yup, let's get that Build Verification Test taken care of."

"Daphne did that already" Bianca said. "Remember? That's what we started with."

"Oh yeah" Lucas said.

"But I think we need to expand it a bit" Daphne added. "We should include some kind of check that our support for assistive devices is really working."

"In a BVT?" Jason goggled.

"Yup" Daphne affirmed. "I think we should also check that bread still always falls butter-side down - ten times in a row should be sufficient."

"Butter? Bread?" Jason sputtered. "How does that have anything to do with...oh." He trailed off as he got the joke.

"OK, so we have our BVT then" Hazim recentered the discussion. "I think *I* could even write it, our LFM makes things so simple!"

"Looks like someone is volunteering!" Bianca smirked.

"Yea, I guess so" Hazim laughed. "On to prioritizing the rest of our tests then".

"Not quite" Jason interjected. "First we need to define our FVTs."

"FVTs? What are those?" Hazim queried.

"Feature Verification Tests" Lucas replied. "I think of them as the BVTs for each feature."

"And you would be thinking right" Jason agreed. "Just like BVTs tell us whether the build itself is worth using, FVTs tell us whether each individual feature can be tested."

"Do we really need to spell them out?" Hazim asked. "Looking at our LFM, it seems that if we simply call every LFM method with some basic value that would give us what we want."

"Spelling them out is good," Daphne replied, "because it makes clear exactly what we care about. But you're right that calling each LFM method is about all it takes. Let's see…how about this." She started inking into her copy of the shared notebook.

1. Open a coloring book near the middle of the library.
2. Choose a page near the middle of the coloring book.

"I think that's all we need for the library feature" she finished.

"That does seem to cover the most important bases" Oliver said. "Let me take a crack at the drawings FVT."

1. Open a coloring book.
2. Choose a page.
3. Set the drawing's name.
4. Close the drawing.
5. Reopen the drawing.

"It would seem we're missing an action" he noted. "We need a Logical.Drawing.Close."

"Good catch!" Jason approved. "And that seems like all we need for the drawing feature."

"What about the duplication?" Hazim asked. "We're opening a coloring book and choosing a page...well duh, I guess we kind of have to, don't we?"

"Yup" Daphne agreed. "A lot of our setup and teardown code will look alike. But we'll make that work for us."

"The FVTs for colors should be more complex" Bianca said. "The one for UseColor should use each color in every possible context - filling an area, drawing a line, and stamping a stamp. The FillArea, DrawLine, and Stamp FVTs should go through those point test cases we identified earlier. And the Stamp FVTs should stamp every stamp at each of those locations."

"That's good thinking, Bianca!' Daphne exclaimed. "But I think that's a little too detailed for an FVT. Remember that the point of an FVT is simply to check whether a feature is worth testing. You described several good comprehensive tests, but I think this is sufficient for the color FVT:"

1. Open a coloring book.
2. Choose a page.
3. Fill an area covering approximately one-third of the page.
4. Change to a different color.
5. Draw a line roughly around the filled area.
6. Change to a different color.
7. Stamp three different shapes once each on the filled area.

"I think I get it" Bianca said. "As long as at least one stamp works we can test most of the stamp feature, so the FVT doesn't need to check that they all work."

"You got it!" Daphne agreed.

"The comprehensive tests we outlined above don't involve much interaction between features" Lucas noted. "It seems like we should do some testing to see what happens if, for example, we draw a line over itself, or over a previously stamped stamp, and so on."

"Definitely!" Oliver replied. "I remember we did some - was it, gorilla testing? - last time, that gave us much of that testing by randomly clicking around the app."

Lucas laughed. "That was *monkey* testing, Oliver! This app seems eminently monkey testable as well - this application doesn't seem like it will have any holes to trap the monkey."

"Definitely!" Jason chimed in. "It won't take more than an hour to get this LFM plugged into our monkey infrastructure, I don't think. Daphne and I can get started on that just as soon as the BVT and FVTs are coded. And of course we'll all be doing gobs of manual banging around as well, which will also hit all sorts of interesting conditions."

Hazim had been only half-listening to all this. "I'm wondering whether the BVT Daphne suggested goes too far."

Posted by The Braidy Tester at 07:30 AM  Permalink