THE BOOK OF TESTING

Thoughts From a Braidy Tester

April 2006

Walking home last week I noticed a magazine strewed across the dashboard of a parked car. The name of the magazine: Anabolic Insider.

Evidently there's a trade mag for steroid users! My biases are showing here, but I just don't understand why someone would repeatedly stick themselves with needles full of stuff that give you zits, makes you sterile, and makes your eyes bug out just for giant muscles.

Of course, people often think testers are just as crazy. Why in the world would a person subject themselves to applications that crash, trash their machine, and otherwise blow chunks so badly you need a crash helmet?

Because it's fun! And of course testing doesn't have the lasting negative after-effects other forms of torture have. Unless you call being able to reinstall your operating system blindfolded a negative after-affect! <g/>

Have a good story yourself? Email me and I'll highlight it here!

Posted by The Braidy Tester at 07:30 AM  Permalink |

Last week's quiz was testing bold in a word processor. Think this is simple to test? Try these test cases on for size:

• Single character
• Entire word
• Part of a word
• Space
• Tab
• Bullet
• Entire sentence
• Part of a sentence
• Punctuation
• Hyperlinks
• Entire paragraph
• Bulleted paragraph
• Multiple paragraphs
• Entire page
• Entire document
• Selection includes bold + not bold
• Characters from each Unicode range
• Removing bolding of above does
• UI response when entire selection is bolded
• UI response when partial selection is bolded
• Single undo
• Single redo
• Multiple undos
• Multiple redos
• Redo more times than you've undone
• Save/close/reopen
• Save to a previous version of the application
• Migrate to a new version of the application
• Each of the above across cut/copy/paste to/from owner document
• Each of the above across cut/copy/paste to/from other document
• Each of the above across cut/copy/paste to/from other application
• Each of the above across save to/load from other document types (e.g., HTML, plain text)
• Backspace into a bolded line, then continue typing
• Backspace a bolded line into a non-bolded line, then continue typing
• Backspace a bolded line into a bolded line, then continue typing
• Combine with font variations, including italics, fonts, font sizes
• Combine with paragraph variations such as text alignment and line spacing
• Combine with page variations such as margins and columns
• Bolding via hotkeys (typically Control+B in applications that run on Microsoft Windows)
• Unbolding via hotkeys
• Print
• Bold and unbold portions of read-only document
• Bold and unbold portions of a Document Rights Management managed document
• Check what affect each of the above tests has on file size.
• Check what affect each of the above tests has on resource (e.g., memory, threads) usage
• Check whether performance seems reasonable (or, if you actually have a specification, whether performance meets the parameters set by the spec)
• Investigate whether the operating system or other portions of the platform affect the user's ability to bold and unbold text.

To quote participant Sherman, "Boy there is many…". Kudos to Sherman for not only getting just about all of these cases but also finding two bugs! Although it looks like one's fixed in the new version of Microsoft Word and I think the other may be by design. <g/>

Posted by The Braidy Tester at 07:30 AM  Permalink |

You are not done testing yet unless...

You have tested undo and redo. If your application doesn't support undo then you're off the hook, but otherwise, be sure you've done the following:

• Consider whether each user action should be undoable.


• Consider whether each user actions should be redoable.


• Tested one level of undo


• Tested multiple levels of undo


• Tested one level of redo


• Tested multiple levels of redo


• Redo more times than you've undone. In some applications redo is more a "do again".


• Tested intermixed undos and redos


• Verified that each undoable and redoable command is listed correctly in the undo and redo UI


• Tested undo and redo across document saves (some applications toss their undo and redo stacks when you save)


• Tested undo and redo across document close+reopen


• Tested undo and redo across builds, if your application builds code or uses built code (such as allowing the user to reference custom control libraries). The issue here is that the contents of that built code might change - how do you redo an addition of a custom control that no longer exists in the library?

Simple undo/redo testing is easily done manually and will usually find bugs. These bugs are typically simple programmer errors which are easy to fix. The really interesting bugs are usually found by intermixing undos and redos. This can certainly be done manually, but this is one case where automated test monkeys can add value.

You can decide to have one person test undo and redo across your entire application, but I find it works best to have each person test undo and redo for their areas.

Posted by The Braidy Tester at 07:30 AM  Permalink |

<sung>Today is Thursday, and you know what that means. We're going to have a special guest (special guest).</sung>

If you ever watched the Mouseketeers you might recognize that bit o' song. I didn't watch them, but for some reason we had one of their records, which I listened to over and over and over. ("Meeska mooska mousketeers!") Evidently they had special guests each Thursday and they sung the above jingle when introduction time came 'round.

Today isn't Thursday, and I don't have a special guest. Instead I'll post links which are especially relevant to testers and others interested in testing.

Today's link is Hans Bjordahl's very funny comic Bug Bash, which is also published in an internal newsletter. A recent strip is the best visualization of Ultimate Testerness I've seen. Hans gave me permission to post it here since it's not available on his site yet:

Posted by The Braidy Tester at 09:30 AM  Permalink |

Noticing unexpected side effects is an important part of testing. True story: back in the day, a tester was throwing random processor instructions at his operating system, just to see what would happen.

As the day wore on he noticed that one particular service was crashing with some frequency. That seemed odd, so he started poking at it. Eventually he narrowed things down to one particular processor instruction which had unearthed an obscure bug deep in the kernel. Which turned out to be a nasty security bug.

Bugs like these make a tester's day. But we would be even happier if they weren't there to find!

Have a good story yourself? Email me and I'll highlight it here!

Posted by The Braidy Tester at 09:30 AM  Permalink |

Perhaps you're wondering who this Braidy Tester guy is and what I'm doing here. I'm currently a Test Technical Lead (http://www .thebraidytester.com/ usp.html translates that into what I do every day) for Microsoft Expression (Microsoft's suite of tools for professional designers and artists).

Previous to that I tested Microsoft Visio and IntelliCAD, and I was an early alpha- and beta- tester for AutoCAD 14. I've published several articles over the years (my web site has links), and I've been blogging on MSDN for several years now.

First and foremost, though, I'm a tester! I love testing and can't imagine doing anything else. Developers tell me I have the coding chops to join their ranks, but I just don't see any fun in that. There are plenty of devs lying about; what this world really needs is more great testers!

Posted by The Braidy Tester at 09:30 AM  Permalink |

Say you are responsible for testing text formatting in a word processor. Let's keep it simple for now and just look at boldfacing the text. How many different test cases can you come up with? Put your answers in the comments, or email them to me. I'll summarize and answer next week!

Posted by The Braidy Tester at 09:30 AM  Permalink |

You are not done testing yet unless...

You have checked for invalid characters in filenames, and for reserved filenames. Operating systems tend to get grumpy if you try to use wildcards (e.g., '*') in filenames. They may also treat certain filenames specially. For example, Microsoft Windows provides a single API for creating/opening files, communication ports, and various other cross-process communication mechanisms. Well-known communication ports (e.g., COM1) are addressed by "filename" just as though they were a file - kinda handy, but it means that you can't use "COM1" for a physical file on disk.

Testing for this is easy: brainstorm a list of interesting test cases, then slap each one into each of your application's dialog boxes, command line arguments, and APIs that take a filename. Illegal characters will probably throw an error, but trying to open a reserved filename is likely to hang your app.

See the MSDN topic "Naming a file" for the full skinny on reserved characters and filenames on Microsoft operating systems.

Posted by The Braidy Tester at 09:30 AM  Permalink |

Hello, my name is Michael Hunter and I'm a tester. If I had a theme song, this is what it would be:

(With apologies to Monty Python...)

[Fade in to a dev banging away on his keyboard muttering about linked lists and hash tables and third-order dependencies. A management type pokes his head in and tells the dev the feature he almost has finished has been revised into something completely different. When the management type leaves, the dev turns back to his monitor, stares at it for a few seconds, then throws up his hands and says...]

Oh for crying out loud. I never wanted to be a dev anyway. What I *really* want is to be a...tester!

[And then he breaks into song, complete with chorus and full harmonies and everything:]

I'm a tester and I'm okay,
I sleep all night and I work all day.
(He's a tester and he's okay,
he sleeps all night and he works all day.)

I look for bugs, I log those bugs, I do it all with glee.
And the really good ones, I show off repeatedly.
(He looks for bugs, he logs those bugs, he does it all with glee.
And the really good ones, he shows off repeatedly.)

I'm a tester and I'm okay,
I sleep all night and I work all day.
(He's a tester and he's okay,
he sleeps all night and he works all day.)

I pester PMs, I pester devs, I make them despair and cry.
I know I've done my job, when I hear them wail "Why oh why?"
(He pesters PMs, he pesters devs, he makes them despair and cry.
He knows he's done his job, when he hears them wail "Why oh why?")

I'm a tester and I'm okay,
I sleep all night and I work all day.
(He's a tester and he's okay,
he sleeps all night and he works all day.)

I black box test, I white box test, I write docs and PR too.
I wish I was in marketing, 'cause then I'd rule the roost.
(He black box tests, he white box tests, he writes docs? and PR too??
He wishes he was in marketing??? Where's my Nerf gun? Get him!!!)

[The dev runs out, chased by the backup singers who continue to sing:]

He's a tester and he's okay,
he sleeps all night and he works all day.
He's a tester and he's okaaaaaaaaaaaaaaaaaay.....
he sleeps all night and he works all day.

Thank my demented mind if you have that running through your head all day....<g/>

Posted by The Braidy Tester at 09:30 AM  Permalink |