THE BOOK OF TESTING

Thoughts From a Braidy Tester

August 2006

Last week I listed some tests to think about when you test your application's setup program. Y'all had some excellent additions. Today I extend that list with some more specialized conditions to consider. As before, some of these terms are specific to Microsoft Windows, but other operating systems generally have similar concepts.

Local caching. Depending how the setup program was authored, it may allow setup files to be cached on the local hard drive, which speeds up subsequent repair and other setup operations.

• Verify the correct files/archives are cached


• Verify all files shared with another feature or application are handled correctly across installs, uninstalls, and reinstalls


• Verify setups for multiple programs and multiple versions of individual programs share the cache correctly. This is especially important for shared files - imagine the havoc that might ensue if uninstalling one application removed from the cache a shared file other installed applications require!


• Verify deleting a file/archive from the cache causes a reinstall and recaches the file/archive

Setup authoring. Also knows as: Test your setup program.

• Verify every possible path through the setup program (including canceling at every cancel point) works correctly


• Verify the setup program includes the correct components, files, and registry settings


• Verify any custom actions or conditions, creation of shortcuts, and other special authoring works correctly


• Verify the correct install states are available


• Verify canceling an in-progress install in fact cancels and leaves no trace of the unfinished install

Multi-user setup. What happens when multiple users mess with modify the setup configuration of your application?

• Verify your application works correctly for User2 after User1 installs/modifies/damages it


• Verify per-user features must be installed by User2 even after User1 has installed them


• Verify User2's per-user settings do not change when User1 changes them

Network setup. Can you install your app from the network rather than a local CD?

• Verify your feature uninstalls cleanly and correctly when it was installed from the network (sometimes called a post-admin install)


• Verify the correct files are installed a) on the network share, and b) locally (as appropriate), when the application is installed as Run From Network

Posted by The Braidy Tester at 07:30 AM  Permalink |

I'm rereading Jerry Weinberg's An Introduction to General Systems Thinking. One topic he discusses is sets, and how the "correct" members of a set is completely dependent on the observer. For example, take the following sequence:

{ 1, 2, 3, ... }

How many different plausible next members can you think of?

How about for this?

{ Mathew, Mark, ... }

Put your answers in the comments, or email them to me. I'll summarize your answers, provide my answers, and explain how this relates to testing next week!

Posted by The Braidy Tester at 07:30 AM  Permalink |

"I'm wondering whether the BVT Daphne suggested goes too far" Hazim said.

Hazim put the test in question back on the shared display:

Logical.Application.Start()




Logical.ColoringBooks.OpenColoringBook("BasicBook")

Logical.ColoringBooks.ChoosePage(23)




Point topLeft = new Point(42, 24)

Point topRight = new Point(350, 50)

Point bottomRight = new Point(320,300)

Point bottomLeft = new Point(70,180)

Logical.Color.DrawLine(topLeft, topRight)

Logical.Color.DrawLine(topRight, bottomRight)

Logical.Color.DrawLine(bottomRight, bottomLeft)

Logical.Color.DrawLine(topLeft, bottomLeft)




Point insideShape = new Point(150, 150)

Point outsideShape = new Point(10, 10)

Logical.Color.FillArea(insideShape)

Logical.Color.UseColor(System.Color.Magenta)

Logical.Color.FillArea(outsideShape)




Logical.Color.UseColor(System.Color.NavyBlue)

for (int x = 25 to 400)

	for (int y = 25 to 400)

		Logical.Color.Stamp(Logical.Shape.FivePointedStar, new Point(x, y))




Logical.Application.Close()

"You said that the BVT tells us whether the build is completely broken", he said.

"Yup" everyone agreed.

"It seems to me," Hazim continued, "that while using a stamp or two is important, stamping hundreds of stamps across and down the page is rather more than necessary."

"That's a good point" Oliver responded.

"Indeed" Daphne added. "I got a little carried away there. Thanks for noticing, Hazim!"

"So how do we fix it?" Jason asked. "What are you thinking, Hazim?"

"I was thinking to just stamp a couple shapes. It would look something like this, I think." Hazim deleted the nested for loop and started typing:

Logical.Color.Stamp(Logical.Shape.FivePointedStar, new Point(5, 5));
Logical.Color.Stamp(Logical.Shape.PandaBear, new Point(100, 100));

"We do one stamp partially off the page, and one well into the page. Going off the page may still be too far for the BVT, though." Hazim looked questioningly at the team.

"I think it should stay" Lucas said. "Going off the page seems like a fairly common scenario." The rest of the team made assenting noises.

"So now we have our BVT and FVTs" Hazim said. "Do we need to prioritize the rest of the test cases?"

"We could," Bianca said, "but I'll bet we can leave that to the testers. Probably they will bang all of them out in just a day or two anyway."

"Certainly we can help you *developers* write them" Jason said, somewhat archly. "They may be comprehensive-level tests, but I think they're simple enough that even a developer can puzzle them out." He was clearly joking now.

"Actually," Oliver interjected, "didn't we decide in the post-mortem from our last project to try doing dev+test pairs this time? So both of you will be writing them together, right?"

"Yeah, that's right!" Daphne exclaimed. "I'm curious to see how it works out. I'm hoping to pick up some dev tricks, and I'm looking forward to sharing some testing tricks of my own."

Bianca, ever the skeptic, didn't look completely sold. "I don't know about this, but I am willing to give it a shot. I didn't think pairing with that intern last summer would work very well either, but we both ended up learning a lot."

"So what else do we need before we start our first iteration?" Hazim asked.

"We should make our features a bit more finer grained, for one" Jason said. "The coloring features are probably okay as they are, but the other features are pretty broad right now, especially given all that auto-saving and auto-archiving we talked about."

Hazim started a To Do list. "OK, got it. What else?"

"We should find out what our ility requirements are" Daphne said.

"Ility?" Hazim asked.

"All those ilities we always have to worry about" Bianca answered. "Like accessibility, localizability, and performance - which doesn't really end in "ility" but we pretend it does because it fits with the rest so well."

"I'll bring all that up when I meet with marketing this afternoon" Hazim said. "Anything else?"

"We'll need to spin up a build machine and various other project-related infrastructure" Jason said. "I'll take care of that."

"T-shirts!" Bianca said. "We need t-shirts with our project logo on them!"

"Which means we need a project logo" said Oliver. "I already have a few ideas bouncing around my head; I'll work them up and send them around tomorrow morning."

Hazim laughed. "Definitely we need t-shirts! I'll get them ordered just as soon as we agree on a logo. What else?"

"Everything else I think we can get from our company standards" Daphne said. "Which operating systems we have to support, for example."

"If that's everything," Hazim said, "then I think it's time for lunch. On me!"

Posted by The Braidy Tester at 07:30 AM  Permalink |

You are not done testing yet unless...you have tested your program's setup process under the following conditions. Some of these terms are specific to Microsoft Windows, but other operating systems generally have similar concepts.

• Installing from a CD-ROM/DVD-ROM
• Installing from a network share
• Installing from a local hard drive
• Installing to a network share
• Installing from an advertised install, where icons and other launch points for the application are created (i.e., the app is "advertised" to the user), but the application isn't actually installed until the first time the user launches the program. Also known as "install on demand" or "install on first use".
• Unattended installs (so-called because no user intervention is required to e.g., answer message boxes), aka command line installs. This can become quite complicated, as the OS's installation mechanism supports multiple command-line options, and your application may support yet more. Oh the combinatorial testing fun!
• Mass installs, via an enterprise deployment process such as Microsoft Systems Management Server.
• Upgrading from previous versions. This can also become quite complicated depending on how many versions of your app you have shipped and from which of those you support upgrades. If all of your customers always upgrade right away, then you're in good shape. But if you have customers on five or six previous versions, plus various service packs and hotfixes, you have a chore ahead of you!
• Uninstall. Be sure that not only are all application-specific and shared files removed, but that registry and other configuration changes are undone as well. Verify components which are shared with other applications are/not uninstalled depending whether any of the sharing apps are still installed. Try out-of-order uninstalls: install app A and then app B, then uninstall app A and then uninstall app B.
• Reinstall after uninstalling the new and previous versions of your application
• Installing on all supported operating systems and SKUs. For Microsoft Windows applications, this may mean going as far back as Windows 95; for Linux apps, consider which distros you will be supporting.
• Minimum, Typical, Full, and Custom installs. Verify that each installs the correct files, enables the correct functionality, and sets the correct registry and configuration settings. Also try upgrading/downgrading between these types - from a minimum to complete install, for example, or remove a feature - and verify that the correct files etc are un/installed and functionality is correctly dis/enabled.
• Install Locally, Run From Network, Install On First Use, and Not Available installs. Depending on how the setup was created, a custom install may allow the individual components to be installed locally, or to be run from a shared network location, or to be installed on demand, or to not be installed at all (I use that one a lot for clip art). Verify that each component supports the correct install types - your application's core probably shouldn't support Not Available, for example. Mix-and-match install types - if you install one component locally, run another from the network, and set a third to Install on First Use, does everything work correctly?
• Install On First Use installs. Check whether components are installed when they need to be (and not before), and that they are installed to the correct location (what happens if the destination folder has been deleted?), and that they get registered correctly.
• Run From Network installs. Check whether your app actually runs - some apps won't, especially if the network share is read-only. What happens if the network is unavailable when you try to launch your app? What happens if the network goes down while the application is running?
• Verify that "normal" or limited-access (i.e., non-admin) users can run the application when it was installed by an administrator. Especially likely to be troublesome here are Install On First Use scenarios.
• Verify the application works correctly under remoted (e.g., Microsoft Remote Desktop or Terminal Server), and virtual (e.g., Microsoft Virtual PC and Virtual Server) scenarios. Graphics apps tend to struggle in these cases; I've seen apps just plain refuse to boot when running under Terminal Server.

Update 25 Aug 2006: Reader Jon B. suggests some additions to my list:

• Perform a Typical install followed by a Modify operation to add additional features.


• Perform a Custom install followed by a Modify operation to remove features.


• Perform a Typical install, delete one or more of the installed files, then perform a Repair operation.


• Perform a Custom installation that includes non-Typical features, delete one or more of the installed files, then perform a Repair operation.


• Patch previous versions. Patching is different from an upgrade in that an upgrade typically replaces all of the application's installed files, whereas a patch usually overwrites only a few files.


• Perform a Minor Upgrade on a previously patched version.


• Patch on a previously upgraded version.


• Upgrade a previously installed-then-modified install.


• Patch a previously installed-then-modified install.

Posted by The Braidy Tester at 07:30 AM  Permalink |

My September DDJ arrived this week. One article I especially enjoyed was Ivar Jacobson, Pan Wei Ng, and Ian Spence describing the Essential Unified Process (EssUP). Ivar, of course, is one of the geniuses|culprits who is to be thanked|blamed for bestowing|inflicting the Rational Unified Process (RUP) upon us all. Depending on your point of view, the RUP is either the best thing since sliced bread or a process-heavy monstrosity good only for killing lots of trees.

Lo and behold, Ivar and company have been listening to the debates and flames and discussions and have an answer: what they call the Essential Unified Process. I'm no RUP expert, so I can't say whether the EssUP is merely the RUP with its many components un-conjoined. Certainly I expect that the EssUP has been informed by the RUP. But the EssUP seems to be more of a, well, not even a spin-off, really, and certainly not just RUP V2.

The gist of the EssUP seems to be a) making the process the primary thing, rather than all the practices and stuff you have to do to implement the process, and b) making concerns that should be orthogonal to each other actually be orthogonal to each other. So, for example, you can take up just those processes (or parts of a process) that seem useful right now, enabling you to pick up individual pieces parts that will have a big impact now and slowly integrate your way over to full-on EssUP (which, actually, you would seem to be doing the moment you decide you only need a single small chunk of it).

Of course there's plenty of stuff you can buy to help the process along. EssUP has picked up the card theme Extreme Programming has used since ever, even going so far as to talk about dealing artifact, activity, and competency cards out to your team so they know what they're doing this iteration. This all seems a bit cutesy to me; a nicely medium-sized laminated card should make a nice reference, but is any team lead really going to pass these out to their team saying "Here's what you're doing today, have fun"? (Yes, I know, team leads will - those same team leads who don't really understand what they're doing today, but fake it with lots of process.) And perhaps all this card-related talk is more a metaphor than something expected to be done. It still seems cutesy to me.

But I do like the general idea: Here are a bunch of process-y things that might help you develop better software more rapidly. Take just those pieces that seem useful, integrate them in to your existing process, then come back for more when and if you need it. This might actually catch on!

Nah - it's much too practical. <g/>

Posted by The Braidy Tester at 07:30 AM  Permalink |

A friend recently pointed me at Bob Korn's site Responsible Thinking. Responsible thinking is really nothing more than critical thinking, but Bob named it differently in order to emphasize some key differences, namely a focus on avoiding false beliefs rather than identifying false arguments, and on inspecting and critiquing one's own thinking rather than other people's. Plus of course the common wisdom that critical thinking is dry, boring stuff! <g/>

Bob's site is plumb stuffed with interesting reading that took me days of lunches to get through. Which is fine, since it's the kind of information that I find best to read in small chunks, with lots of musing time in between. Bob sees a lack of critical thinking as one root of many of the problems society has today. He provides copious examples covering just about every topic imaginable. And if you don't agree with him on a particular point, I expect he would be the first to say you should apply Responsible Thinking to his argument in favor of Responsible Thinking!

So why is this in a blog about testing? Because critical thinking helps me test better. I find it useful for the same reason I find systems thinking useful (despite I. M. Testy's belief that it's not): they remind me to step outside of my normal context and look at my application/my team/my problem-of-the-moment from other points of view. I have yet to find a program or person that is not part of at least one system; thinking about its/their relationships to that system always (thus far, at least) helps me gain different understandings of the program/person, which oftentimes gives me ideas of new ways to break it/work with them.

Some people think systems thinking or critical thinking or insert-favorite-methodology-here is Da Bomb, others think it's bunk. Me, I'm more pragmatic: if it helps me do my job, I'll learn about it and use it but leave the philosophizing about it to others. How about you?

Posted by The Braidy Tester at 07:30 AM  Permalink |

You are not done testing yet unless...you have tested the following input methods:

• Keyboard. Duh, right? But it's important to remember that testing keyboard input doesn't just mean verifying you can type into text boxes. Scour your application for every different control that accepts text - not just as a value, but also shortcut key sequences and navigation. (Yes, there's some overlap here with Dialog Box Navigation and Accessibility.) If your application uses any custom controls, pay them especial attention as they are likely to use custom keystroke processing. Make those mouse-hating keyboard wizards happy!
• Mouse. Duh again, but again it's so obvious that it's easy to miss. And again, pay especial attention custom controls as they are likely to do custom mouse handling.
• Pen input. Depending on your target platform(s), this could mean pen input direct to your application, filtered through the operating system (e.g., the Tablet Input Panel on Microsoft Windows), and/or filtered through third-party input panels. Each input source has its own quirks that just might collide with your application's own quirks.
• Speech input. Depending on your target platform(s), this could mean speech input direct to your application, filtered through the operating system, and/or filtered through third-party speech processors.
• Foreign language input. On Microsoft Windows this usually means an Input Method Editor (IME), either the one that comes with the operating system or one provided by a third party. These can be troublesome even for applications that do not do any custom keystroke processing. For example, a Japanese-language input processor likely traps all keystrokes, combines multiple keystrokes into a single Japanese character, and then sends that single character on to the application. Shortcut key sequences should bypass this extra layer of processing, but oftentimes they don't. (Note: turning off the IME is one solution to this quandary, but it is almost never the right answer!)
• Assistive input devices such as puff tubes. The operating system generally abstracts these into a standard keyboard or mouse, but they may introduce unusual conditions your application needs to handle, such as extra-long waits between keystrokes.
• Random other input sources. For example, I have seen games where you control the action by placing one or more sensors on your finger(s) and then thinking what you want the program to do. Some of these devices simply show up as a joystick or mouse. What happens if someone tries to use such a device in your application?
• Multiple keyboards and/or mice. Microsoft Windows supports multiple mice and keyboards simultaneously. You only ever get a single insertion point and mouse pointer, so you don't have to figure out how to handle multiple input streams. You may, however, need to deal with large jumps in e.g., mouse coordinates. Oh the testing fun!

Posted by The Braidy Tester at 07:30 AM  Permalink |

Recently I booked a trip online. I briefly glanced through the confirmation email to - erm - confirm that everything was correct, which it seemed to be. Some weeks later I received another confirmation of my itinerary, and then a week later another one. This finally caught my attention and so I looked at this third itinerary more closely.

This closer inspection turned out to be important, for the itinerary was rather screwed up. It had me on two different flights for each leg of my trip, and one flight for each leg had a travel time of zero hours and a travel distance of zero miles. Additionally, the end of the itinerary included some twenty "Placeholder comment" comments.

Over the next two weeks I received three more of these.

Finally I called my travel agent to find out what was going on. It was a good thing I did, because it turns out my airline had canceled my original flights and rescheduled me on different flights that left some five hours earlier! If I hadn't noticed all this and had arrived at the airport based on the original schedule I would have been pretty grumpy with myself!

Evidently the drastic reschedule completely befuddled my travel agent's itinerary-generating software. Bad them. Airlines reschedule flights all the time - usually changing times by just a few minutes - so my receiving a flurry of updated itineraries was not out of the ordinary. I did review each update, but because I wasn't expecting much of a change my review was only cursory. Bad me.

Moral: When you're dreaming up test cases, remember to think waaaaaay out of the box. An airline replacing a flight with one half a day earlier wouldn't have been the first test case that would have come to my mind. But clearly it happens.

Moral: Strings like "Placeholder comment" probably don't belong anywhere in your production system. Checking for strings like this is easily automated, but some manual poking around is good too. I mean, would you think to include "Well, that was a dumb thing to do, stupid user you" in your list of Strings That Shouldn't Make It Into Production? (It did, true story. The customer was not amused.)

Moral: When your travel agent sends you a new itinerary, review it verrry carefully! <g/>

Posted by The Braidy Tester at 07:30 AM  Permalink |

You are not done testing yet unless…you have checked the following points for each and every dialog box in your application:

• Verify that the menu command which launches the dialog ends with an ellipsis (e.g., "Create New Document..."). This is the convention on Microsoft Windows at least; for other operating systems check your style guide.
• Verify that the size of each control and spacing between each pair of controls matches that specified by your style guide.
• Verify that the dialog box is sized correctly relative to its controls.
• Verify that ninchable controls display correctly (per your style guide) when they are ninched. (Generally, they should grey out or otherwise make obvious their ninched state.)
• Verify that any samples in the dialog reflect the actual contents and formatting of the current document. Or reconsider showing samples! <g/> Dialogs which affect document formatting often purport to preview the effect its settings will have on the active document. Bringing the actual document (or an appropriate piece, such as the current selection) into the preview greatly enhances the value of the preview. If your preview simply presents some preset, made up content that hopefully looks somewhat like the real document, you might as well not have a preview at all.

All this fit-and-finish stuff can seem like a waste of time, but it matters. Although they likely aren't conscious of it, these details affect people's evaluation of your product's quality just as much as how often it crashes does. In fact, if the first impression a potential customer has is that your application is unpolished, they will tend to view the rest of their experience through that lens as well. So polish that chrome!

Posted by The Braidy Tester at 07:30 AM  Permalink |

"So let's stop here and prioritize the test cases we have so far" Hazim suggested.

Hazim had been taking notes throughout the test case brainstorming. He switched the shared display back to his laptop. "Here's everything you all said."

What Happens When I...

• Open every coloring book
  • By browsing from the first book on the shelf to the desired book
  • By browsing from the last book on the shelf to the desired book
  • By browsing from somewhere in the middle of the shelf to the desired book
• Navigate to each page in each coloring book
  • By browsing from the first page in the book to the desired page
  • By browsing from the last page in the book to the desired page
  • By browsing from somewhere in the middle of the book to the desired page
• Name a drawing using each of the standard string and filename test values
• Name a drawing the same name as another page in its coloring book
• Name a drawing the same name as a page in a different coloring book
• Name the drawing the same as a read-only page
• Save to
  • Hard drive
  • Floppy drive
  • Zip drive
  • CD-ROM
  • CD-RW
  • DVD-ROM
  • DVD-RW
  • USB drive
  • Network share
• Print to loads of different printers
• Switch to every color from every color (full cross-product)
• Different screen resolutions
• Changes in screen resolution
• Switch from every tool to every other tool
• Use each tool exactly at and one pixel away from each corner of the page (e.g., 0,0; 1,0; 1,1; 0,1; -1,1; -1,0; -1,-1; 0,-1; 1,-1)
• Use each tool in each quadrant of the page
• Use each tool in the middle of the page
• Fill an area that is a different color from what it will be filled with
• Fill an area that is the same color as what it will be filled with
• Fill an area that has never been filled
• Fill a line
• Fill a filled area
• Fill a stamped shape
• Use every stamp shape
• Monkey tests for interaction testing

"Shall we start by defining our BVT?" he asked.

"Let's start at the very beginning, a very good place to start" Daphne sung.

Lucas grinned. "Yup, let's get that Build Verification Test taken care of."

"Daphne did that already" Bianca said. "Remember? That's what we started with."

"Oh yeah" Lucas said.

"But I think we need to expand it a bit" Daphne added. "We should include some kind of check that our support for assistive devices is really working."

"In a BVT?" Jason goggled.

"Yup" Daphne affirmed. "I think we should also check that bread still always falls butter-side down - ten times in a row should be sufficient."

"Butter? Bread?" Jason sputtered. "How does that have anything to do with...oh." He trailed off as he got the joke.

"OK, so we have our BVT then" Hazim recentered the discussion. "I think *I* could even write it, our LFM makes things so simple!"

"Looks like someone is volunteering!" Bianca smirked.

"Yea, I guess so" Hazim laughed. "On to prioritizing the rest of our tests then".

"Not quite" Jason interjected. "First we need to define our FVTs."

"FVTs? What are those?" Hazim queried.

"Feature Verification Tests" Lucas replied. "I think of them as the BVTs for each feature."

"And you would be thinking right" Jason agreed. "Just like BVTs tell us whether the build itself is worth using, FVTs tell us whether each individual feature can be tested."

"Do we really need to spell them out?" Hazim asked. "Looking at our LFM, it seems that if we simply call every LFM method with some basic value that would give us what we want."

"Spelling them out is good," Daphne replied, "because it makes clear exactly what we care about. But you're right that calling each LFM method is about all it takes. Let's see…how about this." She started inking into her copy of the shared notebook.

1. Open a coloring book near the middle of the library.
2. Choose a page near the middle of the coloring book.

"I think that's all we need for the library feature" she finished.

"That does seem to cover the most important bases" Oliver said. "Let me take a crack at the drawings FVT."

1. Open a coloring book.
2. Choose a page.
3. Set the drawing's name.
4. Close the drawing.
5. Reopen the drawing.

"It would seem we're missing an action" he noted. "We need a Logical.Drawing.Close."

"Good catch!" Jason approved. "And that seems like all we need for the drawing feature."

"What about the duplication?" Hazim asked. "We're opening a coloring book and choosing a page...well duh, I guess we kind of have to, don't we?"

"Yup" Daphne agreed. "A lot of our setup and teardown code will look alike. But we'll make that work for us."

"The FVTs for colors should be more complex" Bianca said. "The one for UseColor should use each color in every possible context - filling an area, drawing a line, and stamping a stamp. The FillArea, DrawLine, and Stamp FVTs should go through those point test cases we identified earlier. And the Stamp FVTs should stamp every stamp at each of those locations."

"That's good thinking, Bianca!' Daphne exclaimed. "But I think that's a little too detailed for an FVT. Remember that the point of an FVT is simply to check whether a feature is worth testing. You described several good comprehensive tests, but I think this is sufficient for the color FVT:"

1. Open a coloring book.
2. Choose a page.
3. Fill an area covering approximately one-third of the page.
4. Change to a different color.
5. Draw a line roughly around the filled area.
6. Change to a different color.
7. Stamp three different shapes once each on the filled area.

"I think I get it" Bianca said. "As long as at least one stamp works we can test most of the stamp feature, so the FVT doesn't need to check that they all work."

"You got it!" Daphne agreed.

"The comprehensive tests we outlined above don't involve much interaction between features" Lucas noted. "It seems like we should do some testing to see what happens if, for example, we draw a line over itself, or over a previously stamped stamp, and so on."

"Definitely!" Oliver replied. "I remember we did some - was it, gorilla testing? - last time, that gave us much of that testing by randomly clicking around the app."

Lucas laughed. "That was *monkey* testing, Oliver! This app seems eminently monkey testable as well - this application doesn't seem like it will have any holes to trap the monkey."

"Definitely!" Jason chimed in. "It won't take more than an hour to get this LFM plugged into our monkey infrastructure, I don't think. Daphne and I can get started on that just as soon as the BVT and FVTs are coded. And of course we'll all be doing gobs of manual banging around as well, which will also hit all sorts of interesting conditions."

Hazim had been only half-listening to all this. "I'm wondering whether the BVT Daphne suggested goes too far."

Posted by The Braidy Tester at 07:30 AM  Permalink |

You are not done testing yet unless…you have checked the following points for each and every dialog box in your application:

• Verify that the correct system controls display on the title bar (e.g., some dialog boxes can be maximized while others cannot) and work correctly.
• Verify that the default edit-focused control and default button-focused control are correct.
• Verify that the dialog can be canceled by
  • Pressing the Escape key (regardless of which control has focus)
  • Pressing the system close button (the 'x' button on Microsoft Windows dialog boxes)
  • Pressing the cancel button on the dialog
• Verify that the dialog can be closed and its contents accepted by
  • Pressing the Enter button (regardless of which control has focus, although multi-line text boxes may need an exemption)
  • Pressing the accept or OK button on the dialog
• Verify that the keyboard navigation order is correct. (Microsoft Windows apps often refer to this as "tab order" as the convention on that operating system is that pressing Tab moves you through the dialog box.)
• Verify that every control has a shortcut letter, that every shortcut works, and that each shortcut is unique within the dialog box.
• Verify that each control's tooltip is correct.
• Verify that any mutually exclusive controls work together correctly.
• Verify all dialog states, such as different sets of controls being visible due to application state or "more details" and "less details" buttons on the dialog box being invoked.
• Verify that all ninchable controls (controls which can be in an indeterminate state; for example, the Bold button would be ninched if the current selection contains some text that is bolded and some text that is not bolded) do in fact ninch as appropriate.
• Verify that editing a ninched value has the correct effect (i.e., applies the new value(s) to all items which should be affected; to revisit the text example, all text should be bolded).
• Verify that each control responds correctly to valid input and invalid input, including appropriate boundary cases. For example, invalid input might cause a message box to be displayed, or highlight the control in some fashion.
• Verify that the dialog displays and functions correctly
  • With different color settings
  • With different font settings
  • In high contrast mode
  • In high DPI mode
• Verify that all images and other media in the dialog box are localized correctly.

  Posted by The Braidy Tester at 07:30 AM  Permalink |

  
  August 02, 2006
  

  Yes, I'm A Wein-borg

  
  

  I freely, willingly, and proudly admit to being a Jerry Weinberg follower and fan. Jerry not only was here when the first computers arrived, he was actually a computer himself! (That was the name given to people who calculated numbers all day, you see.)

  Not only has Jerry been around since the beginning of the computer era, he has been dispensing wisdom regarding programming and interpersonal relationships (because even when you think you have a software problem, you really have a people problem) that entire time. Jerry travels the world helping teams in the flesh, but his wisdom is also available via a plethora of books. I consider the following to be indispensable:
  

  • Are Your Lights On? (with Don Gause). Don't be fooled by the easy read - this book is packed with ideas for powering up your thinking process.
  
  
  • Becoming a Technical Leader. If you want to be one, this book will help you get there. If you think you already are one, this book will help you be a better one.
  
  
  • An Introduction to General Systems Thinking. Don't start reading this unless you are prepared to start seeing systems everywhere you go.
  
  
  • Secrets of Consulting and More Secrets of Consulting. Oh, you're not a consultant, you say? Read these books anyway. While a few consultant-specific topics (such as setting your price sheet) are covered, mostly these books are about working effectively with people. And who couldn't use some tips about that?
  
  
  • What Did You Say? (with Charles N. Seashore and Edith Whitfield Seashore). Those three of you who never have to give feedback, don't bother reading this book. Everybody else, though, if you would like your feedback to be more effective, do yourself a favor and read - nay, study - this book's teachings.
  
  
  • Weinberg on Writing. Almost certainly your job involves some form of writing. If you'd like that writing to be better, this book will help. Even if you only write status reports!

  While these titles are top of my list, really each of Jerry's books is a great read. Unless of course you don't want to learn anything...
  

  Posted by The Braidy Tester at 07:30 AM  Permalink |