You Are Not Done Yet: Performance and Stress

You are not done testing unless...you understand the performance characteristics of your application and the manner in which your product deforms under stress. Performance testing can seem straightforward: verify the times required to complete typical user scenarios are acceptable - what's so hard about that? Simulating those scenarios sufficiently realistically can be difficult, however. Even more so when it comes to stress testing! For example, say you are testing a web site which you expect to become wildly popular. How do you simulate millions of users hitting your site simultaneously?

I do not have any easy answers for these questions. What I do have are several scenarios and conditions to consider:

Performance

• Verify performance tests exist for each performance scenario, and are being executed on a sufficiently regular basis


• Verify performance targets exist for each performance scenario, and are being met


• Verify the performance tests are targeting the correct scenarios and data points


• Verify performance optimizations have the intended effect


• Verify performance with and without various options enabled, such as Clear Type and menu animations, as appropriate


• Compare performance to previous versions


• Compare performance to similar applications

Stress

• Run under low memory conditions


• Run under low disk space conditions


• Run under out-of-memory caused via automation (e.g., a use-up-all-available-memory utility)


• Run under out-of-memory caused by real world scenarios (e.g., running multiple other applications each having multiple documents open)


• Run under a heavy user load


• Run over a network which frequently drops out


• Run over a network with a large amount of traffic


• Run over a network with low bandwidth


• Run on a minimum requirements machine


• Open, save, and execute (as appropriate) from floppies and other removable disks

As you do all of this performance and stress testing, also check for memory and other resource leaks.

Posted by The Braidy Tester at 07:30 AM  Permalink