You Are Not Done Yet: APIs

If your application installs EXEs, DLLs, LIBs, or any other kind of file - which covers every application I've ever encountered - you have APIs to test. Possibly the number of APIs *should* be zero - as in the case of for-the-app's-use-only DLLs, or one - as in the case of an EXE which does not support command line arguments. But - as every tester knows - what should be the case is not always what is the case.

• Verify that all publicly exposed APIs should in fact be public. Reviewing source code is one way to do this. Alternatively, tools exist for every language to help with this - Lutz Roeder's .Net Reflector is de rigeur for anyone working in Microsoft .Net, for example. For executables, start by invoking the application with "-<command>", ":<command>", "/<command>", "\<command>" and "<command>" command line arguments, replacing "<command>" with "?" or "help" or a filename. If one of the help commands works you know a) that the application does in fact process command line arguments, and b) the format which it expects command line arguments to take.
• Verify that no non-public API can cause harm if accessed "illegally". Just because an API isn't public does not mean it can't be called. Managed code languages often allow anyone who cares to reflect into non-public methods and properties, and vtables can be hacked. For the most part anyone resorting to such tricks has no cause for complaint if they shoot themselves in their foot, but do be sure that confidential information cannot be exposed through such trickery. Simply making your decryption key or license-checking code private is not sufficient to keep it from prying eyes, for example.
• Review all internal and external APIs for your areas of ownership. Should they have the visibility they have? Do they make sense? Do their names make clear their use and intent?
• Verify that every public object, method, property, and routine has been reviewed and tested.
• Verify that all optional arguments work correctly when they are and are not specified.
• Verify that all return values and uncaught exceptions are correct and helpful.
• Verify that all objects, methods, properties, and routines which claim to be thread safe in fact are.
• Verify that each API can be used from all supported languages. For example, ActiveX controls should be usable (at a minimum) from C++, VB, VB.Net, and C#.
• Verify that documentation exists for every public object, method, property, and routine, and that said documentation is correct. Ensure that any code samples in the docs compile cleanly and run correctly.

Posted by The Braidy Tester at 07:30 AM  Permalink