|
Agile programming methodologies have reduced a software product's time to market, but have left IT and QA professionals with few tools to track of the rapid changes occurring in the applications they manage. The current development environment can't always reach the underlying application infrastructure and test the effect of changes to one part of the system on the rest of the system.
Enter iTKO's Lisa 3.5. This monster of a test environment is daunting at first, but its modular design makes building and executing new test cases for SOA implementations an almost pleasant experience. Lisa does extensive testing of many protocols related to SOAP, with a series of plug-in components that can tax SOAP 1.2 over HTTP, HTTPS and JMS (Java Messaging Service); WS-Addressing; WS-Interoperability; WS-Security, and WSDL 2.0 discovery and interoperability.
JUnit-based testing is acceptable--and necessary--for J2EE-based software, and products such as Parasoft's SOAPTest and Mindreef's SOAPScope do SOAP-specific testing. However, few products go further than exercising the external face of your SOA services. SOAPTest and SOAPScope, for example, can tell you something's wrong with a service, but they can't necessarily dig in and tell you that the real problem is with an SQL query executed by an EJB (Enterprise JavaBean) somewhere deep within the service's business logic. Lisa can, and does, provide this level of detail.
Lisa's collaborative, modularized environment supports many app-infrastructure products. Separate tests that plug into Lisa's base platform delve into EJBs, databases, JMS topics and queues, the MQ Series messaging bus, RMI objects, Microsoft COM objects, Web services, and plain old HTTP Web sites. While Borland SilkPerformer SOA Edition can exercise your SOA infrastructure, it does not cover nearly as many underlying systems. Lisa also comes with an in-container testing option, which allows you to dig deep into the application stack.
Lisa's Method
Lisa separates test cases from staging documents, a smart, convenient move that increases consistency within the test environment. Test cases define which components are tested; staging documents define how those tests are run. The staging documents can be reused across the enterprise, and allowed us to define scenarios with different numbers of users and periods of time for average and peak loads. Once scenarios were created, we could use them to test different parts of our infrastructure. This important--but uncommon--facet of load testing makes it easier to find bottlenecks by determining each subsystem's performance under various load conditions.
We exercised Lisa 3.5 within NWC Inc. and were pleased to be able to test all aspects of our Web-based widget sales systems' infrastructure. Lisa even taxed our Oracle 10g and SQL Server 2005 databases. The option to test our MQ Series messaging bus was a boon, and we could use Lisa to perform capacity testing on our entire application infrastructure. Like other products that integrate with databases and messaging products, it's necessary to load the appropriate drivers into the system. Fortunately, the process was simple and the integration painless. Lisa had no issues with our .NET and J2EE-based SOAP services, and executing tests against BEA AquaLogic Service Bus 2.1 confirmed Lisa's interoperability with other SOAP stacks.
We were particularly pleased with Lisa 3.5's ability to perform continual, automated testing--similar to the ongoing testing provided by an application performance monitoring system. Lisa 3.5 can be configured to alert staff when any piece of infrastructure fails, regardless of the deployment phase in which you are currently testing.
Lisa imparts thorough information about errors encountered while tests are being executed. For example, we ran into an error as we tried to load test services provided by our BEA ALSB (AquaLogic Service Bus) 2.1 implementation. Lisa displayed a simple error, but gave us an option to show the entire Java stack trace. Many testing tools lack this simple feature, which made it much easier to determine whether the error was in our test script or in the service. (It was in the script, of course.)
Lisa's offers wide variety of reporting formats, including PDF, HTML and CSV, but these capabilities could be improved. The simplistic real-time charts lack depth, and there's no ability to drill down into specific points from the real-time display. While the data being displayed is configurable, few of the options make sense in this type of display. Nevertheless, considering the depth of the testing and the detail provided regarding specific events, this shortcoming is not a show-shopping issue.
Like the few other complete testing suites in the market, Lisa is not priced for the faint-hearted. Pricing for load-testing capabilities starts at $35,000, and there is also per-seat pricing for workstation capabilities, ranging from $3,000 to $10,000 per user. That can quickly add up, and puts it in the same price range as Borland's SilkPerformer. But even at the asking price, Lisa can save you just as much in man-hours--and even more in soft costs incurred by defects, failures, or other errors that may be inadvertently introduced into your application infrastructure by fast-changing services. n
Lori MacVittie is a Network Computing senior technology editor working in our Green Bay, Wis., labs. She has been a software developer, a network administrator, and a member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].
