With us today is Hon Wong, CEO and co-founder of Symphoniq, a company that specializes in managing Web application performance and infrastructure. Hon holds dual degrees in electrical engineering and industrial engineering from Northwestern University and an MBA from the Wharton School at the University of Pennsylvania.
DDJ: Hon, in a recent blog you quoted market-research firm Ovum as saying that 20 percent of all SOA deployments are so mired in complexity that they fail to deliver on the promise of cost savings and agility. How does this compare to your experience?
HW: Although we are at the tip of the SOA deployment iceberg, some of the IT professionals I talked to are already facing the "side effects" of the complexity caused by SOA deployment. The attraction of SOA is certainly the ability to use and reuse existing application services based on industry standard protocol and messaging format, like SOAP and XML, without having to reinvent the wheel. The bad news is that one does not know the performance characteristics of these services because the original developers are long gone or they are supplied by third parties. In effect, these Web service modules are just functional black boxes to the developers. As a result, how can one guarantee that these are not "weak links" in terms of application performance and availability? And if they are, how can IT proactively detect the problems and identify the offending services before they impact end user satisfaction? This lack of visibility negatively impacts the quality and speed of application deployment, and negates potential cost savings or gain in agility expected from SOA deployment. A better approach is to implement, as part of an SOA deployment, a monitoring solution that measures the application performance as experienced by real users, and if any ill-performing transaction is detected, be able to trace it from the browser to the database through the layers of Web service calls so that the hidden problem can be easily isolated.
DDJ: Likewise, you quote Forrester as saying that 25 percent of Web applications are created with performance deficiencies. Do you agree?
HW: I absolutely agree. In fact, the number of Web applications with performance defects should increase over time as new technologies or techniques like .NET, J2EE, SOA, AJAX, and Flash "mashups" are used to shorten time to market and deliver richer user experience. The drive towards more efficient application delivery infrastructure does not help either. As virtualization techniques are deployed to increase infrastructure utilization and consolidate servers, or when third-party services like content delivery network (CDN) or Software as a Service (SaaS) vendors are used to deliver critical functionalities, IT's ability to ferret out performance bottlenecks within the Web application or infrastructure is severely curtailed. And there's more bad news. Web applications are becoming more business critical as enterprises and organizations alike are pushing business functions onto the Web to reduce cost, and at the same time, Web users are getting less tolerant of performance snafus. As a result, while more applications are created with performance deficiencies because of technological complexity, the consequence of performance issues is becoming more severe at the same time. IT should implement effective Web application performance solutions from the get-go, and not wait till there is a crisis situation.
DDJ: One last quote from a market-research firm: Gartner says that 30 percent of developer's time is spent fixing production problems? Right or wrong?
HW: I think Gartner is being charitable in its assessment. In dealing with complex Web application performance problems, developers as well as network, server, and database administrators, and other functional experts are frequently asked to serve on triage teams. By analyzing and correlating gigabytes worth of log files and/or data collected by server or network monitoring tools, the triage team attempts to recreate the problem and assign blame to the guilty (or sometimes innocent) party responsible for fixing the problem. Given the complexity of Web applications and their delivery infrastructure, this triage team approach could easily turn into a "blame game" with no immediate tangible results other than fostering ill-will among the various functional groups. As a result, developer's time is being wasted on attending meetings, speculating on the differences between the development and the production environments, shifting blame, or doing everything but what he/she is tasked to do, that is, developing new applications.
DDJ: Okay, so what's the solution? What are architects and developers supposed to do in the face of these challenges?
HW: For Web applications, there is a two-step solution. First of all, IT has to be able to detect Web application performance problems by measuring response time directly at user's browser. The legacy approach of individually monitoring the up-time of various infrastructure silos like network, servers, database, etc., just won't cut it any more because of application and infrastructure complexity. As a second step, ill-performing transactions detected at the real user's browser is tagged and traced through the entire infrastructure from browser to database so that the cause of slow-downs can be isolated. Architects and developers can use this two-step approach during application deployment to monitor load-generated traffic or beta users. This will help them identify performance bottlenecks in the production infrastructure. Furthermore, problematic or most frequently used method calls or SQL queries can also be identified to get more bang-for-the-buck in performance tuning or optimization activities. After the Web application is deployed, this two-step approach can be used by operations personnel to detect and isolate problems in production. As a result, only issues that can be attributed to application issues are referred to the developers for remediation, drastically reducing the time developers spent dealing with production issues.
DDJ: Architects and developers typically don't receive direct feedback from end users on Web application performance other than in crisis situation. What can they do to avoid being blindsided?
HW: With the exponential growth in Web application complexity, architects and developers have a vested interest in maintaining the performance of applications, else risk spending all their time fighting fires. There are two different ways they can avoid being blind sighted by performance issues. The traditional approach is to build monitoring capability directly into the application to measure key performance parameters. In addition, a data collector, UI, and reporting tools will have to be built to collect and analyze the volume of performance data. Even if there is development bandwidth to provide these extra functionalities, the onus will be on the developers to maintain and up-grade the monitoring code as the application and/or infrastructure are revised. Furthermore, this approach is not practical if third-party modules or packaged applications are incorporated into the application as it is difficult to convince third parties to build and maintain performance monitoring functions in their code. I can recall a social networking site faced such a quandary. Instead, they decided to implement a dynamic instrumentation approach where performance monitoring instrumentation is injected -- using the Web server or load balancing appliance -- into the page as it is being sent to the user. This non-intrusive approach requires neither source code changes nor end user agent or applet downloads. Using a third-party data collector and console, developers and ops personnel can share real-time performance data on key URLs based on actual real users running real transactions. This allows developers to stay on top of performance issues before the help desk folks even receive a complaint.
DDJ: Is there a web site readers can go to for more information on these topics?
HW: Please visit www.symphoniq.com for more information. My blog can be found at www.symphoniq.com/true-resources/true-insights.php. I welcome your readers' comment.
