Build on the ESB Foundation
The ESB can be used from within an application server, a lightweight framework, and even a standalone application. However, in practice, the ESB itself is not an application server, a messaging server, or a single piece of software that is installed and run on a piece of hardware. Instead, it's a framework and a set of tools that you can choose from to achieve distributed application development, integration, and deployment.
Of course, you get the most value from an ESB when you take advantage of all of its capabilities, but it's not necessary. You can migrate an existing system, or set of systems, to the ESB slowly over time using a phased approach if circumstances require. In fact, this is exactly how many corporations begin with the ESB, as illustrated in the case studies explored later in this article. The most benefit you will see from an ESB will be from the SOA standards and software design patterns that it enables you to follow when building distributed software.
Overall, the foundation of the ESB includes enterprise messaging, SOA standards, message transformation, and message routing services, upon which you can efficiently build robust, distributed applications. Let's begin by looking at how enterprise messaging is a critical piece of the ESB.
Build a Communications Infrastructure
The enterprise service bus evolved from the growing need for enterprise messaging systems and the integration of SOA-based software components. With distributed software, computer-to-computer communication is a top issue; the quality of communication quickly becomes a key differentiator, and can determine the software system's success or failure. In the past 10 years or so, as companies rushed to make their applications and services available over the Internet, much of this infrastructure was custom developed. Over time, as organizations deployed different messaging infrastructures and different platforms and protocols for their SOA-based systems, it became clear that standards would be needed to drive down the cost of integration.
Moreover, these standards need to account for platform and language independence. The adoption of standards further eliminates costs by enabling organizations to leverage their existing software systems and developer knowledge. It also lets developers focus on building software functionality, and not infrastructure, thereby easing the pain of distributed software development.
Communication Standards. There are many choices for application component communication, from the simple method call, to the robust, yet heavy and complicated, Common Object Request Broker (CORBA) infrastructure. In between, developers can use low-level network sockets, older communication protocols such as the FTP, modern protocols such as HTTP, hub-and-spoke architectures, email systems, and other platform-specific, proprietary, communication protocols (see Figure 4).
Thanks in part to the growth of the Internet, and Web standards, three forms of messaging have become popular and standardized, at least in definition:
- Request-reply messaging. A software system sends a request for data to another software system, which in turn replies to the request.
- Topic-based messaging. Software components are developed to send messages to, or listen for messages from, abstract topics, such as "World News," "Stock Quotes," "Sports," and so on. This is often referred to as publish-and-subscribe messaging.
- Queue-based messaging. Software systems cooperate disjoint through an abstract entity (the queue) which stores and forwards messages as they are placed on and taken from the queue, respectively.
Topic-based messaging (shown in the top portion of Figure 5) is often referred to as "publish-and-subscribe" because components send messages to abstract topics, which other components subscribe to and receive.
Each subscriber to a topic receives every message published to that topic (a multicast), without regard to the software that publishes the messages, or the other components that have also subscribed. In this sense, communication is anonymous -- publishers and subscribers don't need to know of one another's existence. It's important to note that messages published to a topic for which no subscribers exist are generally discarded.
Queue-based messaging (see the lower portion of Figure 5), sometimes called store-and-forward, is quite different than topic-based messaging, and more closely mirrors that of an email system. Messages are placed on a queue and delivered to only one receiver (not multicast). If there are no queue listeners when a message is sent, that message is stored and delivered when a listener becomes available.
Request-and-reply messaging (see Figure 6) is best described as the dialog that occurs between a web browser and a web server.
A request message is directed to a specific server, and a message is sent to that client in response, with the data to satisfy the request. This form of messaging is straightforward and obvious, but has some shortcomings. First, senders and receivers need to know of each other's existence, which builds hard-coded dependencies between them. Second, scalability and reliability become an issue, especially if one component in the request/reply sequence fails. In practice, request-reply messaging is usually implemented through either topic-based or queue-based messaging, where the message itself supplies a means to reply to the message sender.
Reliable Messaging Explained. Regardless of the communication paradigm you choose (topic or queue), it's important that it be reliable. When a message is sent, you must be guaranteed that it is received in the same form it was sent, or the integrity of your distributed software system is at risk. Enterprise messaging systems provide message delivery guarantees, as well as transactional support, most often with two-phase commit. This provides the ESB, along with the applications built on top of them, a level of built-in reliability required for mission-critical application. For software components developed for an ESB, there is generally nothing special that needs to be done to tap into this service. It's provided transparently by the ESB and the enterprise messaging backbone it itself is built upon.
Asynchronous Event Processing. Enterprise messaging systems, and the ESB, provide the capability for software components to send, listen for, and process messages asynchronously. This helps to decouple components in a distributed software system, further enabling system reliability and scalability. For instance, with asynchronous message processing, a component that sends messages to a queue does not need to wait for each message to be processed before sending the next one (see Figure 7). In contrast, the sequential (synchronous) processing of messages can lead to a semi-unresponsive system that would appear to behave "choppy" to users. Asynchronous messaging lets the sender component continue processing in parallel as its sent messages are received by other components, or stored for future retrieval.
Of course there are also valid reasons to use synchronous message processing, such as with certain types of financial transactions, and with systems that require messages to be processed in a well-defined order. Because of this, the ESB provides both synchronous and asynchronous messaging support.
However, most distributed software systems benefit from the event-driven behavior asynchronous messaging provides. The obvious example is an email system: when you send an email message, you are free to send more email messages, even if the recipient has not yet read the first email you sent.
More importantly, the ESB brings the benefits of event-driven processing to the world of batch-processing systems, such as the many legacy applications used in banks and financial institutions across the globe. SOA alone may allow you to integrate a legacy system with a newer, web-based, application. However, with the features of an ESB, there are strategies you can employ to migrate a legacy system from batch processing, to semi real-time processing behavior, over time. In the modern world, where up-to-the-minute information is available on demand not only to institutional traders, but also to a bank's checking account customers, this feature has become a requirement. The ability to change a batch system to an event-driven system incrementally is another benefit of the ESB.
The Java Message System (JMS). Entire books have been written about JMS, message-oriented middleware (MOM) software, enterprise messaging, and enterprise application integration. Since the ESB helps to eliminate or at least hide many of the issues around messaging and integration, you really don't need to have more than a basic knowledge of JMS. However, it does help to have some exposure to JMS and its concepts, so let's go over it quickly now.
The Java Message Service (JMS) specification describes an enterprise messaging system for Java that supports both topic-based and queue-based messaging paradigms. JMS also specifies many types of message objects with the goal of interoperating with other, non-Java/JMS, messaging systems. With enterprise messaging -- and in particular, JMS -- you can build distributed software systems that are truly platform and language-independent.
In a nutshell, JMS is a specification [Hapner02] that describes the properties and behavior of an information pipe for Java software. It also describes how Java client applications interact with the information pipe. These are two very important distinctions that must be understood before implementing JMS software. They are named a JMS provider, and a JMS application, respectively:
- JMS Provider: This is an enterprise messaging implementation that adheres to the JMS specification for message definition, delivery, storage, processing, and error handling. It is the implementation of the message pipe itself, not the software that uses it. The specification suggests, but does not require, that JMS providers be written as 100% pure Java applications. The intent is to allow the provider to be as platform independent as the Java software that uses it
- JMS Application: Sometimes called a JMS client, this is the Java software that uses a JMS provider to send and receive messages over an information pipe. This software can be any application, small or large, that generates or processes messages. The JMS application generally has no knowledge of the implementation behind the information pipe itself (the JMS provider); it's developed using the JMS interfaces as defined in the JMS specification
The ESB is built directly on top of the JMS information pipe, and leverages everything that JMS has to offer. The remainder of the ESB functionality is layered on top of JMS in a way that its messaging features are abstracted nicely, but are still available for components to plug into directly, if there is a need to do so.
