Message Transformation
As mentioned in the previous section, merely sending a message is not always enough. The components that need to receive that data will require it to be in a form that they can process. Many services will accept messages in the form of XML, but they may need that XML to conform to a particular XML schema. As more receiver components are plugged into an ESB-based deployment, the more potential data transformations will need to take place. Developing custom code to transform the same message to conform to a different XML schema for each client is inefficient and costly. Worse, there will be many components that send their data in JMS messages. In these cases, the data in the message will need to be converted to XML in general-never mind subsequent XML transformations that will need to occur-for XML-based receivers to process them.
Fortunately, the ESB provides built-in message transformation services to take care of this for you. In many cases, the ESB will normalize all message flow within its boundaries to be XML-compliant, regardless of the form the message data started in. This allows each listener component plugged into the ESB to specify a format for which it requires the data. In most cases, this format will be described with an XML Schema. In other cases, a component may request all message data be delivered as a JMS message. Either way, the ESB will perform the transformation for the listening component, without any specialized code required on the part of the sender or the receiver.
The net effect is that the receiver's view of the data is decoupled from the sender's view, the overall system's view, or that of any other receiver in the system. This message normalization and decoupling is the feature that begins to set the ESB apart from just another message-oriented middleware (MOM) product. Whereas typical MOM software, such as a JMS provider, will treat most messages as opaque, and not get involved in what they contain, the ESB gets intimate with the data.
Message normalization and transformation is a feature that will also save you a lot of development, testing, and maintenance effort. Because the ESB decouples both the message sender and receiver from the message format (and hence one another), the development of either component will not impact the other. Normalizing message formats doesn't just benefit message sender and receiver component developers; it enables the ESB to work with the messages; such as to route then effectively across the network and to the systems that require them. Message routing is the next part of the ESB foundation that we need to look at.
Message Routing
With the ESB, applications, services, and components do not interact directly. The ESB serves as an intermediary for communication and the routing of messages between components. This is an important advance, especially compared to how message exchanges work in the classic cases of web services or JMS. In both of those cases, message senders and receivers would have to find one another, and invoke each others services. (Perhaps this applies more to message receivers, but since many components are both senders and receivers, we'll look at it from both perspectives.)
The problem with this approach is that every component has some sense of awareness of the other. In most cases, this awareness is hard-coded within the component in a way that makes it difficult and expensive to change and redeploy. The system-wide dependencies that this model creates can quickly bring a system to a halt and put system administrators into a frenzy when parts of the distributed application are changed, even in minor ways. This is an obvious waste of time and money for system maintenance, as well as development, as developers must spend time implementing service lookups instead of implementing customer features.
The ESB separates service lookups from the components that require the services. Hence, developers spend less time implementing this code for every component in the system. This ESB feature is called message routing, and it has some additional benefits aside from the obvious savings of time and money. It allows you to define how messages and events can be chained together, resulting in abstract message flows (also know as workflows) that components can plug into at runtime dynamically -- no hard-coding required. We'll take a quick look at what a message flow is below.
Abstract Service Endpoints. The first part of message routing and workflow definition is to define an abstract service endpoint. A service endpoint is defined as a discrete operation [Chappell04] devoid of its implementation. A software architect or ESB administrator can choose a component to fulfill the role of the service endpoint. This component may be a web service, a Java EE application, or an adapter to a third-party application of unknown implementation. So long as it conforms to the service endpoint definition, the servicing component can be selected at runtime.
Service endpoints are abstract in the sense that you do not need to worry about how they're implemented, where they reside, or how they're plugged into the ESB. You simply know they exist based on their service definition, usually in the form of Web-Service Definition Language (WSDL), which is an XML-based standard for defining business logic operations in a SOA.
The definition of a message flow begins by chaining together various service endpoints (and hence the components that implement those services), applying any required message transformations, along with any rules for the overall workflow. The rules of the workflow can be defined in terms of the content within the messages, or they can be tied to a generalized workflow, also known as itinerary-based routing.
Content-based Routing. The act of routing messages based on content is generally implemented as script components deployed within the ESB. The script can be code in the form of JavaScript, or it can be in the form of Business Process Execution Language (BPEL) rules. The routing can be based on message validation to ensure that applicable messages contain the data they're supposed to, or that they conform to an expected message format.
The routing can also be based on more complex rules, such as decision points that are made based on the presence of certain fields or field values within a message. For instance, the presence of favorable data within a credit field of a message may lead to a trade between two counterparties in a trading system. In many cases, content-based routing may be all you need to properly define a workflow.
In reality, this may not be enough. The contents of a single message may not be complete enough to make all of the decisions required within a business process. There may be a chain of events that need to occur, and applied to one another, to complete a workflow. This is where itinerary-based routing comes into play.
Itinerary-based Routing. Itinerary-based message routing is based on pre-defined workflows, or step-by-step processing. This type of routing is similar to a Java Servlet filter chain, where the response from one Servlet becomes the input to the next Servlet in the chain, and so on. Whereas the Servlet chain is defined in a Java EE configuration file, the itinerary-based routing workflow is defined with routing rules in the ESB. The sequence diagram in Figure 12 illustrates an involved, but realistic, business process that occurs during a foreign exchange trade.
First, a trader places a sell order for a particular currency pair (the two currencies to be traded). Some time later, another trader places a buy order for the same currency pair. According to the workflow, the Market Manager component is notified of all order entry requests. When it gets notified of both the buy and sell orders for the same currency pair, it checks for a potential trade. In this case, a trade does occur, and the Market Manager sends a trade notification message which is routed to the counterparties involved, including the Ticket Generator component. Finally, the Ticket Generator component generates a trade ticket message, which is also routed to the counterparties involved in the trade.
It's precisely the set of events, and the proper event ordering, that is described through message routing rules in an itinerary-based message workflow. Without hard-coding the workflow within the components themselves, the process can be modified by redefining the rules on-the-fly, without the need to change any code. For example, let's say we need to change this workflow to include the content-based routing rules for a credit check, as discussed in the previous section. A credit manager component can be deployed in the system, and the routing rules modified to include a credit check before a trade notification is sent. If the credit checks out, based on the data within the existing messages, the workflow will continue and a trade will occur. If the credit check does not pass, the workflow will stop and the trade will not occur.
This type of routing, and rules that describe it, is the beginning of a service orchestration model that the ESB enables. With service orchestration, seemingly unrelated sets of services and even entire applications can be choreographed to work together to define new business processes, and solve new problems.
Orchestration and Business Process Management
BPEL is an XML-based language specifically designed to describe and orchestrate asynchronous business processes. A business process is really nothing more than a piece of business logic (code) that performs some data-related task, but which is meant to be part of a larger set of processing. In other words, a business process is a relatively small data-crunching task that is meant to be chained with other business processes to perform an overall larger task. These tasks (their methods, inputs, and outputs) can be combined in meaningful ways with BPEL. Plugging SOAP-driven business process components into the ESB and controlling them with BPEL allows you to build reliable, distributed, systems with central orchestration and monitoring capabilities, as in Figure 13. Further, the ESB lets you dynamically change the orchestration of these business processes dynamically, while the system is running, without changing code or deploying new software.
The distributed, asynchronous, nature of the ESB, combined with its support for SOAP and BPEL, provides a standard method for web service integration that is consistent and straightforward to implement and deploy. This powerful ESB feature allows you to create virtual edge servers, which are those that combine the data and operations from other services in order to make the resulting combination available to other applications. Best of all, the ESB lets you do this in a standard way.
