Bridging the SOA/BI Gap

In the previous post I talked about the misalignment between SOA and BI. I briefly explored three options to incorporate BI and SOA without breaking SOA boundaries, then added that they all fall short. The aim of this post is to look at the fourth alternative which, as you probably guessed by now, is my favorite approach.

In my opinion, the best way to tackle BI in SOA is to add publication messages into the contract. By "publication messages", I mean that the service will publish its state either in a periodic manner or per event to anyone who listening. To make this complete, you can add additional requests/reply or request/reaction messages to retrieve initial snapshots. Following this approach, you get an event stream of the changes within the service in a manner that is not specific for the BI. Having other services react on the event stream can increase the overall loose coupling in the system, but that's another story.

Why is this better than the other approaches? For one thing, you can get a good picture of what happens within the service. However the contract is not specific for the BI and can be used by other services to cache the service state (thus increasing their own autonomy), for reporting (you can see an early draft of the aggregated reporting pattern), and for BI purposes. By working against a steady stream of events, the BI platforms can analyse treands, keep history and get the complete picture they need.

The approach above is sometimes referred to as "Event Driven Architecture" (EDA) and while I (and others) see EDA as another facet of SOA, not everyone agrees. Gartner, for instance, sees EDA as another paradigm and SOA just for request/reply, or client/server. That said, Gartner recently published a paper that calls this "Advanced SOA". I tend to agree more with the "advanced SOA" definition and don't see a contradiction with EDA and the SOA definitions. We are still using the same components and the same relations only adding an additional message exchange pattern into our toolbox. (I guess it might be good to post my definition of SOA. Maybe I'll do that soon.)

A note on implementation: If you are implementing SOA over an ESB that is rather easy to implement, (most ESBs support this out of the box) using WS* stack, you have WS-BaseNotification, WS-BrokeredNotification and WS-Topic set of standards. When using REST you need to implement publish/subscribe by yourself. Once you generate the event streams on the BI side you may want to look at CEP (complex event processing) tools to analyze and react on the events.

Posted by Arnon Rotem-Gal-Oz at 11:07 PM  Permalink