IF YOU BUILD IT

... Will they Come?

February 2007

We’ve nearing the end of the second iteration of the main project I am running this year (a multi-modal biometrics backend platform), and we're scheduled to do an architectural review (and demonstrate a working architectural skeleton).

So here I am again, preparing an architecture presentation. The first thing I do when I have to describe architecture is choose which views to present. This time I decided to build the views around the key quality attributes (those that were highlighted in the system’s utility tree).

So what’s the structure of this presentation?

The presentation begins with a system overview which includes the vision and key scenarios for the system.

The next view is an architecture overview which has a block diagram, major components (and sub-systems) and the major decisions (e.g. the leading architectural style for the system is SOA etc.)

Next I have a functional view where I’ll present how some flows go through the architecture. This can either use communication diagrams or maybe I'll present them on top of the block diagram.

The next views are performance and scalability, security and privacy, flexibility, adaptability and extensibility, compliance and standard, testability and maintainability. Each of these views has a main presentation and a set of decisions. For each decision I present the alternatives and why the decision was made

Lastly I am going to show some of the design highlights -- these are major design construct for each component that show how the architectural decisions have been translated into design

This structure is a little different from previous presentations I used. One view I usually have is what I call the "module blueprint" which shows how the design and call sequence for a nominal flow would look like when it is built in the system. However, for this project this is too early in the game for that we have a few flows but it as we will implement more we will finalize the design details. Also we plan to produce a software factory and more guidance as part of the release of the platform, since the platform itself is not an end product rather it is an infrastructure for building solutions.

Another view that is missing this time is deployment view -- this is included as part of the performance and scalability viewpoint. Again the main reason for this difference is that the system is a platform and not an end-product.

It is important to select appropriate views for the system you build. Granted there are some views that will be more common than others, but in the end each system is unique.

Posted by Arnon Rotem-Gal-Oz at 11:47 AM  Permalink |

There’s a thread on the Microsoft’s Architecture Forum where someone asked what is the most valuable principle in design?

There are many good candidates -- loose coupling, abstractions, flexibility, KISS, YAGNI, separation of concerns, to name a few. However, in my opinion the most important principle is to get a practical solution.

"Practical" means a solution that take into consideration the (limited) time, budget, resources, capabilities of the team members, and so on. It is about getting the balance right. To do that you have to be pragmatic. While some solution might be the most perfect technical solution ever, it is just as worthless as a bad design if it can’t fit within the project’s other (non-technical) constraints.

It is my experience that unless you are dealing with simple projects you can’t have it all -- you can't get all the quality attributes in one system, so you have to prioritize. The challenge of getting this prioritization right is probably the number one reason the most important trait for architects is experience. You have to see the consequences of decisions in the long term in order to be able to understand the implications of what need to be done. Experience also lets you grow the needed pragmatism and to understand that the "best solution" is a relative term.

Posted by Arnon Rotem-Gal-Oz at 06:22 AM  Permalink |

Rich Internet Applications (RIAs) are becoming more popular. What slight improvement of usability by getting data in the background (AJAX) is moving onward to making Web UIs more complex and advanced.

Advanced or Rich Web UI brings a whole horde of challenges. For example, if you work with AJAX you make a lot of (small) round trips for the server which on a large application can cause network and latency problems. So we get COMET -- but again that is already old news since we already solved this problem in client/server ages ago.

I guess the root cause for going with Web technologies was really a deployment problem -- but what we (that is, the software industry) did was throw out the baby with the bathwater. But not all is lost. We are starting to see Rich clients in disguise with technologies like Adobe Flex 2 (also see Flex 2 and Rich Internet Apps) and WPF/E (or WPF Interoperability), which are dubbed "Web technologies" because they run in the browser but download to the client when accessed.

Posted by Arnon Rotem-Gal-Oz at 08:30 AM  Permalink |

Udi has some comments about my SOA definition. Udi says that the definition I provided does not support the notion of publish/subscribe using topics for services. My answer to this is yes and no.

First of all, I never said (or at least I never meant to say) that contracts are only limited to incoming messages. Contracts contain incoming and outgoing messages. I probably should have stated it more clearly though. Udi says "Contract: Who owns the message type being published? The publisher or the subscriber? Common SOA knowledge would say that the message belongs to the contract of the service that receives it."

I don’t know who "Common SOA knowledge" is. In my opinion, this thinking is a wrong "even" for request/reply. The reply message belongs to the service the sends the reply.

Regarding endpoints. If the subscribers go to a topic as in "ServiceName\TopicName" then yes I would call that an endpoint since this is a well-known address consumers (subscribers) go to find messages published by a service.

Regarding consumers Udi says:

Is the publishing service "using" the subscriber when it publishes a message? I don’t think so, and the subscriber definitely isn’t using the publisher at that point either. So, we’ve got some inter-service message-based communication going on and it isn’t clear if we even have a service consumer. In fact, if all a service ever did was subscribe to some topics, and publish messages on other topics, it looks like we’d have very loose-coupling but be straying from the common SOA wisdom.

Maybe that’s just semantics but I don’t see why the subscriber isn’t using the publisher.The publisher publishes a message on a topic this is part of its offering. The subscriber chooses to consume that information and maybe do some stuff with that -- possibly publishing some other messages. That’s a "using" relationship to me.

Nevertheless, SOA is not a synonym for "Distributed system" so there are cases when distributed components that communicate through messages aren’t SOA. For example, publish/subscribe using topics where the topics are common and shared between components so that multiple services can publish on the same topic does not, in my opinion, fall under the definition of SOA . This doesn’t say that this is a bad architecture in any way -- but it isn’t SOA either.

As I said in the "What is SOA posts" for an architecture to be SOA you need autonomous components , that publish and accepts messages defined in contracts, delivered at an endpoint and governed by policies to service consumers -- no more, but no less either.

Posted by Arnon Rotem-Gal-Oz at 03:26 PM  Permalink |

Looking at the SOA definition in Part IV, you can see it puts a lot of emphasis on interface.

Starting from the messages which are the parts of the interface, the contract which is the collection of the messages, the endpoint where the contract is delivered and the policy which governs the behavior of the endpoint. Thus SOA has a total of four different components that deal with the interface versus, for example, OO which only has one. The focus on interfaces is what gives SOA the ability to create loose coupling, composable components, reuse and achieve the various design goals. Another nice attribute of this definition is that we can use as a base for both the technical and the business perspectives of SOA as the common elements of both perspective are used in this definition.

Thus, even though there are a lot of misconceptions and hype surrounds SOA, there is value in the term. Multiple definitions do not have to translate to ambiguity if they are just different workings for the same concepts. We do however have to be careful not to be fooled by the hype and misconceptions. I hope that the definition provided here helps achieve this goal.

You can also download all the five parts as a single document from my site (670K PDF).

Posted by Arnon Rotem-Gal-Oz at 04:46 AM  Permalink |

To be able to converge between the technical and business viewpoints, we first need to differentiate between an architectural style and its application. Once we define what SOA is, we can apply it at an organization level to get an SOA initiative where services will encapsulate business function. However, we can also apply SOA on a single project and get services whose content revolves around technical issues, such as security or management.

We also need to differentiate between design goals such as loose coupling or business alignment and architectural building blocks and constraints like coarse-grained services or policy-based interactions.

Lastly, if we look at definitions of other architectural styles like Client/Server, Layered, or REST we can see that we can see that architectural styles are defined in terms of components, their attributes, their relations and the rules or constraints that govern them.

Based on that we can define Service-Oriented Architecture as an architectural style for building systems based on interacting coarse grained autonomous components called services. Each service expose processes and behavior through contracts, which are composed of messages at discoverable addresses called endpoints. Service's behavior is governed by policies which are set externally to the service itself. you can see the SOA componets and their relations in this diagram.

Let's take a look at each of these components (excerpted from my upcoming SOA Patterns book).

Service. The central pillar of SOA is the service. Merriam Webster defines service as "a facility supplying some public demand". A Service should provide a high cohesion and distinct function. Services should be coarse grained pieces of logic. A Service should implement at least all the functionality promised by the contracts it exposes. One of the characteristics of services is service autonomy. Autonomy means the services should be self-sufficient, at least to some extent, and manifest self healing properties.

Contract. The collection of all the messages supported by the Service is collectively known as the service's contract. The contract can be unilateral, meaning a closed set of messages the service chooses to provide. A contract might also be multilateral or bilateral, that is, between a predefined group of parties. The contract can be considered the interface of the Service akin to interfaces of object in object-oriented languages.

Endpoint. The Endpoint is an address, a URI, a specific place where the service can be found and consumed. A specific contract can be exposed at a specific endpoint.

Messages. The unit of communication in SOA is the message. Messages can come in different forms and shapes; for instance, http GET messages (part of the REST style), SOAP messages, JMS messages,and even SMTP messages are all valid message forms. The differentiator between a message and other forms of communication such as plain RPC, is that messages have both a header and a body. The header is usually more generic and can be understood by infrastructure and framework components without understanding, and consequently coupling to, every message type. The existence of the header allows for infrastructure components to route reply messages (e.g. correlated messages pattern) or handle security better (see Firewall pattern).

Policy. One important differentiator between Object Orientation or Component Orientation and SOA is the existence of policy. If an interface or contract in SOA lingo, separates specification from implementation. Policy separates dynamic specification from static/semantic specification. Policy represents the conditions for the semantic specification availability for service consumers. The unique aspects of policy are that it can be updated in run-time and that it is externalized from the business logic. The Policy specify dynamic properties like security (encryption, authentication, Id etc.) , auditing, SLA etc.

Service Consumer. A service doesn't mean much if there isn't someone/something in the world that uses it. So to complete the SOA picture we need Service Consumers. A service consumer is any software that interacts with a service by exchanging messages with the service. Consumers can be either client applications or other "neighboring" services their only requirement is that they bind to an SOA contract.

Posted by Arnon Rotem-Gal-Oz at 03:07 AM  Permalink |

Looking beyond the hype and misconceptions, it seems we can narrow SOA into two points of view, both which seem to be valid -- one from the business perspective and another from the technical one.

At the enterprise architecture level, it is always about the business. This is not a bad thing. On the contrary, the enterprise architecture perspective should be focused on the business needs to make sure IT serves the business, and not vice versa.

The emphasis from the business perspective is on "service orientation". Consider, for example, the SOA definition from Service-Oriented Architecture (SOA): A Planning and Implementation Guide for Business and Technology, by Eric A. Marks and Michael Bell.

SOA is a conceptual business architecture where business functionality, or application logic, is made available to SOA users, or consumers, as shared, reusable services on an IT network. "Services" in an SOA are modules of business or application functionality with exposed interfaces, and are invoked by messages.

Looking at other "business oriented" definitions of SOA, we can see they follow the same reasoning. In a nutshell, they can be summarized as follows:

From the business point of view, SOA is about analyzing the business to identify business areas and business processes, followed by defining services to represent these "areas". Services expose their capabilites through message interfaces. The services can then be choreographed or orchestrated to realize the business processes. The goal of SOA is to increase the alignment between business and IT and achieve business agility -- the ability to respond to changes quickly and efficiency.

And then there's the other face of SOA -- the technical point of view.

On the business side of the fence, the emphasis is on "Service Orientation" (SO). On the technical front, the emphasis is on the "A" of SOA -- Architecture. True, there isn't a single unified definition for SOA; however, just like the many definitions of software architecture, there are several characteristics that are more common and frequent than others. Looking at definitions of SOA such as those from Wikipedia, O'Reilly , JavaWorld, and Microsoft, etc., you can see that SOA is commonly thought of as an architecture or an architecture style that builds on loosely coupled, interoperable, and compostable components (or software agents) called "services." Services have well-defined interfaces based standard protocols (usually web services but most definitions mention that it is not the only possible implementation) as well as QoS attributes (or policies) on how these interfaces can be used by Service Consumers. SOA definitions mention the basic communication pattern for SOA is request/reply but many definitions also talk about asynchronous communications as well.

If we look at the business and technical approaches for SOA we can see that there's still hope to achieve convergence as there are some common grounds -- the next section will try to do just that.

Posted by Arnon Rotem-Gal-Oz at 07:07 PM  Permalink |