THE SOFTWARE SIMPLIST

Web Services Wisdom.

July 2006

There been a flurry of activity around the whole SalesForce API thing (linked summary here) going on about "Contract first", XSD, and WSDL. Some of the key quotes:

Steve Loughran: "Me, I blame WSDL and XSD."
Sanjiva Weerawarana: "Middleware vendors who claim that taking some customer's old object oriented code and auto generating services is a good thing should be shot."
Arjen Poutsma: "I don’t like writing WSDLs. I think they force a wrong model. Unfortunately, they are the standard."

One of the things that I think important in software is separation of concerns. Separating the contract from the internals of the service (like its domain model) is paramount in that respect.

The service's contract is made up the logical message types (not necessarily XML) it "owns". There are certain kinds of messages that a service may send or receive but that it doesn't own. For instance, a service may receive messages published by a different service notifying it about important events. Also, a service will communicate with another service requesting it to perform some action by sending it a message. In both these cases, the message types being sent are not part of the first service's contract.

So, why does WSDL dictate that services need to expose operations in order to receive messages they don't own? These message types are not owned by the service and, as such, will have different versioning behaviors than message types owned by the service. So why on earth should they be bundled in the same WSDL file? Even if they're in separate WSDL files, and on separate endpoints, the semantics of these "operations" are totally different. I wouldn't want just anyone to be able to send a message to those endpoints - only the service to which I've subscribed should be sending me those messages.

But there's policy to manage that, right? Unless, of course, the service decides to switch the location where it sends messages from - which is its perrogative. But then we can write custom policies to handle that, right? Unless we didn't know that the service changed its sending policy - which we shouldn't know, autonomy and all that.

But, but, but...

WSDL hurts. Its a crappy standard. But, like Arjen said, "they are the standard." The thing is, WSDL doesn't add anything of value beyond basic XML once you set out to build loosely coupled systems. Loose coupling is, first and foremost, an architectural goal. Its a pity that an implementation detail like WSDL gets in the way. There are alternatives, like REST, but they have their faults too. The point being, if you're XML based, you'll be interoperable. The tools may not necessarily work well with it, but after the first 90% of the work they do automagically for you, you still have the other 90% to do by hand anyway. So doing 100% by hand is really only a 10% increase :)

Posted by Udi Dahan at 03:06 PM  Permalink |

The other day I got an email from a guy named Bill who was having some trouble understanding the state of the guidance around SOA coming from the different vendors. I thought that the back and forth I had with Bill would be valuable to others so I decided to relate it here.

Hi Udi,

WCF defines a service to be implemented as single .NET class which implements one or more interfaces (contracts). These contracts are then exposed as one or more endpoints.

It’s a technological view, but I don’t like it. A class, or its corresponding object, do not a service make.

Other definitions seem to indicate that a service exposes the functionality of a whole application. You indicated in your section in Jimmy Nilsson’s book that it would constitute one EXE or one Web application root. This would imply multiple ASMX or SVC (in WCF) files per service. This seems to be contradictory to the WCF definition of a service.

I have no problem contradicting Microsoft, considering the fact that they so often contradict themselves.

Moreover, general consensus is that in order to maximise autonomy between services, they should not share a common database. If we are to always implement an entire service using a single .NET class, then this would imply one database for every service component.

Agreed on the autonomy. A DB per class is obviously wrong.

Microsoft certainly does not make it easy to share data types between different “services” – which would make sense if we are to implement a separate database for each service component (since they operate on different data domains); but doesn’t make sense if we are to have multiple service components at different addresses sharing the same database as part of the same logical “service”.

All these service classes should be working against the same domain model. There should be very little logic in the service classes themselves – translate the message/call to calls on the domain model (within a transaction as necessary), then build the response and send it. You shouldn’t need much interaction among the service classes.

So does this mean that a “service” has a dual meaning – one is an individual service component and the other is a collection of service components constituting a single unit of deployment? If this is the case, then I think the industry needs to work towards clarifying this definition and perhaps describing the two concepts with different terms. For example, the 4 tenets of service orientation wouldn’t apply between individual service components as part of the same service – but reading the MS technical articles seems to imply that they do (since the word “service” is applied universally).

No dual meaning. The guidance that I’m putting out calls these kinds of classes “message handlers”. No room for confusion there. You could easily have multiple message handlers in the same service. A service could also, therefore, have numerous endpoints and support various transports. Microsoft has to do a better job in the guidance its putting out.

I thank you for your time and hope that you might be able to help clarify some of these loosely defined concepts in the community.

You bet. I hope that this makes better sense now. Don’t get too hung up on what Microsoft says, do what you think is right, and getting a second opinion is a great way to do that. Let me know how it all goes.

--

Many thanks to Bill for his great question. If you too have a question regarding SOA or Web Services, don't hesitate to fire off an email to Blog@UdiDahan.com, and you can always check out my site for more in depth information: www.UdiDahan.com.

Posted by Udi Dahan at 04:51 PM  Permalink |

Get it here.

This week we brave the flames of the ongoing REST vs. Web Services debate. REST, or REpresentational State Transfer, is an architectural style founded on the simplicity of HTTP, while Web Services are, in essence, a set of specifications supported by several vendors in various phases of standardization. Join us for an examination of the benefits REST-ful thinking can bring to service oriented architectures.

Resources:

Wikipedia definition of REST
Ask Udi a question on SOA
Subscribe to the Ask Udi podcast
Visit Udi's blog for more information.

Enjoy.

Posted by Udi Dahan at 04:14 PM  Permalink |

Rohan Thomas, a fellow MVP, has written up some (a lot) of notes on my podcast "Does an SOA require an ESB" - it's actually more of a transcript. If you didn't have the time to listen in, this may be just what you're looking for.

Posted by Udi Dahan at 04:01 PM  Permalink |

In the comments on my previous blog post, Reliable Messaging, Transactions, and Messages – oh my!, my fellow DDJ blogger - Eric Bruno made a number of points that I think are pertinent, and to which I wanted to respond (originally posted here).

"I would go even further and say that all messages need to be reliable. Even a missed stock update can result in lost money if the update brought the stock value above or below a threshold for a buy/sell order. I cannot think of any application I've written where a message was sent that I said, "it doesn't really matter if it gets there." Because if that were the case, why send it?"

I wouldn't go that far. The point is not whether or not it matters if the message gets to its destination. The point is that different kinds of data have different volatilities - the amount of time the data is valid for is different for a command sent by a user and for a stock update that is published 6 times a minute. While we may not want to ever lose the user's command (buy, sell, or whatever), if the network isn't able to transfer a stock update for over 10 seconds, it might as well throw it away since the new update that was just published is now that valid one. Again, we're talking about failure cases here - not the common case.

So, for each message type we can/need to decide how volatile the data is that the given message type will carry, and based on that define the recoverability interval - 10 seconds in the above example. If we were to define this in code, we might use an attribute that looked like this:

[Recoverable(Timeout=TimeSpan.FromSeconds(10))]

On to transactions. Bruno states:

"Breaking it down further, at the basic level, one could say that are two types of messages: transacted or not."

I agree, but I'd say that the transactional behavior is already defined by the recoverability description. A recoverable message should be handled within a transaction, as I said in the previous post - a transaction that wraps the pulling of the message from the queue, any database changes, and the sending of other messages. Non-recoverable messages don't need this level of transactional behavior. Although they may well involve transactional business logic against the database, as in Bruno's example, sell when the stock reaches a certain threshold, the failure cases that we deal with put a new spin on it.

If the server that is currently handling the message restarts for some reason, what should happen? If you take into account that a restart can take upwards of 15 seconds, if you go to handle that same message again, you may well be getting yourself into trouble since "the rest of the world" has already moved on. You may not want to sell if you find out that after the restart the previous stock price is no longer true.

So, at the service level, all you need to deal with is recoverability - it'll define service-level transactional behavior. Internal transactions are orthogonal and deal more in the actual data in the message (the stock quote itself) rather than the message type.

Posted by Udi Dahan at 03:40 PM  Permalink |

Following up on my recent posts on SOA Testing and on the structure of services, I wanted to finish up the issue of unit testing in SOA. Since the main difference between a "regular" application and a "service" is that a service communicates only via messages/messaging, and by using a bus to keep our message handlers technology independent, if we keep them isolated from our Domain Model, then the only difference between unit testing a regular application and a service is unit testing the message handlers.

So, how do we unit test a message handler?

Well, by having an IMessageHandler interface that exposes a single "void Handle(T message);" method, all message handler classes will need to be tested via that interface. Note that a single class may implement this interface for numerous message types - this is especially important in workflow scenarios, but that's a topic for a whole 'nother post.

In order to unit test any class, the most important thing to do is understand what other classes it interacts with. In the case of a message handler, it would most commonly interact with 3 kinds of classes:

1. Domain Model classes that contain the logic
2. Persistence classes to persist changes that occur in the domain model
3. The bus to communicate changes back to the world

By keeping our Domain Model independent of cross-cutting concerns like persistence, we can then use these same classes for unit testing. The persistence and bus classes, on the other hand, need to be stubbed, or mocked out, as do any other classes that may adversely affect the unit under test.

The kinds of unit tests that are written for message handlers often look something like this:

1. Mock the bus and persistence classes and pass those objects to the unit under test (the message handler)
2. Set expectations for behavior that will occur as a result of handling the message
3. Create a message and call the "Handle" method of the handler
4. Verify that the expectations from (2) were met.

These expectations are often in the form of:
* When the "Get" method of the persistence class was called, have it return some data that we can check against later
* The "Reply" method of the bus was called, and the message passed to it of the correct type and has the correct data - checked against what was prepared by the persistence stubs

As you can see, unit testing a message handler isn't that much different than unit testing any other class. And that's the bottom line. Correctly designing your services will lead you to easy to test code, making unit testing a service just like unit testing any application.

Posted by Udi Dahan at 09:19 AM  Permalink |

After doing a bunch of deeper, technical posts, I wanted to zoom out a bit and talk about higher level concerns. I think I’ll start out by discussing “service partitioning”. A couple of days ago, after finishing a presentation on SOA and workflow, one guy came up to me and asked me some questions about a system he was working on. I don’t remember exactly what those questions were, but I answered something to the effect of “well, that means you’ve divided up your services wrong”. “Is there any example I can look at that shows a correct division of services”, he asked. I was sincerely surprised that I couldn’t think of a single one.

I’ve noticed this same problem popping up in discussions about SOA online. For instance, in the service-oriented-architecture Yahoo group there’s a heated discussion about REST’s application to SOA in the context of… a light bulb. Yes, it’s true. By analyzing the behavior of a light bulb, we will be able to reach valid conclusions about enterprise systems – won’t we?

So, I began thinking what example I could give that would be complex enough to demonstrate the value of SOA, yet familiar enough to be easily grasped by most practitioners. After expanding the scope from light bulb, to room, to building, eventually what I came up with was a movie theater – something that would warrant software to run it. I think that we can come up with some basic requirements for an automated movie theater that would make sense.

First of all, a movie theater has a number of screening rooms, each room with light bulbs that need to be dimmed and lit, a projector for showing the movie, curtains to show and hide the screen, etc. We’d want the ability to plan which movie would be shown in which screening room. When the time would come for a movie to show in a specific screening room, the lights should dim, the curtains open, the projector activated. There would also have to be some maintenance done on these rooms – cleaning after a movie’s over, and fixing equipment that breaks down.

If we were to perform a service-oriented analysis of this problem, we might come up with the following services:

A Movie Assignment Service, where someone would plan which movie would show in which room and at what times. They would probably take into account if its opening night and the popularity of the movie. The user would also want to create several alternatives for the same date, several days in advance. At the beginning of each day, a (possibly different) user would choose which plan to execute. This plan would be published by the service as the plan to be executed.

Each screening room would have an instance of the Screening Room Service to run it. This service would subscribe to the plan to be executed event published by the Movie Assignment Service. It would also control the lights and the projector, curtains, etc for the specific room. When the time would come for it to screen a movie, it would start dimming the lights, open the curtain, start the projector, etc and publish that its state is “now screening” (you wouldn’t want a janitor to come sweep the floors in the middle). It might also publish the status of each light bulb, but that seems kind of silly. What makes more sense is for it to publish if it has any burnt out light bulbs or other faulty equipment.

Finally, the Maintenance Service would subscribe to the above events and page janitors and service personnel as needed to handle cleaning and repairs of the screening rooms in such a way as not to interfere with the movie goers.

Obviously we could expand this to include billing and other interesting topics, but I think that this is enough to get a feel for good service partitioning.

Possibly the most important difference between service-oriented architectures and the still popular layered architectures is the primary level of decomposition. We don’t automatically separate out the presentation layer and go from there. Rather, we try to gain an understanding of the business processes that take place in the domain, and divide the system up by ownership of those processes. There is absolutely nothing wrong with using layered architectures within a service. It’s just that layers are the wrong top-level abstraction for complex, distributed systems.

One last thought: notice how all the technical concepts we’ve discussed so far fit into the above services. Pub/Sub communication between services; Transaction boundaries that don’t cross service boundaries; Workflow kept inside services; and so on.

Forget about the holy grail of IT/Business alignment. Use SOA to build higher-quality systems today.

Posted by Udi Dahan at 09:16 AM  Permalink |

Here's Ask Udi #4, where we drill down into the layers that make up a single service, connect the concepts of ESB and Domain Driven Design, and enumerate the various kinds of workflows that can be found within, and between services. (MP3, 20:03 mins)

The podcast is online here. Below are some useful references:

1. Paul Gielens – Applying the application layer in Domain Driven Design
   
2. Reliable Messaging, Transactions, and Messages – oh my!
   

To ask Udi a question on SOA mailto:podcast@UdiDahan.com

Subscribe to the Ask Udi podcast (or http://tinyurl.com/e52tc)

Posted by Udi Dahan at 04:06 PM  Permalink |

I'm proud to anounce that shortly after the third installment of the “Ask Udi” podcast came online, we hit our one thousand subscriber mark! Just so you know, number four is on its way up, and number five is currently being edited.

Podcast number 4 is about Domain Driven Design (DDD), layered architectures, the different kinds of workflow, and of course SOA. This podcast was triggered by Paul Gielens blog post on “Applying the application layer in Domain Driven Design” where he discussed the combination DDD and workflow of various kinds. The reason I found this topic pertinent to SOA has to do with the total lack of discussion around how services should be structured.

I mean, everybody’s going on and on about SOA this, ESB that, EDA over here, and 2.0 over there, but nobody’s talking about the details. Probably all the devils down there scared them off.

So, in podcast 4 we get into the different kinds of layers described in DDD – specifically the Application Layer, and discuss what exactly goes into this layer when developing a service and not an application. Also, we mention the different kinds of workflow (business rules, long running processes, and orchestration/choreography) and what layers they belong to.

And in podcast number 5 we get right in the middle of one of the most heated debates around SOA: REST vs. Web Services, so stay tuned!

If you haven’t yet subscribed, what are you waiting for?! Get your weekly SOA fix delivered right to your desktop here.

Also posted here.

Posted by Udi Dahan at 04:56 AM  Permalink |