Some More Thoughts on Cross-Service Transactions

I received a few interesting comments following my post "Transactions Between Services? No, No, No! ". So here are a few clarifications.

The post was triggered by an article that promoted flowing transactions (i.e., you perform a transaction against one or two services and then one of the services calls an additional service and it joins the transaction). It is important to say that I think transactions between services should be discouraged regardless of automating extension of transactions. Transaction flowing just makes the matters worse.

There might still be some edge case where you have to have an atomic transaction from a service consumer to the service. I think that in the vast majority of SOA implementations you shouldn't do that and I would think real hard about the other options before allowing it in my architecture.

One of the comments I received began with the following:

"Cross-service transactions are a sure way to introduce coupling and performance problems into your SOA." I'm not sure I agree with that thought. Logically speaking, cross service transactions are a must. The question is how to implement them. There are two mechanisms we can use for implementing TXs: (1) ACID TXs; (2) Long-running TXs. The latter is preferable for the cases Arnon is talking about (large geographical distances, multiple trust authorities, and distinct execution environments). ACID TXs are more suitable for what Guy has mentioned (DeleteCustomer service invokes the DeleteCustomerOrder service internally). I agree with Arnon the a-synchronicity is preferable, but we all have encountered use-cases where ACID-ness is required from a business requirement level... [snipped]

One minor point in regard to this comment is that I don't like the term "long running transaction". There is a long running interactions between services and I think the term "SAGA" describes them better. SAGAs are made of a series of business activities that flow back and forth between services to realize a larger business process. Note that these interactions doesn't necessarily have transaction-like behavior.

Which brings me to the more important point of looking at the statement "Logically speaking, cross service transactions are a must." I don't think so. Consider, for instance, if a service that manages the inventory in a warehouse receives a request for some items and later a cancelation of that request. The first request can trigger the inventory service to order some more items from a supplier. Whether the cancellation would cause a cancellation of the order of the supplier depends on the business rules of the inventory service for inventory levels for the items ordered. it might also depend on whether or not the items have already been received etc. The cancellation (the "abort") of the original request does not have to translate to an abort (or compensation) on the request receiver. Furthermore if the service communications model is based on the push model (e.g., using EDA with SOA) the cancellation notice would just be propagated without regard to the inventory service. It is the inventory service's responsibility to understand the ramifications of this event and act accordingly. Even the example given in the comment "DeleteCustomer service invokes the DeleteCustomerOrder service internally" is not a good candidate from ACID transactions (there's also a problem of service granularity here; I'll talk about it later). Since when the customer service decides to delete a comment and request the Orders service to delete orders, there's a reasonable chance that some of the orders are already paid for but not delivered. In this case the customer cannot really delete the customer until all the paid orders are resolved. Or maybe the order service is a facade to a night batch that does the actual deletion. I know I am just fantasizing with these examples but the point is that the customer service has no knowledge on the order service or the inventory service above except the messages supported in their contract. To assume something about the internal behavior is problematic. Even if you know about the internal structure on the onset, the whole idea of SOA is that the services can evolve independently from each other.

Another thought triggered by the example in the comment originated by the granularity of the services (DeleteCustomer service vs. a Customer Service that also supports deleting customers) is that we should be really conscious to the difference between other architectures like 3-tier client/server and SOA. SOA is actually more distributed than 3-tier. We cross a distribution boundary every time we pass a message from a service to a service and not just when we move a massage from a client-tier to an application server. We add this distribution to gain advantages in flexibility and agility. However, we should note that this is a weakness of SOA (considering, for example, that Martin Fowler's first law of distributed object design is "Don't distribute your objects!") and we should really pay attention to the way services interact with each other.

• The granularity of services. Having a lot of fine grained services means there will be a lot of interactions over the wire (even if you don't go out to the network you still have to serialize/deserialize, follow the security policy etc.) rather than internal interactions that much faster.
• The Granularity of messages. The same considerations should also guide us to try to create larger and fewer messages. for the example above . Instead of a DeleteCustomerOrder message maybe something like an UpdateCustomersOrders message that can hold a list of customers and orders and the status changes or,by the way this would also support off-line clients better since they can accumulate changes.
• The assumptions we can make on the other service's availability, performance, internal structure, the trust we have for it etc. We should try to minimize the assumptions we make and concentrate on what can be inferred from the contract. Remember that policies can change externally so the business logic within a service cannot count on them being constant. this brings us back to the issue of transaction. every cross-wire interaction increases the chances of failure -- in transactions one failure invalidates all the transaction is invalidate. every cross-wire interaction within a transaction increases the length of time we lock internal resources (even if we do trust all the involved parties) -- especially if that transaction can extend itself automatically. Also as I've mentioned in the previous post the transactions also open the door for denial of service attacks.

If we want to reap the benefits that are sold under the SOA moniker, like flexibility and agility, we really have to pay attention to this extra distribution and design our services differently than we would omponents in a 3-tier architecture -- but hey, that's why they pay us the big bucks, right ? :)

I should probably also say here that building SOAs is not a goal in itself. We can build perfectly good solutions using other architectures, but if we find that we do need SOA (or any other architecture for that matter) we have to pay attention to the way we implement it to both keep its benefits and not harm other quality attributes like performance, security etc..

One last point (and, I guess, a word from our sponsor :) ) regarding the transactionless solution of eBay. I've got another comment that pointed out the it is most likely that eBay just don't use distributed transactions across their databases but do use local transactions. This sounds reasonable and I also believe that , though it isn't completely clear from looking at the presentation's slides. I hope I'll be able to find out the reply in July when I'll hear Dan Pritchett's presentation in Architecture & Design World 2007 :)

Posted by Arnon Rotem-Gal-Oz at 08:23 AM  Permalink