THE SOFTWARE SIMPLIST

Web Services Wisdom.

January 2007

I've began rethinking certain assumptions about how I use message passing in my distributed systems after reading things like "SBA & EDA Lessons Learned" from the excellent "Panic from Fuzzy" blog. Since I was only familiar with the Tuple Space theory but have never employed space technologies like JavaSpaces, I did the only thing I could do -- I convened a session on it at the Software Architecture Workshop so as to benefit from the knowledge and experience of those frightfully smart guys. [Originally published here]

The functional issue that attracted me to spaces was its ability to do very dynamic like content-based filtering. By subscribing to notifications based on a template, I could replace a broad and deep topic hierarchy and handle some other interesting scenarios as well. For instance, when a given user is working on a certain set of data -- a tree of specific instances, I would like that machine to only get updates on that data, which could be quite a substantial savings in terms of network load. Another case is where the user is only allowed to see certain instances of the same type.

About three quarters of the way through the session, when I was still thinking that I could put entities in the space, someone called my attention to the fact that I would have to give up the cross-entity transactional semantics that I was so used to. For instance, in order to implement a business rule when an order is cancelled the customer may also need to be updated to a non-preferred status. This calls for a transactional scope around both of these updates -- on the business level. If the entities were in the space, the naïve solution would be to Take the order, change its status to cancelled, Put it back in the space, Take the customer, update its status, and Put it back in the space. By not being able to perform all this work transactionally, failure cases would need to be handled by compensating transactions -- a significant jump in complexity.

Needless to say I left that session quite a bit cooler on Space-Based Architectures than I was going in, but later on in the day, over a beer with one of the guys, he suggested that maybe spaces could be used as an alternative way to do message passing. Instead of Taking and Putting entities, we could do the same for messages -- since the business-type of transactions would by-and-large correlate to a single message. We could benefit from the dynamic subscription behavior and possibly do away with a deep, and sometimes fragile, topic hierarchy. Once again my enamourment was on the rise.

But as do all fickle loves, this too was not destined to last. After doing some more thinking I remembered about handling failure cases in terms of message passing. When dealing with reliable messages, the receipt of a message, its handling, and the subsequent sending of new messages is at times required to be all transactional. I was once again required to perform a Take operation as well as multiple Put operations in a single transaction.

I hardly believe that this is the end of the story for me and spaces. I'm still getting my head around using spaces as a technology choice in my current architectures and examining new architectural possibilities around it as well. We are definitely living in interesting times.

Posted by Udi Dahan at 04:58 PM  Permalink |

Mark McKeown brings up an interesting topic in light of Pat Hellands recent paper Life beyond distributed transactions: an Apostate's opinion. There is a common belief in the "inifinte scalability" of the web. REST proponents would have us believe that by designing our systems on the same concepts, those systems would be magically endowed with the same abilities (hardware/infrastructure never being mentioned, go figure).

I read with great skepticism about a "scale agnostic layer" containing all the applicative functionality, that would, once again, be magically scalable on a "scale aware layer" that maps to the web. I've been working on large-scale high-performance mission-critical systems for some time now, and the most common scalability error I continue to see is developers assuming that some infrastructure will handle everything.

It does not work. It has never worked. I highly doubt it will in the future, given this track record.

Application level partitioning, both in terms of data and functionality, supplemented by proper hardware/infrastructure while taking into account availability and fault tolerance has been the only way I've seen projects consistently succeed.

You know what, I don't care about infinite scalability, I need just enough - coupled with all the other ilities, I've got my hands full.

Posted by Udi Dahan at 10:10 AM  Permalink |

Get it here.

[Salt and Pepper, Peanut Butter and Jelly, Toast and Vegemite (if your down under) - these are all things that go together. But what goes with SOA you ask? Why Workflow of course! On this episode we are joined by Microsoft Archtiect MVP Udi Dahan who gives us his insight into putting these two technologies together.]

I just can't seem to shut up about these things. It's gotten so bad that I'll be inflicting the same session I gave in TechEd Developers Europe on our local Israeli crowd at the end of this month. You can find out what other horrors Microsoft has in store here.

Posted by Udi Dahan at 03:48 PM  Permalink |