Application ignorant, infinite scalability?

Mark McKeown brings up an interesting topic in light of Pat Hellands recent paper Life beyond distributed transactions: an Apostate's opinion. There is a common belief in the "inifinte scalability" of the web. REST proponents would have us believe that by designing our systems on the same concepts, those systems would be magically endowed with the same abilities (hardware/infrastructure never being mentioned, go figure).

I read with great skepticism about a "scale agnostic layer" containing all the applicative functionality, that would, once again, be magically scalable on a "scale aware layer" that maps to the web. I've been working on large-scale high-performance mission-critical systems for some time now, and the most common scalability error I continue to see is developers assuming that some infrastructure will handle everything.

It does not work. It has never worked. I highly doubt it will in the future, given this track record.

Application level partitioning, both in terms of data and functionality, supplemented by proper hardware/infrastructure while taking into account availability and fault tolerance has been the only way I've seen projects consistently succeed.

You know what, I don't care about infinite scalability, I need just enough - coupled with all the other ilities, I've got my hands full.

Posted by Udi Dahan at 10:10 AM  Permalink