Among the reactions I got for my previous post on the Singleton pattern in .NET were a couple that talked about the design rationale behind the solution I posted:

Adi Avnit posted on the risk of using a "generic singleton":

"However, there is one possible pitfall to this approach, as it makes this code possible:
Singleton obj = Singleton.Instance;
MyClass obj2 = new MyClass();

While I personally like the idea of having the freedom to use the same class in two different ways throughout the application, I know some people like their Singletons - well, single.
On the other hand, if you write a class from the start as a Singleton this is not an issue.

There is an inherit risk in decoupling a class from it's expected behavior, so take this into consideration before using this pattern."

"don't believe discoverability was a motivation for Singleton. The purpose was to ensure only one instance existed (a print spooler, a file system). You can get discoverability through the technique you explain but the primary purpose of singleton was a pattern of prohibition - to stop a programmer doing something they shouldn't do by accident by enforcing the primary goals of single point of access.

It is more than a global, and the OP, while comparing them to globals, does not acknowledge that this solves many of the problems which made "global" a 4-letter word. Knee-jerk reaction against globals is not healthy once you've mitigated their drawbacks.

It is true that usage of singleton can be misguided and cause coupling - which is why you need to ensure that the usage of the pattern matches the motivation in the first place.

This is a key point of design patterns which Alexander should have made clear in http://en.wikipedia.org/wiki/N...is_of_Form - the pattern is a result of a resolution of system of forces. It only satisfies that system - moving it to another different system will result in poor fitness. "

"design patterns are a sign of a deficiency of a language for the purpose that the design pattern addresses. In other words, the Visitor pattern used in Java points to the fact that Java is deficient in terms of list processing: the `map` and `filter` constructs need to be emulated with lengthy OO incantations."

1. The pattern is needed
2. The solution in the pattern is the best one for the language.
   

In the case of the Singleton pattern (mentioned in the previous post). The intent is to "Ensure a class only has one instance, and provide a global point of access to it". An implementation using templates (or generics) is superior since it provides the same intent but also adds better compliance with the Single Responsibility Principle. While it is possible to create the class as a non-singleton on the one hand it solves the multi-threading issue in one place preventing both duplication of code and mistakes (again this can be especially important in non-forgiving environments like C++). It also lets you (not in the implementation I provided) add flexibility and e.g. let singletons die and (if needed) resurrect themselves etc. Note that even if you are not convinced that using generics is better, there's no doubt that using .NET's thread-safe static readonly variable (public static readonly MySingleton = new MySingleton();)  is a much simpler solution than the GoF one and again, answers the original intent with a different solution.

Another example would be the Template method pattern. The idea behind the template method is to support the Open closed principle (classes should be open for extension but closed for modification) or as the GoF defines the intent:

"Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template method lets subclasses redefine certain steps of an algorithm without changing the algorithm strucuture"

    class Program

    {

        static void Main(string[] args)

        {

            var alg = new Algorithm();

            alg.DoSomething(Funky);

        }


        static bool Funky(string value)

        {

            return value == "somevalue";

        }

    }



    class Algorithm

    {

        public void DoSomething(Func<string,bool>CanGoForward)

        {

            string someVariable = "somevalue";

            if (null!=CanGoForward && CanGoForward(someVariable))

            {

                //do one step

            }

            // whatever

        }

    }

You should note that this "refactoring to fit the language" phenomena isn't limited to GoF patterns (or .NET :)) , for instance you can consider the use (or lack thereof) of Dependency Injection in languages such as Ruby which I wrote about a few months ago.

To sum up - when you talk about design patterns you need to consider the implementation language, design pattern deal with deficiencies in the language and different languages have different deficiencies and may have better solutions to the problems solved by the patterns

Simon @ CodingTheArchitcture recently asked "How big is your software architecture document? (and who reads this stuff anyway?)"
He notes that in a UG meeting most of the attendees has SADs that were more than 50 pages long.
It would probably not be too surprising if I say than in my opinion the answer is that it depends. Reflecting back on some of my past projects I had SADs that varied in range from a 200+ "write-only"* document to a less than 10 pages lean document. And the sizes match the intended usage of the documents. for instance in the two extremes mentioned. The first case it was a huge mission critical project with a specific requirement from the customer to have an "official" SAD and it was written to satisfy some project milestone (PDR) . Where the second extreme is an agile project where the architecture document was a working document, written some 10 iterations into the development to highlight some of the emergent guidelines.

What is common to all the SADs I wrote (or was responsible to) is that they all tried to grasp the essence of the design, all used multiple viewpoints to describe the solution, all were focused on quality attributes and all explained the rationale behind the decisions.

• If you drone endlessly with details you don't see the forest from the trees.
• if you don't use multiple views - you are likely to miss important aspects of the solution
• if you aren't focused on quality attributes that you are most likely documenting design and not architecture
• and if you don't explain the rationale then the document doesn't have a lot of added value beyond the code itself
  

A while ago I asked "who tests the test":

" One question I don't hear asked too much is "who tests the tests?" - after all we are writing all this additional code - if we write so many bugs in our production code that we need tests - what are the chances the test code is clean?"

One answer I had was making sure to keep the triad of tests->code->acceptance tests. so that each two test the third. This is especially good since, usually, the acceptance/integration tests are not written by the developer who wrote the code.

One answer I got was that tests are usually simple enough to be self evident and when tests contain complex constructs you should probably test them as well.

Basically, tests can have two types of problems

• Not testing the right thing
  
• Not testing the thing right

1. We have a component that interacts with resources in different URIs, so when resources wake up they register themselves with that component. Resources register two URIs -> one for control and one for interactions. We've added a business rule that both URIs should be on the same host (naturally a new test for that was also added but that not the point) suddenly a bunch of the old tests started failing. Sure enough, I took a look at the new code, found a bug, fixed it, but the tests were still in the  red. In the end it turned out one of the Uris that is registered during the tests had an extra "L" (locallhost instead of localhost) so the tests failed with the new rule - as they should..

2. After writing a piece of code to "just work" it had to be made multi-threaded. I needed to write a test that would make sure the component can handle multiple concurrent requests. Sure, I thought, I'll just run few or even all of the other tests in parallel and be done with it. The only problem was that the other tests were written to be isolated (i.e. new instance per test) running them in parallel had no effect what-so-ever.

What can we do to help us locate problems and bugs with tests?

• remeber that Test code is code - so as you write more tests you should also refine the tests design and refactor the code accordingly. For instance in the first example above, it was easier to diagnose the problem since the faulty setup was done by a single method used by alll the failing  tests (Single Responsibility Principle @ the method level).
• Test-First - One of the reasons test-first is beneficial is because you can actually see the test fail before you implement the code to make it work. This helps you make sure that the test actually tests the behavior you want. When the code is already in place it is harder to make sure if the test works because of the change or not. In the second example running all the tests in parallel everything was still green, which triggered me to write specific tests for checking for multi-threading issues.

Someone calling himself r r left the following comment on part IV of my series of posts on SOA definition:

"I keep trying to read this series on SOA unfortunately suffers from the same disease as the rest of literature on the subject. stays general to a comfortable level so it can't really be applied anywhere, tends to complicate things where is not clear if it's needed, and encourages philosophical debate on what ultimately is a business (and so concrete) requirement. Meanwhile the serious (IMO) issues stay untouched - how does one actually approach an integration project with functionality, performance and security in mind. Which should be the standards used (considering the tens of standards on WS out there). How granular should the WS be (I'm done with answers like "not too much, but enough", or "well, depends on your project"). "

That said let's take a look at the more interesting parts of this comment. First, the thing that bothers me about this reaction is (what seems to me as) the quest for final and concrete recipes. For instance consider the comment on service granularity

"How granular should the WS be (I'm done with answers like "not too much, but enough", or "well, depends on your project"")

Nevertheless it is not a pure guess either. You can use some guidelines and measure them against your specific project/system/enterprise needs. Personally The set of guidelines I use is based on the fallacies of distributed computing :

1.  The network is reliable
2.  Latency is zero
3.  Bandwidth is infinite
4.  The network is secure
5.  Topology doesn't change
6.  There is one administrator
7.  Transport cost is zero
8.  The network is homogeneous

Regarding the other questions (how do you approach a real system), well, if you pardon me for banging my own drum, that's exactly why I started to write my experience on these matters as patterns. for instance if we look at the saga pattern (one of the patters I published online). you'd see that it is talking about achieving distributed consensus in a transaction-like manner. I talk about the problems of using distributed transaction etc., offer an architectural solution (the saga ) and then discuss relevant technology issues (e.g. WS-BusinessActivity ) as well as its implication from quality attributes perspectives (Integrity and reliability). Nevertheless even these patterns aren't an end-all solution. different circumstances require different solutions
Both my previous job and my current one involves building a scalable solution on-top of algorithmic engines. In my previous job I  managed the construction of a biometric solution that allows using multiple biometrics. In my current job I manage the development of  a mobile visual search solution . Again, while on the surface both needs to get some data, run a few  algorithms and produce an answer. These systems have very different quality attributes. On the first system we had to handle very large databases, hundreds of queries, an emphasis on modifiability and security, the current one needs millions of queries, almost no database, low latencies and emphasis on usability.  These differences result in radically different solutions, with different services, different interactions , use of different patterns etc. There's no "one right answer" (tm)

//bad implementation - don't use

//also note some initialization etc. is omitted for brevity

 public static Foo Instance

{

      get

       {

        // This lock allows thread safety.

             lock (_mutex)

             {

                   if (_instance == null)

                        _instance = new Foo();

                  return _instance;

            }

      }

   }

public  sealed class singleton

{

        public static readonly MySigleton Instance = new MySingleton()

}

But that's not all- see, one of the problems of the Singleton pattern (besides the ones mentioned above) is that it is also a violation of the Single Responsibility Principle (SRP see OO primer). You can solve this problem if you use Generics and create something like C++'s template based Singleton class:

namespace ConsoleApplication5

{

    class Singleton<T> where T : new()

    {

        private static readonly T inst = new T();


        public static T Instance

        {

            get { return inst; }

        }

    }

}



and then use that simply by doing something like Singleton<MyClass>.Instance

And yet, last week when we were pushing for completing our first major milestone I decided to try real sticky notes, and finally I understood the difference - constant visibility. Computerized tasks seem to be just as accessible as physical cards but in fact they aren't. Yes, everyone can see the status whenever they want. But the fact that you can doesn't mean that you do. Digital cards don't have the "in-your-face" kind of effect that always visible notes has. I've seen a notable  difference in maintaining the dev team's focus and making managers (the CEO in my case) feel they are in the loop and that progress is constantly made.
My conclusion is clear. While I still use electronic tools, I guess, until we'd have large enough e-ink boards to allow us to get the same effect physicals cards give us I'll just have to keep restocking my sticky notes inventory :)

The next architectural style SOA builds on is Pipes and Filters, Unlike Layers and Client/server which I described in previous installments, Pipes and Filter is not also a base style for REST. This basically, this style is where SOA and REST begin to diverge.
The pipes and filters architectural style defines two types of components - yep you've guessed it, Pipes and Filters.

Filters -  are independent processing steps they are constrained to be autonomous of each other and not share state, control thread etc.
Pipes - are interconnecting channels

The pipes and filters style brings to SOA things like the autonomy of services, the sense of explicit boundaries. For instance, this is the basis for why you wouldn't want to do distributed transactions across service boundaries, which I blogged about several times before.

The pipes part of the "pipes and filters" also means that the wiring can be taken care of outside of the services themselves and that you can control them externally, this works well with ithe use of middleware (service bus). Additionally Fielding (you know, the REST guy) also mentions that

"One aspect of PF styles that is rarely mentioned is that there is an implied "invisible hand" that arranges the configuration of filters in order to establish the overall application. A network of filters is typically arranged just prior to each activation, allowing the application to specify the configuration of filter components based on the task at hand and the nature of the data streams (configurability). This controller function is considered a separate operational phase of the system, and hence a separate architecture, even though one cannot exist without the other."

So as you see, pipes and filters is one of the important pilars of SOA, in the next part (unless I'll have to clarify things about this post) I'll talk about the last architectural style SOA builds upon "Distributed Agents".

If you are a .NET developer/architect and wanted you may know that to implement a RESTful application in Windows Communication Foundation (WCF) you really have to jump through hoops.For instance  you have to go back to basics and use the HttpRequest and HttpResponse, handle the breakdown and parsing of URI hierarchies yourself not to mention  fight  with the  bindings .

Fortunetly this all changed with WCF 3.5. True, .Net doesn't have (to my knowledge anyway) something like RESTlets, but at least building REST on http is pretty straightforward...

    [ServiceContract(Namespace = "http://paperlnx.Contracts/2007/12", Name = "ISessions")]
    public interface ISessions
    {
        [OperationContract]
        [WebGet(UriTemplate = "/Sessions/{sessionId}")]
        [ServiceKnownType(typeof(Atom10FeedFormatter))]
        SyndicationFeedFormatter ListSessionStatus(string sessionId);
	.
	.
	.

1. Integrated support for HTTP verbs  - The sample above shows the support for GET. You can get the other verbs almost as easy with the WebInvoke Attribute. To do that simply specify the verb you want e.g.   [WebInvoke(Method = "PUT")] , [WebInvoke(Method="DELETE")] etc.
2. Support for URI templates -  In a way not too far from Joe Gregorio's IETF draft , WCF supports the notion of providing a way to describe families of URIs. This is done using the UriTemplate class. The WebGet and WebInvoke attributes also accept URI templates as variables and map the variable values (the curly brackets ones {}) to parameters of methods.
3. Support for standard  formats - you can use plain XML or you can choose to use RSS and ATOM syndication formats. In its most basic form you just create a syndicationfeed and format it to atom feed. Which is what we do for error messages:

           public static SyndicationFeedFormatter GenerateAtomError(string errormessage, string description,Uri location)
           {
               SyndicationFeed feed = new SyndicationFeed(errormessage, description, location);
               return new Atom10FeedFormatter(feed);
           }
   

   Naturally you can also add items and element extensions to all elements (e.g. the feed or items)

• Technical infrastructure; OS, directory Services etc.
• Application infrastructure; Apps, Portals, DBMSs
• Application Landscape; SOA, EDA
• Bushiness Processes; BPM

Jack uses the term "tier" for layers within the same level of abstraction. For instance, he gives the following examples:

E.g. the layer of business services may be arranged in the tiers: front-office, mid-office and back-office. At the next lower layer, the application layer, services may be arranged in the tiers: UI, business logic and data persistency. The interaction of services between two tiers may be bidirectional (but may also be constrained to unidirectional).

The current game plan is for Eric to focus on the SOA pitfalls ("anti-patterns") part of the book, Udi to provide a "putting-it-all-together" chapter , and for me to cover what’s left. I am sure however, that their experience and insight will also help make the other parts of the book (even? ) better.

If you are not familiar with the book, you may want to take a look at the first chapter and/or some of the published patterns like Saga, Service Firewall, Gridable Service, Edge Component and (a very early draft of) Aggregated Reporting pattern. Also you can take a look at the slides from my "SOA Patterns" presentation at Dr. Dobb's Architecture & Design world last year, which illustrates some additional patterns.

1. How does this [layers - ARGO] play with two services talking with each other? One pubs to the other's subs?The other requests to the first's response?

2. How valuable is the layered abstraction?

1. As I explained in the previous post. Layers does not necessarily mean unidirectional relation from a top layer to a lower level one -- it does mean that a layer can only know a layer that is diretly above or below
it. In other words, the bidirectional interaction between two services -- i.e., the request, reactions, events, etc. flowing between them -- do not violate the layered style constraints.

2. So, how valuable is the layered abstraction to SOA? The short answer is -- very :). Again, as I mentioned in the previous post, the main reason layers don't seem that valuable is because they've been misrepresented and misused. Layers bring added flexibility to SOA. The fact that a service or any other SOA component cannot see beyond the next layer enables things like the ServiceBus, Edge Component, Service Firewall, etc. Without layers it would be harder to have autonomous services as other services could (potentially) have access to the innards of the
service adding more coupling and preventing independence.

Udi and I usually see things eye-to-eye. So I guess that if I managed to get him confused, more clarification is warranted :) I'll do that in two posts, this one which will explain the concept of layers and the next which will explain why it is paramount to SOA (and answer the two questions).

Usually when I review an architecture, one of the first (sometimes the only) architectural artifacts I am shown is a "layered diagram" of the architecture; e.g., something like the following:

These sort of half layers/half block diagrams with or without the common three layers (which also appear in the diagram above) of "UI" "Business" and "Data" give the whole idea of layers a bad name.

The key differentiator between layers and just a bunch of blocks is the limitations on the allowed communication paths between the components (layers vs. blocks). In the previous post I quoted an old (2005) definition I had for layers where I said the following:

Typically a layered is allowed to call only the layer below it and be called only by the layer above it (but there are variants e.g., a layer can call to any layer below it; vertical layers that can call multiple layers; etc. -- all is fine as long as the layers communication paths are limited by some rules).

Alas, I was too quick on the copy/past and everything in the brackets (bold) is wrong -- it should actually say -- "but there's another variant where layers are allowed to call the layer above it or below it". The other variants (like a layer that can all any below it) just muddy the water, makes it hard to distinguish between layers and regular components and thus make layers seem unimportant. consider the following diagram:

So, in the above diagram, the relations are the Component D and Component C know each other. Component D is made of two layers (A and B). Note that a more proper representation should also explain the relations allowed between the layers, i.e., is it unidirectional or bidirectional (unless there's a common convention in the project).

Why is the distinction between layers and other type of components important? Because Layering gives you some benefits which "just components" don't:

1. Layers allow information hiding. Since we don't know the inner working of what's beyond the layer.
2. Layers allow separation. Things beyond the immediate layer are hidden from each other. This means that things which are beyond the layers are loosely coupled in a way that allows for flexibility and the addition of capabilities. for instance adding a firewall between your computer and the Internet.
3. Layers allows changing the abstraction level. Since layers are hierarchical in nature, moving through the layer "stack" you can increase or decrease the level of abstraction you use. This allows expressing complex ideas with simple building blocks. The best known example for this is the TCP/IP stack moving from an abstraction level close to the hardware of the network interface layer to the application level protocols such as HTTP.

On the downside, layers hurt performance by adding latency. Also, too many layers introduce added complexity to the overall solution (e.g., it is harder to debug).

It it interesting to note that Interfaces are in fact leaky layer abstractions (versus, for example, SOA contracts which are not leaky), since when you use an interface you still need to instantiate the object which is otherwise hidden behind the "layer" (interface). This is basically the reason we want something like dependency injection (DI) -- to help make the abstraction complete and why languages like Ruby where the contract abstraction is complete - you don't need DI (I discussed Ruby and DI in another post)

Another issue which I mentioned here is the difference between logical layers and physical (or potentially physical) layers. I usually call the first kind layers and the latter tiers. logical layers are local and can assume a lot about their neighboring layers. Tiers or physical layers can be distributed, which carries a lot of implications (something I discussed here recently in relation to Microsoft Volta).

One important message is this: Software is becoming a media business. The Net is not only a universal medium, a distribution channel for words, sounds and images. It is also turning into a universal computer -- the machine we use to run software and store data.

You might also want to check out a presentation a few of my colleagues and myself prepared about half a year ago where we talked about the same phenomena; see Future of Home Computing (5.5Mb).

In this sense the Microsoft-Yahoo merger (if it will follow through) will result in a company which is focused more on the Internet aspects (Yahoosoft?) rather than the more traditional (some would say legacy) desktop aspects of Microsoft.

It is also interesting to note in this sense that while winning the web search (and the related ad-revenue) is something Microsoft is very interested in. The eyes of Microsoft (and Google for that matter) are on the next battlefield -- mobile search. While there are something like 305 million broadband subscribers worldwide, the number of mobile phones sold just in the last quarter of 2007 -- 334 million -- is far greater, and the total of mobile phones worldwide is in the billions. An added bonus here is that Google is yet to take over this market (al though it is moving in that direction with things like the Android platform and location-based search and services).

We are judging two approaches to analyzing massive amounts of information, even for less structured information.

The problem with that from there they continue to define a problem in database terms, then show how MapReduce will not be as good as a database in solving it. Well, duh.

The fact that isolated queries may run better in a pre-indexed database should come as no great surprise. As I noted in the previous post on the subject, MapReduce can be used to create the appropriate index or
partition the data into smaller chunks that would be easier to use to answer the type of queries David and Michael mention.

As Mark Chu-Carroll explains Map/Reduce and databases don't solve the same kind of problems.

Also what happens when the database is constantly updated?! I don't mind how scientifically accurate are the measurements that say database scale like no other things. I am more comfortable with the empiric
experience by companies like Amazon, Diggs, Google , and eBay who found they have to shard their data to support their scalability needs and not use distributed transactions/distributed databases.