IF YOU BUILD IT

... Will they Come?

August 2006

Over the last few months I’ve blogged about many object-oriented principles, including Single Responsibility Principle, Open Closed Principle, Just Enough Design Up Front, and You Ain’t Gonna Need It, among others).

I’ve assembled (and slightly edited) all these posts into a single white-paper and a presentation (actually the presentation preceded the posts and triggered them). You can get them both here.

Posted by Arnon Rotem-Gal-Oz at 07:52 AM  Permalink |

As mentioned in the previous post on the architect's soft skills, strategic thinking is the second soft skill essential to understanding where the solution/system should be going. While system thinking takes care of the system (in its environment), strategic thinking is about understanding where the organization is heading and its long term goals so that the solution being developed is in sync. (I guess it is easy to see why this is important for internal projects but I think it is also important for product/solution companies.)

Strategic thinking involves the techniques and thinking processes essential to setting and achieving the business's long term priorities and goals. Strategic thinking (understanding the problems) is the preamble to strategic planning -- but we'll leave planning to management and focus on the understanding which is important for the architects.

Ruth Malan and Dana Bredemeyer define the Strategic Thinking soft skill (which they call "Strategic Perspective"):

[An architect who has strategic perspective (ARGO)] understands the industry, market, customers, competitors, suppliers, partners and capabilities of the business. Identifies opportunities and threats, and actively identifies trends and future scenarios.

This is a good definition because it really explains why this skill is so important for architects. One of the architect's core roles is to understand what the different stakeholders need, then balance these needs to create a usable, robust solution. Their needs expressed as quality attributes are the driving force of the architecture.

Furthermore. if you think about all this "align business and IT" stuff we constantly hear about these days (especially in regard to SOA) -- it is evident that all the careful planning of how technology and software can help in getting that alignment is useless unless we really have a good understanding of where we are going and what this alignment really is. Thus, the way to do this is for the architect to first understand what the business is about and where it is going, then take these insights and translate them to technological and architectural decisions that ensure the solution suites the business needs.

Posted by Arnon Rotem-Gal-Oz at 03:04 AM  Permalink |

Another OO principle is "Inversion of Control" (IoC). While not as basic as some of the others (such as the Single Responsibility Principle or Liskov Substitution Principle), IoC has been in wide use as techniques that implement it (Dependency Injection, for instance) become more prevalent.

IoC means that an object that wants some functionality from another object (let's call it "provider") gets called by the provider, rather than the usual flow of logic where the object calls the provider when it needs it (hence the "inversion"of the control since it is the provider that controls who gets called where). Say, for example, that you are writing a GPS-based navigation program and you want to draw a car symbol on a map to show the current position. One way to do that would be that everytime the car position changes, you call the map object and tell it to draw the car at that new position. Another way to do it would be for the map object to call the car object whenever it is convenient for the map (x times a second, for instance) and ask for its position. This second option uses Inversion of Control.

Why do this? In the scenario above, one motivation might be the load. The GPS might update its position tens of times per second, but for a smooth movement on the map you only need to update it 5-10 times a second. In the more general case, it is just simpler to get the functionality you want if you use IoC. Again, for the example above, instead of understanding the ins-and-outs of the map object, you just implement an interface to return the symbol of the car and retrieve the position, then let the map object worry about the rest.

This is why IoC is a popular technique used in frameworks (see also Framework vs. Libraries), or as Martin Fowler said, it is the almost the defining characteristic of frameworks.

Some people confuse IoC with the Dependency Inversion Principle; for example, see the IoC definition in Wikipedia. However, they are quite different. DIP talks about type dependency (classes should depend on abstractions) while IoC talks about flow of logic.

Another misconception about IoC is that it is equal to Dependency Injection; see the IoC definition in HiveMind, an IoC-based container. I think that the source of this (and the previous) misconception comes from Dependency Injection enabling containers like Spring or Hivemind.

DI containers utilize IoC. The container, for example, controls the lifecycle of the objects involved. Another example is that the container also performs the injection on its discretion. The end result (of the dependency injection) is that the two (or more) objects only have a dependency on interface (that is, the object that got the dependency injected into it only knows the "dependency", the other object, by its interface). The end result is an implementation of the Dependency Inversion Principle (only depend on abstractions). To top that, the words "injection" and "dependency" occur too much between those three terms "Dependency Injection," "Inversion of Control", and "Dependency Inversion Principle".

You may also want to read Martin Fowler's article "Inversion of Control Containers and the Dependency Injection Pattern" which explains the difference in more detail.

Posted by Arnon Rotem-Gal-Oz at 04:38 AM  Permalink |

When talking about multi-tiered architectures, we need to remember that the tier boundary is significant. The tier boundary is where distribution happens and if you remember the "fallacies of distributed computing", you know not to take that lightly.

A tier is a physical boundary (versus an Edge in an SOA which is a logical boundary, for example) and the implications are numerous. For instance, you need to consider:

• Trust--who do you let in?
• Security--what do you send out?
• Performance--you need to serialize to pass the boundary, and remote data is expensive to fetch.
• Availability--what happens if you crash?
• Manageability--can anyone see what's your state? Help you recover?
• Temporal coupling--can you afford to make synchronous calls?
• and many similar questions.

Yet many times people think passing a tier is as simple as passing a logical layer. I should know. I made this stupid mistake more than 15 years ago in one of the first distributed systems I designed. I planned this beautiful separation of the UI controls from the business logic (I didn't know it was called "MVC" and that someone else had figured it out ages ago, so I was pretty proud of myself). When you clicked on a button you just used metadata to say that BL should catch it. I had all this wonderful "infrastructure"that handled passing the call to its destination.

But then we wanted to take this n-layer application and put the BL in an "application server" which will handle multiple clients. Oh--now we need to move events over the wire , handle calls from multiple unrelated clients, pass a lot of data back and forth, and what about security... you can imagine the fiasco.

Thus, as Niels Bohr once said, "An expert is a person who has made all the mistakes which can be made in a very narrow field." But you don't have to make the same mistakes. Just remember that a tier is a natural boundary. You know what? You should probably even want to consider it the edge of a cliff at the end of your application--and be careful not to fall down.

Posted by Arnon Rotem-Gal-Oz at 05:17 AM  Permalink |

My previous post on the architect's soft skills discussed leadership. If you have good leadership skills, you can get people to follow you. This increases your responsibility to actually know where to go. Two soft skills that can help you with that are:

• System Thinking, which I address in this post.
• Strategic Thinking, which I'll address on the next part of this series.

Microsoft Encarta defines system as "any collection of component elements that work together to perform a task." (You may want to take a look at some of the characteristics of systems by Donald E. Gray). When we get a problem--that is, a software solution that needs to be built--we tend to think about breaking it down into manageable "parts" (the subsystems/components/services/objects) that makes that solution. This, however is not enough. We also need what is known as "Systems Thinking".

System Thinking originated as a way to think about social systems but has emerged as a way for problem solving problems for systems in other practices. In essence it is about understanding that system parts that work together behave differently then each part alone ("The whole is greater than the sum of its parts"). Which is, by the way, the reason "loose coupling" is such a holy grail--as it helps reduce the parts interdependence and interactions (and thus simplify the system).

One important trait for understanding systems behavior and component interaction is the ability to create abstractions and models; that is, simplifications of the reality which contain enough detail to be useful (Another thing that is needed is the ability to communicate that to the different stakeholders which is another soft skill I'll expand upon). It is important to remember that mental models limit the perspective we have which is why we need to have several models and why it is beneficial to have more than one person working on a problem.

As it happens this is also aligned with my definition of software architecture:

Software architecture is the collection of the fundamental decisions about a software product/solution designed to meet the project's quality attribute requirements. The architecture includes the main components, their main attributes, and their collaboration (i.e. interactions and behavior) to meet the quality attributes. Architecture can and usually should be expressed in several levels of abstraction (depending on the project's size).

This definition gives interactions and the environmental implications on the system as a whole the same weight as designing the parts themselves. if the architect doesn't (or can't) understand the effects of the components playing together the quality attributes (performance, availability etc.) of the system will suffer and the system will not operate as planned.

For an introduction to the subject, I recommend Gerald M. Weinberg's Introduction to General Systems Thinking. It lives up to its name.

Posted by Arnon Rotem-Gal-Oz at 07:49 AM  Permalink |

Another OO principle worth discussing is "Don’t Repeat Yourself" (or "DRY" for short). I’ve seen it referenced as "Single Point Control".

The principle simply states that:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

Actually saying that this is an "object-oriented principle" does a disservice to this principle because it is a fundamental principle for software engineering. For example, consider:

• Database normalization rules. Redesigning tables so that data will only be defined once.
• Refraining from magic numbers. Adding constants, variables.
• Parameterizing algorithms. Instead of rewriting them for new sets of requests.
• Writing functions instead of repetitive code.
• Even go-to statements.
• etc.

For object orientation DRY means things like using inheritance, template methods etc. DRY is also related to several refactorings like extract class, extract method, extract supereclass, etc.

The most obvious benefit of DRY is increasing maintainability and testability--you have one place to go when you need to change something and you only have to test once for all. if you fail to apply DRY a maintainer/tester will have to repeat their efforts as well (plus the overhead of understanding what is it exactly that happening and why the results are different).

The pitfall of DRY is trying to generalize too much which can result in over complication (also see YAGNI).

Posted by Arnon Rotem-Gal-Oz at 05:16 AM  Permalink |

Liskov Substitution Principle (LSP) is yet another object-oriented principle. However, unlike the Dependency Inversion Principle I recently discussed it is much more subtle. Still, it is just as fundamental.

LSP was defined back in 1988 by Dr. Barbara Liskov. The original definition is a little hard to follow, but goes like this:

What is wanted here is something like the following substitution property: If for each object o1 of type S there is an object o2 of type T such that for all programs P defined in terms of T, the behavior of P is unchanged when o1 is substituted for o2 then S is a subtype of T.

In plain English this means that any subclass of a class should always be usable instead of its parent class which makes Subclassing a public behavior preserving operation. Another way to look at this principle is known as design by contract:

• A sub class should Honor the contracts made by it parent classes
• Pre-conditions can only get weaker
• Post-conditions can only get stronger

The classic examples for violation of this principle are an ellipse that inherit a circle or a square that inherits a rectangle (hint: the rectangle has a setWidth and SetHight methods). I'll try to show that by an example of how Microsoft violates LSP within .NET class hierarchy

.NET (1.1) has ContextBoundObject (which inherits from a MarshalByRef object). A nice feature of ContextBoundObjects is that .NET is that the .NET CLR provides a transparent proxy to class derived from it, intercepts the call and even lets you extend its behavior by providing custom sinks and adding them to the chain called by the interceptor. (This feature has been used to create simple AOP solutions in C# (you can see an example here. Note that using this for AOP not recommended by Microsoft). So far so good. Another useful class in .NET is ServicedComponent (your primary and almost single gateway to COM+ services in C# (before COM+ 1.5 Services without components or System.Transactions)). As it happens ServicedComponent is a subclass of ContextBoundObject, indeed Microsoft uses the proxy and sinks to add the different COM+ services to objects that inherit from ServicedComponent. The LSP violation is that while ContextBoundObjects lets developers add their own sinks ServicedComponents do not... I once sat for a day trying to figure out what went wrong in a piece of code that worked perfectly when we used "plain" ContextBoundObjects and suddenly failed miserably with Enterprise Services.

Violations of LSP results in all kind of mess--can get superclass unit test to fail, a lot of unexpected or strange behavior (e.g the example above) and also a lot of OCP violations--as you have if-then statements to resolve the correct subclass. LSP is the reason that it is hard to design and create good deep hierarchies of sub classes and the reason to consider using composition over inheritance. It is sometimes hard to see the problems that will be caused by violating LSP - and you may be tempted to think everything would be okay. But it will come back to haunt you and since you need to dig in into the internal working of the specific subclass it is hard to track these problems.

Posted by Arnon Rotem-Gal-Oz at 06:04 AM  Permalink |

I often see on forums questions like "I've just been entrusted with developing an architecture for XYZ, but I'm new at this. Can anyone tell me where I can find information on the appropriate architecture?" or "I have this simple 3-tier architecture I am building. Should my UI call the DAL or should my BL do that etc.?"

Surely enough kind members of the forum will point these poor souls to vendor X or vendor Y shelf architecture which clearly (yeah, right) explain how to do their favorite architecture du-jour.

I have one thing to say about this: Architecture cannot be defined without context and specific requirements. Okay it can, but the end result of that defining architectures *without* context or requirement is either something which is way too complicated ("we have two clients doing data entry and one server, but we have all this AJAX, web-services, Oracle RAC--okay so we have 2 servers--and an ESB infrastructure to boot") or too simplified. In any event, it likely doesn't fit your needs.

This is not to say that made-by-vendor components (Microsoft's application blocks now dubbed "factories", for isntance) can't be used as-is (or adapted) to your architecture. However this will most likely come with a price tag and you need to understand the implications and the effects of this choice on your application. The chances that you'll get a perfect fit are very low. For most application, this would either be too simplistic or too complicated. Think about your system's quality attributes before you commit to some off-the-shelf architecture.

Posted by Arnon Rotem-Gal-Oz at 09:44 AM  Permalink |

[edited 08/15 - fixed typos and phrasing (thanks to Paul Gardner)]
One of the simplest, yet most fundamental, object-oriented design principles is Dependency Inversion Principle (not to be confused with Dependency Injection or Inversion Of Control).

The principle simply states that:

• Higher level modules should not depend on lower level modules. Both should depend on abstractions (interfaces or abstract classes).
• Abstractions should not depend on implementations.

The principle is simple since unlike some of the other principles it is structural and doesn't require a lot of thinking (vs. other principles like YAGNI, OCP, ISP, etc.).

The principle is fundamental as it is one of the main difference from procedural programming (if you remember what that is). In procedural programming, you had a program that depended on its modules which then depended on its functions. If a detail in a function changed, it could have severe ripple effects on all the hierarchy that depended on it. Applying DIP means that both the program and the classes depend on interfaces (and the whole interface-based programming idea). Other implications of DIP are things like layers and the afore mentioned Dependency Injection pattern.

Applying DIP helps avoid the above mentioned problems by increasing loose coupling. Abstract interfaces are usually more stable (compared with their implementation) so it is easier to update implementation without affecting the class consumers. This also helps increase testability (easier isolation) and decrease the rigidity of the design

To sum up, DIP is easy to follow and provides some very nice benefits for your OO designs.

Posted by Arnon Rotem-Gal-Oz at 05:31 AM  Permalink |

Solutions architects are the "technical managers" of projects. This means they are responsible that all the designs and code is aligned to the functional requirements and that the quality attributes are kept.

However,architects are seldom the direct managers of a development team (and even if the architect is the manager, you still need to inspire your workers. And tyranny? Well, that just doesn't work). You might think that establishing yourself as a technical authority will be enough (that's why they made you the architect in the first place, right?) but it isn't. You need to cultivate your leadership skills as well.

What is leadership anyway? Leadership is about exerting influence on people and thus increase the chance that people will follow your vision and decisions. To do that you need to gain the respect of the teams you work with, communicate your vision and designs clearly and be trustworthy.

So how do you do that? I don't have a definitive answer to that but here are a few things to think about which I think are useful:

• Provide direction. To lead you need to know where you are going and make the decisions that will get you there.
• Explain your decisions. Deus ex machina doesn't count. You work with intelligent people, they may not have your experience or the same depth of knowledge but they want to know why they are doing something.
• Listen to what others have to say. They may actually say something valuable you know :)
• Don't postpone decisions and don't avoid conflict. This will not make them go away. Do try to manage your conflict though.
• Motivate people. This can be done by things like mentoring and teaching, allowing people design freedom. (Letting others make the decisions in their relative fields/areas even if the solution they propose is not perfect.)
• Set an example. For example, you can sit (pair with) other developers in the team to design/code important things together. (This is probably where the views on architect must code code comes from.)

Leadership is only one of the soft skills that architects need, but it is one of the most important. As I mentioned earlier, architects are usually not the managers but they do need to lead if they want to ensure the stakeholders needs (the system quality attributes) will indeed make it into the solution.

Lastly, I don't know about a lot of courses that address the architect leadership skills; in fact I only know of one. Dana Bredemeyer has a three-day workshop on the architect's role which includes leadership training. However, considering it is so important I think there should be more.

Posted by Arnon Rotem-Gal-Oz at 02:46 AM  Permalink |

In my previous post I mentioned Robert Glass's book Software Conflict 2.0 which is a reprint of a 1990 book. I gave one example of the thoughts in this book that are still relevant today.

It seem odd that a 16-years old book contains relevant information for building computer systems today. Nevertheless looking at my bookshelf I see quite a few other books (some of which are older) which contain essential information and brilliant ideas that are still very relevant today. Here are just a few examples:

• Peopleware Second Edition by Tom Demarco and Timothy Lister (Second edition 1999, original 1987). Development should be centered on people and not on methods

• The Mythical Man-Month: 20th Anniversary Edition by Fred Brooks (20th anniversary edition 1995, original 1974). "No silver bullet" don't just add people to a late project it will only slow it down and many more.

• Code Complete by Steve McConnell (Second Edition 2004, original 1993). A great book on coding.
  
  
• An Introduction to General Systems Thinking (Silver Anniversary Edition) by Gerald M. Weinberg (silver Anniversary Edition 2001, original 1975). Observations on thinking processes
  
  
• About Face and About Face 2.0 (2.0 Edition 2003, original 1995). UI mistakes and guidance, the second edition also details a scenario based approach to user experience design.
  

I'm not the only one to notice this. I recently saw a post by Rob Keefer at his pomiet blog that notes that the idea of TDD was mentioned 35 years ago in Gerald Weinberg's Psychology of Computer Programming.

With so much good advice lying around for years (11-40+) and the fact that only about 30% of the projects are successful (on-time;on budget; on scope) I think one question we should all ask ourselves is -- don't we ever learn?

Posted by Arnon Rotem-Gal-Oz at 06:26 AM  Permalink |

I just finished reading Software Conflict 2.0: The Art and Science of Software Engineering, by Robert L. Glass.

This is a reprint (with few new retrospective additions) of his 1990 book. While the technology mentioned in the book is outdated, many of the author's ideas and views are still valid. The book is a collection of short articles on various subjects, and one of the more interesting articles is about the cognitive side of design.

Glass explains that research done showed that design includes the (obvious) steps of understanding the problem, decomposing it into goals and objects etc. The essence of design, however, are the mental steps taken by designers:

1. They construct a mental model of proposed solution
2. They mentally execute the model (i.e. simulate the model to see if it solves the problem)
3. If the model isn't good enough (e.g., too simple) replay the simulation to find what wrong and remodel
4. Repeat 1-3 until the model seems to solve the problem

Glass also said that people tend to start with a model that worked on a similar problem and that good designers perform this process extremely fast. As I was thinking how one can train himself to get better at performing this task, it occurred to me that the practice of Test Driven Design actually mimics this process and makes it explicit.

Thus not only does TDD will help make your design better it actually trains you to design better altogether.

Posted by Arnon Rotem-Gal-Oz at 11:27 AM  Permalink |