IF YOU BUILD IT

... Will they Come?

July 2006

Malcolm Davis made the following comment on my attempt to define software architecture:

not a complete definition

Arnon's (and most of the industry's) definition of software architecture is counter to the point of architecture and engineering in the world of construction. Architecture approaches the problem from a user/business problem space. Engineers solve the technical space, and not necessarily the user/business space. The following is from Alan Cooper: "Architects synthesize people, purpose, and technology. If you just take people and technology, you have art and entertainment. If you just take technology and purpose, it's engineering. And people and purpose without technology is psychology. Architects have to synthesize all this, to create a vision of a solution. People must get something practical achieved."

Arnon's vision, as with most, sees architecture as an abstract/higher level of engineering and design. The overreaching theme is technology and purpose. In reality it is the inclusion of business and people. And business and people are left out of Arnon's final definition for software architecture.

I cannot count the number of interviews I have had with so-called "Architects". One of the first questions I always have is about the business model, and how the application is integrated into the business model. Many don't have a clue about the business model or the application's ROI. For instances, how is the quality ROI determined?

Architecture does not end upfront, but continues throughout the construction lifecycle.

The reason for an architect is that requirements analysis and design do NOT fully solve the problem.

I don't agree that I left out people. I did talk about the "system quality attributes" but I probably should have explained better what quality attributes are.

Quality attributes are non-functional requirments that are derived from stakeholder's needs.

The most common quality attributes used as input for architectural design are those seen from the end-user's perspective:

• Performance
• Availability
• Usability
• Modifiability
• Security

However, business stakeholders will probably be more interested in things like:

• Time to market
• Cost vs. Benefit
• Interoperability (e.g with legacy systems)
• Projected life time
• ROI
• etc.

Developers concerns are again probably a little different; for example:

• Maintainability
• Portability
• Reusability
• Testabilty

There are many stakeholders to the project and we have to balance all these sometimes conflicting attributes. ISO/IEC9126-1:2001, a standard for the evaluation the quality of software, provides a hefty list of quality attributes to draw from few examples include:

• Functionality. A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs.
  • Interoperability
  • Compliance
  • Security
• Reliability. A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time.
  • Recoverability
  • Fault Tolerance
• Maintainability. A set of attributes that bear on the effort needed to make specified modifications.
  • Stability
  • Changeability
  • Testability
• Portability. A set of attributes that bear on the ability of software to be transferred from one environment to another.
  • Installability
  • Replaceability
  • Adaptability

If we go back to Malcolm's comment then yes, the architecture is an "as an abstract/higher level of engineering and design"; that supports/allows several quality attributes (assuming the architect did a good job) to be met. That is because the architecture is the "blueprint". The architect's role on the other hand is in fact to work with the different stakeholders and balance thier relative needs including business aspects (which,as mentioned above, I find helpful to express as system's quality attributes).

P.S.

Malcolm and others, I apologize for the time it takes me a lot of time to react to comments and queries. Unfortunately, I don't get notified when commens are left on the site. I do try to scan all the posts, but as the number of posts get higher it gets harder and harder. Dr. Dobb's technical staff is looking into this problem, but meanwhile, if you want to make sure I don't miss your comment also drop me an email on arnon@rgoarchitects.com ...

Posted by Arnon Rotem-Gal-Oz at 08:54 AM  Permalink |

In a comment to my previous post on Architecture vs. Design, Yoni said:

It seems you are categorizing technical issues as architecture and logical issues as design. I think Martin Fowler's definition of "Making sure important things remain decoupled and easy to change" transverses both categories and is easier to follow.

I have a few things to say about this.

First, I don't categorize technical things as "architectural" and logical ones as "design." What I do say is both "architecture" and "design" are types of design where with one you focus on the wider aspects of the solution and quality attributes, while with the other you focus on local and functional aspects.

I don't see how the definition Yoni brings up is a better way to distinguish between the two? Who is to say what is important and what is not? Isn't decoupling an important trait at all level including so called "detailed design" level (e.g. utilizing dependency injection at the class level will give you better testability). Moreover, decoupling is important but sometimes you need to trade that to be able to satisfy a more prioritized quality attribute (if you want to meet a projec's quality, budget and schedule targets); see my definition of what software architecture is.

Another thing is that it doesn't matter that much where the line between architecture and design passes. The distinction between architecture and design is a semantic one that reminds us that the design of a system needs to be done in several levels of abstraction (provided the system is not too trivial). We need to abstract certain aspects of the system in order to be able to grasp the big picture. You cannot (well, I can't anyway) think at a 100 man-years project. You can only think about it at the class level and understand how everything will work together. Again, architecture is there to remind us to focus on a level of abstraction that lets you deal with non-local decisions and to make sure quality attributes are met if we cross the line to design. No biggie.

Posted by Arnon Rotem-Gal-Oz at 07:42 AM  Permalink |

There's a lot of discussion about the hard skills software architects need to have; for example, see one example at the Software Engineering Institute (SEI).

Also, here's another example by Rob Daigneau. Architects need to be familiar with a wide range of technologies, methodologies, understand the software lifecycle, have design experience, and some say an architect must write code, and so on. Hard skills are important--very important--but it doesn't stop there. There are also several soft skills that (good) architects should have.

The following is my current list of soft skills an architect should strive to have:

• Leadership. Influencing others to accomplish tasks and following your guidance
• System thinking. Understand decisions and constrains in the wide scope pertaining to whole of the solution at hand. This includes the ability to abstract problems.
• Strategic thinking. Understanding decisions and constrains and their alighments to the overall business of the company.
• Organizational politics. Understand the environment you operate in and how it influences you.
• Communications. Making sure you get your point across.
• Human relations. Understand the "people" aspects or human factors and dynamics. This includes things like negotiation, pragmatism etc

On the next posts on this subject I expand on each of these skills and explain why I think they are important for software architects.

I'd also love to hear if you think that there are other soft skills needed that I am missing (or if you think I am completely wrong :) )

Posted by Arnon Rotem-Gal-Oz at 07:47 AM  Permalink |

I just saw (via K. Bhagvan's "Architect's Corner") that Naked Objects Version 2.0 has been officially released and a Version 3.0 is under development.

Naked objects, the brainchild of Richard Pawson, is an approach to building software which is based on domain modeling and automatic generation everything else (i.e. persistence and user interface) so that once you finish with the business logic you have all you application running (and less maintenance to worry about). I think the approach can be beneficial when you do domain-driven design as you can get to play with or demonstrate the domain object as you design them. This will especially be true with Version 3.0 (which will use POJOs).

I would be wary of releasing a finalized solution that is solely based on naked objects as it also has several limitation (e.g. the UI is, obviously, very limited), but there are situations where it can be made to work. I didn't have a chance to work or review a lot of architectures based on naked objects. One architecture I did review (about a year an a half ago) was for the Irish Department Of Social and Family Affairs and found few risks in regard to the O/R mapping, heavy use of reflection, scalability, and security. It seems that recently the application has been successfully deployed (which prompted release 2.0 of naked objects), but not without problems -- or as the naked object site states:

Based on this highly successful deployment of a mission critical application, .... However, it must also be stated that the project involved a substantial effort of systems integration to link Naked Objects 2.0 to the other elements of their distributed architecture, and to the database in particular.

To sum things up, naked objects in an approach which is worth knowing about and it can save you a lot of time if its ground rules fits your application needs or you decide to use it to help model you domain model. In most cases I wouldn't use naked objects as the base for a released product. I guess the most disturbing thing for me is thatI believe that UI entities and business entities should be two separate things (maybe I'll write more about this in another post)

Posted by Arnon Rotem-Gal-Oz at 08:55 AM  Permalink |

Udi and Eric, a couple of my fellow Dr. Dobb's bloggers, exchanged comments on the topic of "reliable messaging" and "transactions" over at Ud's blog, so I thought I might as well jump in too.

Regarding reliable messaging: Eric commented that reliable messages should always be used, although Udi thought that it isn't always the case. I would add that another reason for not using reliable messages everywhere (besides cyclic messages, as mentioned by Udi) is inter-business communications or even the lack of homogeneous infrastructure within an organization. You can't always count on the other side to support reliable messaging and if you are using web services it is even harder (WS-ReliableMessaging is more like TCP vs. UDP and not store-and-forward). Nevertheless, you'd probably want to encourage idempotent messages in either case.

Regarding transactions: Udi said that "transactional behavior is already defined by the recoverability description [of the message ARGO]. A recoverable message should be handled within a transaction". I think you should be careful with this since if the transport you worked with is not transactable, your reading transaction can rollback. However, the message--reliable as it is may be--would be gone. In this case, you have to becareful about making the message transaction like by "bringing your own transaction" (i.e., compensation logic) or work with a transactable resource.

Posted by Arnon Rotem-Gal-Oz at 05:02 AM  Permalink |

Since some requirements are bound to change over time, Big Design Up Front (BDUF) can end up being a waste of time.

Another problem with BDUF is that it can be overly complicated (mentioned in my "7 Deadly Sins of Design") and/or "goldplating". The balancing force on the road to Just Enough Design Up Front is the "You Ain't Gonna Need It" (YAGNI) principle.

I have to admit my take on YAGNI is a little different from the eXtreme Programming principle:

Always implement things when you actually need them, never when you just foresee that you need them.

I think that there are things that you know you going to need and that are hard to add later on (for example, if you design object for local use and then later you suddenly decide they should reside on separate tiers). However, these thing are relatively few and it is well worth your time thinking whether you know you going to need something or you just foresee it and if the thing being designed (or coded) is something that would be easy to add later or not. When something can be added later just as easily it is better to postpone it unless it is needed right now.

*I'm wondering if I'll manage to blog about all the acronyms before I retire :-)

Posted by Arnon Rotem-Gal-Oz at 05:33 AM  Permalink |

When Dr. Dobb's offered me the opportunity me to blog here, we set its scope to include both architecture and design. While I've been covering both themes, I thought that since it was way back in April that I provided a definition for "architecture", it is about time I also define difference between architecture and design.

Two distinct criteria that comes to mind are:

• Design deals with local decisions, where architecture is broader. For instance, you "design" the interfaces for your classes, but you "architect" the division into tiers.
• Design is mostly about the functional requirements, while architecture is mostly about quality attributes. You design how a specific workflow will fulfill a certain use case, but you architect the solution to the system's availability.

It is probably quite evident that this distinction only provides blurry borders between architecture and design; for example, when you have a multi-tier solution and you "architect" the UI and say it will implement MVP pattern. Can this be considered local decision and thus design or is this the overall decision (for the UI) and thus architecture?

The way I see it the exact cross-point from architecture to design is not that important. The point in talking about two distinct activities in the development process is to maintain separation of concern. you need to handle both to make sure a solution will actually work whether you do a little design while architecting or do a little architecture while designing really doesn't matter. Also architects should be involved in both activities anyway.

Posted by Arnon Rotem-Gal-Oz at 08:02 AM  Permalink |

Well, it seems Richard Monson-Haefel* thinks just that.

In an article published at SearchWebServices.com, both Richard and Zapthink's Jason Bloomberg opined that the Java Enterprise Edition will go the way of the dodo within five years. It is important to note that they talked about the JEE platform and not Java itself.

The two main points they make are:

• JEE is complex
• JEE is not a good fit for SOA.

Let's examine both these claims.

First JEE is complex. Well of course it is. JEE helps solve complex problems. Enterprise systems (and pardon me for the cliche) pose all sorts of complex problem related to security, managability, data interaction, scalability, availability . It should be noted that not all enterprise applications are created equal. In fact I believe many so called "enterprise applications" don't need to solve all the aforementioned requirements--and those applications can probably be better served with simpler solutions (picocontainers, Ruby On Rails and few other choices comes to mind). Nevertheless there are those systems that do need to solve the more complex problems and they can benefit from a fully featured application server offered by JEE (esp. now, with EJB 3.0 which seems to be taking the right direction).

The other point regarding SOA seems to me completly bogus. The SOA style has to do with how services interact, the separation of the service edge from the service business, reliance on policy, granularity of components, and so on. It is true that "SOA and Web services diminished the importance of what you have running on the backend" but this is only true for what the world external to the service see. You still have to go an implement that damn service somehow and EJBs is a fine, vialble option to do that (the service implementation).

While I truly hope that in five years JEE will not look the same as it does today. All platforms should evolve. However, I certainly don't think we will see it disappearing or diminishing in importance to the level CORBA did.

PS: You may also want to look at the lively discussion @ TheServerSide.com that ensued after the article was published.

Posted by Arnon Rotem-Gal-Oz at 08:31 AM  Permalink |

Don't invest too much in UI without validating it against actual users. More often than not users don't understand what they really want before they get to play with a system. You don't need to validate each and every screen that the system will have. Far from it. What is important is to set the system's "UI Experience."

By "UI Experience" I mean the way users interact with the system as a whole (idioms used, keyboard shortcuts for yes/no items, is the UI built for experienced users or for occasional users?, and so on).

Craig Bailey left a comment on my blogpost on refactoring, offering "maxfactoring" for UI refactoring. While this is an interesting idea, my experience shows it is easier said than done and there's a lot of effort changing UI overall behavior (i.e., more than just simple cosmetics) after you have many forms/pages up and running.

The other option is to create a UI prototype up front. A UI prototype means you code the bare minimum that is needed to give the users the user experience intended. You can also use whiteboard/paper prototypes as an to help make the UI prototype focused on promising directions also it is worthwhile to look at tools like EasyPrototype which lets you take your paper prototypes to another level (take a look and see).

Another thing that can help with easing UI change pains is applying patterns like Model-View-Presenter (MVP) or (the older) Model-View-Controller (MVC). The idea behind these patterns is to remove all code (except the one that has to do with formatting the view itself ) away from the view (the form/web page). If you augment this with making the Presenter only depend on an interface of the view you can also greatly increase the testability of the UI (as you can mock the views) and make the UI changes easier (due to the decreased dependency).

Thus, the UI experience, just like the system's quality attributes mentioned in the previous post, is another area where it is important and worthwhile to spend some design up front to help build a higher quality system--and do it faster (overall).

Posted by Arnon Rotem-Gal-Oz at 08:24 AM  Permalink |

Agilists tend to down play traditionalists reliance on Big Design Up Front (BDUF) and they are right. More often than not, Big Designs without underlying code that demonstrate it actually works and integrates well prove to be a waste of time.

However, that does not mean not to do any design up front--as indeed several people have articulated this (see, for example, Lidor Wyssocky, Keshav, Robert Watkins, Jonathan Kohl and others).

Now some projects allow for no design up front (for example, see Bowling example by Robert C. Martin and Bob Koss). However, I think many projects will greatly benefit from some design up front. How much design you ask? Ah, that's easy--just enough. Hence, Just Enough Design Up Front (JEDUF).

Of course, at the application level it depends. If you are building a safety critical DO-178B certified project, that can mean quite alot. Nevertheless, normal IT projects don't need that much up front design. You should probably spend some time thinking about the system's quality attributes. Analyse them until you can think about them as scenarios in your application; for instance, a perfomance requirement can be written as "Under normal operation, perform a database transaction in 100 miliseconds" or "Under normal or stress conditions, a critical alert generated by the system will be displayed to the user in less than 1 second." The nice thing about expressing quality attributes as scenarios is that it is relatively easy to code a proof-of-concept (or an XP Spike) to check options for actually fulfilling them. The Up Front design occurs when you try to prioritize, integrate, and balance the (sometimes conflicting) requirments of multiple quality attributes to create an integrated architecture that fits your project.

When it comes to class level design, I agree with Lidor Wyssocky. I think you should be pragmatic and apply common sense (well, if you are just starting out with TDD, you probably want to be a little more dogmatic about it so that you would aquire the habit). Also if you solved a similar probelm in the previous iteration or project you can apply a similar solution. The same can be true at the package or module levels.

The point is that is value in design and there is no need to tackle every problem as if it is the first time you see it. You can save time by relying on your past experience.

By the way, this example also shows why a PDOM (problem domain object model) is problematic when it comes to modeling the solution space--but that's another story.

Posted by Arnon Rotem-Gal-Oz at 06:42 AM  Permalink |

I was talking with Udi the other day about how a GIS library that was used on a project we were working on together should have been designed as a framework.

This brought about two interesting questions:

• What is the difference between a library and a framework?
  
• Why would you prefer one over the other?
  

Libraries are the older concept and these are just collection of utility classes/methods your code calsl upon to get some functionality. Frameworks, on the other hand, contain some functionality or flow and calls your code to extend or make the flow a specific one. The principle of frameworks calling your code is known as "Inversion of Control."

There are several mechanisms you can use to provide Inversion of Control including:

• Subclassing
• Dependency Injection
• Template Methods
• Closures
• etc.

While frameworks are newer than libraries, they are nonetheless not very new themselves; see, for example, the 1988 paper "Designing Reusable Classes", by Ralph E. Johnson and Brian Foote. (Granted, the examples they give are outdated, but it still provides a very interesting reading.)

Why use a framework instead of a library? For one thing, it is a matter of visousity. When you have a library you need to understand the functionality of each method and it is relativly hard to create complex interactions as you need to invoke many methods to get to the result. Frameworks, on the other hand, contain the basic flow and since you only need to plug in your behavior it is easier to do the right thing. In the GIS example that prompted this discussion, it would be better to have a framework since there a hundreds of interfaces you can use. However, what most users want is an easy way to make a UI entity appear on a map.

The other advantages of frameworks over libraries is flexibility and extensibility. By definition, a framework provides you with the necessary means to extend its behavior. Many times you can even subclass the framework classes and provide completely new functionality.

The disadvantage of frameworks is that temptation to add more and more functionality creates a lot of bloated frameworks which results in immobility and needless complexity.

While frameworks have been over-hyped and "buzzword-ified" (like many other concepts), packing functionality as a framework is still something you should consider when you identify some piece of code as reusable.

Posted by Arnon Rotem-Gal-Oz at 10:43 AM  Permalink |

Yesterday I attended an interesting presentation on SOA by Dr. Donald F. Ferguson, chief architect for IBM's software group.

I was happy to hear him validate some of my thoughts on SOA (e.g., workflows are better kept inside services rather then outside, transaction boundaries should be inside a service, and so on), and introduced a couple of things I didn't know much about (for example, OSGi, a SOA platform for networked devices that's not based on web services. He also presented some nice insights (for instance, looking at the middleware as an infrastructure service and thus nicely unifying SOA and EDA)

On of the insights Donald presented was the use of heuristics as an aid to modeling and validating architectures. Some of the heuristics he mentioned include:

• Occam's Razor -- avoid needless repetition


• Don't create something new if you can compose existing stuff to get the same result


• externalize volatility -- don't put in the code things that are likely to change


• Focus on "name,value" programming not "offset programming" -- make things easy to understand


• Different is hard

If you interested in using heuristics as an architect tool you may want to look at " The Art of Sysytem Architecting", by Mark Maier and Eberhardt Rechtin. The book discusses the architectures of different system types (collaborative, IT, Manufacacturing, etc.) and provides heuristics for each of these systems.

Heuristics are a good tool to use when you design an architecture and in a way the different design principles (e.g., the single responsibility principle) can also be considered heuristics. Nevertheless it is very important to verify designs by additional methods like code and formal evaluation and not rely on heurisitcs as the only tool.

Posted by Arnon Rotem-Gal-Oz at 06:22 AM  Permalink |

Whether they are on scrapes of paper, whiteboards, or formal and elaborate documents, architectures and designs need to be communicated if they are to be of any value. After all, if you fail to explain what you want, how can you expect anyone to approve or follow your directives?

While designs are usually explained verbally, in most cases the verbal explanation is accompanied by some sort of written form. There are many options to document a design:

• UML diagrams. UML offers 13 different diagrams to document both static and dynamic behavior of components. The nice thing about UML is that it is relatively wide-spread and there's a good chance that a technical audience will understand what you want and that UML diagrams also give a professional aura when presented to managers (and they might even make sense to them if they are technical savvy).

• DSDs. Domain specific diagrams. These are diagrams that are specific for a domain few examples include using BPMNor BPEL for busiess process modeling or using DFDs for threat modeling, and so on.

• Code. Probably the best way to explain detailed design, since this is where you are going anyway....
  
  
• ADL. Architecture Description Languages, sort of domain specific languages DSLs) for architecture description. ADLs have some popularity in the academic world,but they are not very popular in "real life". (You can also see an older post I made on ADLs and DSLs).

• "Non-standard" Diagrams. This is probably the most popular way to describe architecture and design since it includes anything not mentioned above. Probably the most common example is using a block diagram for showing layers. The nice feature of non-standard diagrams is that it is easy to match them to the target audieance
  
  

How much documentaiton? That depends on many factors, like the process you are using (agile vs. formal), company standards, customer requirements, how much details are needed to make the point clear etc.

And while you probably like a style or two better than the others, you need to keep in mind that you are not explaining the design to yourself and that you need to match the documentation style to the target audience.

So how do you communincate your designs?

* Cool Hand Luke, 1967

Posted by Arnon Rotem-Gal-Oz at 08:16 AM  Permalink |