IF YOU BUILD IT

... Will they Come?

April 2008

Reminder - please update your reader to the new blog home on Dobb's CodeTalk

A while ago I asked "who tests the test":

" One question I don't hear asked too much is "who tests the tests?" - after all we are writing all this additional code - if we write so many bugs in our production code that we need tests - what are the chances the test code is clean?"

I would add that this is especially true considering that the same person that writes the code also writes its unit tests

One answer I had was making sure to keep the triad of tests->code->acceptance tests. so that each two test the third. This is especially good since, usually, the acceptance/integration tests are not written by the developer who wrote the code.

One answer I got was that tests are usually simple enough to be self evident and when tests contain complex constructs you should probably test them as well.

Basically, tests can have two types of problems

• Not testing the right thing
  
• Not testing the thing right

1. We have a component that interacts with resources in different URIs, so when resources wake up they register themselves with that component. Resources register two URIs -> one for control and one for interactions. We've added a business rule that both URIs should be on the same host (naturally a new test for that was also added but that not the point) suddenly a bunch of the old tests started failing. Sure enough, I took a look at the new code, found a bug, fixed it, but the tests were still in the  red. In the end it turned out one of the Uris that is registered during the tests had an extra "L" (locallhost instead of localhost) so the tests failed with the new rule - as they should..

2. After writing a piece of code to "just work" it had to be made multi-threaded. I needed to write a test that would make sure the component can handle multiple concurrent requests. Sure, I thought, I'll just run few or even all of the other tests in parallel and be done with it. The only problem was that the other tests were written to be isolated (i.e. new instance per test) running them in parallel had no effect what-so-ever.

What can we do to help us locate problems and bugs with tests?

• remeber that Test code is code - so as you write more tests you should also refine the tests design and refactor the code accordingly. For instance in the first example above, it was easier to diagnose the problem since the faulty setup was done by a single method used by alll the failing  tests (Single Responsibility Principle @ the method level).
• Test-First - One of the reasons test-first is beneficial is because you can actually see the test fail before you implement the code to make it work. This helps you make sure that the test actually tests the behavior you want. When the code is already in place it is harder to make sure if the test works because of the change or not. In the second example running all the tests in parallel everything was still green, which triggered me to write specific tests for checking for multi-threading issues.

Posted by Arnon Rotem-Gal-Oz at 07:13 AM  Permalink |

Reminder - please update your reader to the new blog home on Dobb's CodeTalk

Someone calling himself r r left the following comment on part IV of my series of posts on SOA definition:

"I keep trying to read this series on SOA unfortunately suffers from the same disease as the rest of literature on the subject. stays general to a comfortable level so it can't really be applied anywhere, tends to complicate things where is not clear if it's needed, and encourages philosophical debate on what ultimately is a business (and so concrete) requirement. Meanwhile the serious (IMO) issues stay untouched - how does one actually approach an integration project with functionality, performance and security in mind. Which should be the standards used (considering the tens of standards on WS out there). How granular should the WS be (I'm done with answers like "not too much, but enough", or "well, depends on your project"). "

That said let's take a look at the more interesting parts of this comment. First, the thing that bothers me about this reaction is (what seems to me as) the quest for final and concrete recipes. For instance consider the comment on service granularity

"How granular should the WS be (I'm done with answers like "not too much, but enough", or "well, depends on your project"")

Nevertheless it is not a pure guess either. You can use some guidelines and measure them against your specific project/system/enterprise needs. Personally The set of guidelines I use is based on the fallacies of distributed computing :

1.  The network is reliable
2.  Latency is zero
3.  Bandwidth is infinite
4.  The network is secure
5.  Topology doesn't change
6.  There is one administrator
7.  Transport cost is zero
8.  The network is homogeneous

Regarding the other questions (how do you approach a real system), well, if you pardon me for banging my own drum, that's exactly why I started to write my experience on these matters as patterns. for instance if we look at the saga pattern (one of the patters I published online). you'd see that it is talking about achieving distributed consensus in a transaction-like manner. I talk about the problems of using distributed transaction etc., offer an architectural solution (the saga ) and then discuss relevant technology issues (e.g. WS-BusinessActivity ) as well as its implication from quality attributes perspectives (Integrity and reliability). Nevertheless even these patterns aren't an end-all solution. different circumstances require different solutions
Both my previous job and my current one involves building a scalable solution on-top of algorithmic engines. In my previous job I  managed the construction of a biometric solution that allows using multiple biometrics. In my current job I manage the development of  a mobile visual search solution . Again, while on the surface both needs to get some data, run a few  algorithms and produce an answer. These systems have very different quality attributes. On the first system we had to handle very large databases, hundreds of queries, an emphasis on modifiability and security, the current one needs millions of queries, almost no database, low latencies and emphasis on usability.  These differences result in radically different solutions, with different services, different interactions , use of different patterns etc. There's no "one right answer" (tm)

Posted by Arnon Rotem-Gal-Oz at 04:23 AM  Permalink |

While I personally don't like the Singleton pattern too much (it is essentially an OO mask for Global Variables, it makes it harder to unit test etc.), I still need to use them now and then. Anyway, I saw a post by Jack Altiere about "Using the Singleton Pattern" in .NET and since it presents an implementation that leaves a lot to be desired I decided to comment on that

For one, he presents a solution which locks every time the singleton is accessed (see below). The reason this is flawed is that singletons are instantiated once, but they are accessed a lot of times and with this implementation you pay every time you try to access the singleton

//bad implementation - don't use

//also note some initialization etc. is omitted for brevity

 public static Foo Instance

{

      get

       {

        // This lock allows thread safety.

             lock (_mutex)

             {

                   if (_instance == null)

                        _instance = new Foo();

                  return _instance;

            }

      }

   }

public  sealed class singleton

{

        public static readonly MySigleton Instance = new MySingleton()

}

But that's not all- see, one of the problems of the Singleton pattern (besides the ones mentioned above) is that it is also a violation of the Single Responsibility Principle (SRP see OO primer). You can solve this problem if you use Generics and create something like C++'s template based Singleton class:

namespace ConsoleApplication5

{

    class Singleton<T> where T : new()

    {

        private static readonly T inst = new T();


        public static T Instance

        {

            get { return inst; }

        }

    }

}



and then use that simply by doing something like Singleton<MyClass>.Instance