Five Questions With Chris McMahon

Chris McMahon lives in the deepest depths of the Four Corners boonies, were he telecommutes in to Socialtext. Telecommuting from the boonies seem to have its rewards: Chris spends many of his lunch hours tubing the Animas river.

I first heard of Chris because some time back he was working to start a local market for custom software in his small town. Although he calls himself an enthusiastic beginner, he has presented at numerous conferences and written numerous articles for Better Software. Chris told me that he is the oldest customer of Watir; I am not sure whether he means old-as-in-the-first or old-as-in-age!

Here is what Chris has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
CM: Although I went to three universities on four different occasions, I have no college degree, and I spent my 20s as a semi-professional musician. I got tired of the road and I got my first ever 40 hours/week, 8-5 job at age 32. I was hired to write technical documentation.

I (luckily) shared an office with a senior developer named Pete. I got my first assignment and asked Pete how to run the software. He gave me a funny look and said "you're the first documentation person we've ever had that wants to run the software". In the course of my writing, I had a couple of technical questions and Pete showed me how to use the debugger. I had no idea at the time how unusual that was.

So I wrote my first assignment, using the debugger to check things when I wasn't sure how they were put together under the hood. I turned it in, and the next day the VP came storming into our office really, really mad. She said "Our software doesn't do this!" I said "Yes it does, watch this." I demonstrated what I had written. The VP's jaw dropped -- her first comment was "Oh, sh**!".

Testing takes a certain knack for being the critically important person who is never in the spotlight, working to make things go smoothly and working to prevent things from going haywire. When I was a musician, I was was never a singer or a songwriter or a lead guitarist, I was a bass player. Likewise, I don't want to be a developer or an architect or a CTO, I have a disposition suited for testing.

DDJ: What is the most interesting bug you have seen?
CM: I worked at a company that did data validation for 911 systems, so when you choke on that fishbone and dial 911, they know where your house is. This is a "perfect storm" bug...

The telephone companies add and split area codes pretty frequently. The 911 systems keep numbers around for both old and new area codes for a while before deleting the obsolete ones. The company had a routine little report we ran now and then for finding area codes ready to be deleted. We had someone run the report, but it was configured improperly, and instead of reporting obsolete area codes, it reported *all* area codes.

The sysadmin responsible for deleting area codes knew something was wrong, and resisted deleting the data. And resisted, and resisted, until he was outright ordered to delete the data. No one in management reviewed the report.

When this sort of data gets deleted, a loadable backup copy of each record is routinely recorded, just in case someone makes a mistake, and records have to be restored. And that happened in this case -- except that this was on an operating system where file sizes have to be declared when files are created. When the file is full, the OS returns an error message.

But the script that made the backup copies of the 911 records had never been tested -- it was a one-off created by a sysadmin directly on the production system. No one knew that the script never checked for error messages returned from the OS. So with this immense amount of data, soon the script filled the backup file. Then the script would write the backup record, fail, delete the original, write, fail, delete the original, thus destroying all copies of the records.

We deleted most of the 911 records for a particular Midwestern state that day. We restored them (which is another story altogether) -- but it was a very, very bad situation that could have been prevented at any of four or five different points in the process.

DDJ: How would you describe your testing philosophy?
CM: I know a number of testers who resist being labeled with the term "Quality Assurance". I used to resist it myself, but recently I've been coming back around, and now I'm starting to like the term "QA". Here's why:

I recently started working for a company called Socialtext. Socialtext is a mature startup that sells wiki software to Enterprise customers. The Wired Magazine "HowTo" wiki is the coolest thing Socialtext has done recently.

Businesses compete in three arenas: features; service; and quality. Startups by necessity focus on features first, service second, and quality third. Socialtext is just at the point where we have the features in place, service is maturing, and focus is shifting to quality.

So while testing is a big part of my job, another very important part of my job is to participate in the company culture as a an advocate for quality, to provide measurements that demonstrate quality (or lack of it), and to guide the ongoing internal discourse to discussions of quality.

I guess it's more "Quality Encouragement" than "Quality Assurance", but it's still a lot more than just testing.

DDJ: What do you think is the most important thing for a tester to know?
CM: I think that there are three technical skills that testers must have to be really successful. Testers really should be in a position to address: the filesystem; the database; and the network. Although some testers are subject matter experts, and some testers are process people, even non-technical testers are immensely more powerful if they learn how to manipulate files, speak SQL, and send information around over TCP/IP. It's not very hard to learn the basics, and the value these skills bring is immense.

DDJ: Is there something which is typically emphasized as important regarding testing that you think can be ignored, is unimportant?
CM: Whether in manual tests or automated tests, I follow a very simple rule: do not test for blocking conditions. Do not test that a login succeeds; test for what happens after the login. Do not test that a page loads, test to manipulate elements in the page. Do not test for the presence of an input field; test that the field accepts input. The amount of time and money wasted on pointless tests is really shocking.

What's even more shocking is the number of people that disagree with me!

Posted by The Braidy Tester at 07:30 AM  Permalink