THE BOOK OF TESTING

Thoughts From a Braidy Tester

November 2007

Adam Goucher (not the runner) has been testing for nearly ten years, working for banks and startups and most everything else. After work he teaches testing at his local technical school and writes book reviews for DDJ. Somewhere in there he finds time to spend with his family and many animals, including two horses (which Adam wishes to make clear do not live at his house along with the rest of the menagerie).

Here is what Adam has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
AG: Back in the late 80s I had a subscription to Nintendo Power magazine. In one issue they showed the offices of the team which tested the games. Like most people involved in public school, I thought that would be cool to play video games for a living – but then promptly forgot it. Fast forward to the fall of 1998 and Y2K was just around the corner when I got a job helping to test and manage testing COTS software for a major Canadian bank. That experience lead to another position there, which then got the snowball rolling down the hill to where I am now. Along the way I started to follow the work of people like Cem Kaner and James Bach which thrust me out of the ‘click to break’ mould and into the thinking aspects of testing.

DDJ:How would you describe your testing philosophy?
AG: I once considered starting a company called ‘Quality Chefs’ which would help organization ‘bake in quality’ to their development process. While the idea didn’t go anywhere, the philosophy remains the same; Quality has to be built into an application. I’m a big fan of Continuous Integration and other additions to the development process which allow you to do a lot of your verification and validation up front and in an automated fashion. That leaves us, the testers, to zoom around the application’s riskiest parts and/or those which could not be easily or efficiently automated (to paraphrase Chris McMahon). In addition to the actual testing, it is the role of QA/Test group to identify, advocate and encourage as many of these tools and techniques as possible. Long gone should be the days of getting code thrown over the wall in an unknown state and execute binders of long winded, step-by-step test plans by hoards of unskilled testers.

DDJ: What do you think is the most important thing for a tester to know? To do? For developers to know and do about testing?
AG: From an ‘art of testing’ perspective, there are a number of things I recommend to people starting out on a career in testing (for more general ones, see the final question). In no particular order, here is a handful:

• Learn to read code – There is a big difference between being able to create a program out of thin air and understanding it, much like how most of us could not write a Shakespearian play we could be able slog through reading it. You are artificially limiting the amount of information you can be using as inspiration for testing ideas if you cannot determine what you are being handed. Some companies, most notably Microsoft and Google put high value on a tester’s ability to produce code, but for most companies being able to read it is sufficient. Languages like Python and Ruby are great languages to learn the fundamentals in.


• Learn Unix – Very few testers will go their entire career without having to test something that uses Unix as some component. Testers need not have admin level knowledge, but the ability to navigate the file system, edit files and basically survive is an essential skill in my books.


• Learn a bit of SQL – Too often testers are lured into database courses where they learn about managing table spaces and how to configure the data dictionary. Much like with Unix, testers generally do not have to be DBAs so learning how to find and manipulate information inside the database is more important than how to maintain it.


• You do not have to speak or read another language to do internationalization (i18n) or localization (l10n) testing. But if you do speak or read another one, make sure it is in an obvious spot on your resume.


• When learning a new tool, try to learn the philosophies and processes that the tool is assisting you with, not just the tool. Too often I have seen people market themselves as Automation Specialist when they are really Tool X Specialists. If they find themselves faced with Tool Y they are hopelessly lost. The best testers have some tools they are more proficient in than others, but because they know the theory behind their work the tool is just a means to the end, not the means to the end.


• Take James Bach and Michael Bolton’s Rapid Software Testing course if at all possible. I took it largely because my employer at the time was going to send me to California to take it, but it ended up providing a tonne of ‘ah-ha’ moments and likely should be considered a catalyst for where I am now in my growth as a tester. I know a number of people that have had the same reaction.

DDJ: Is there something which is typically emphasized as important regarding testing that you think can be ignored, is unimportant?
AG: I am so thoroughly sick of The Great Certification Debate. Are they good? Are they bad? If you think they are good, which one do you get certified in? Too much time and emotion has been invested towards this issue by some of the more recognized people in the industry. If there was a chance that the major argument holders could be swayed towards a pan-community agreement I could see there being some value to continuing the discussion, but I don’t see those involved ever reaching such. Let’s just let market forces decide. If the market as a whole overwhelmingly says you need Certification X, go write the test and be done with it. At this year’s CAST there was an entire day devoted to debating this subject and there was barely more people in the audience than on the panel, so I think the silent majority might also be sick of this.

DDJ: Is there anything else you would like to say?
AG: Start a blog - One thing that impressed me when I met James Bach for the first time and later observed him on mailing lists, etc. was his ability to argue his points. It was clear that his arguments were well thought out and from many different angles. The process of getting your thoughts out in writing helps clarify “what you think” and “why you think” it. Blogs are the ideal format for this kind of discussion. And don’t be discouraged if you do not get massive amounts of readers, the action of doing it is more important than the reach.

Coach sports – The skills you need to coach a house league sports team are the same as you need to have to lead a test team. At the younger age brackets you do not even need to know the sport itself. Not only will you improve the soft skills that become more and more necessary as your career progresses, but you can make a kid’s week.

[See my Table Of Contents post for more details about this interview series.]

Posted by The Braidy Tester at 07:30 AM  Permalink |

Alvaro Peon-Sanchez is a self-described end-to-end tester. Microsoft found him during his computer systems engineering studies at Mexico's Tec de Monterrey. After an internship testing PowerPoint addins Alvaro hired on full-time. He spent a stint on a product call information bridge framework, moved to a project which was ultimately canceled, and now works on Office PerformancePoint Server.

Here is what Alvaro has to say:

DDJ: What was your first introduction to testing?
AP: I’ve always been intrigued by how things work. As a kid and teen I wanted to be a chemist so I would spend a lot of time running tests on things. But my first real life introduction to software testing was when I got an internship at Microsoft. Before that my friends that had had internships there explained me the process of how “real” testing works.

DDJ: What did that leave you thinking about the act and/or concept of testing?
AP: Testing left me thinking of how much power testers have (and at the same time don’t) for controlling the quality of the product. Also it left me with a great feeling of getting to know everybody on the product, which you might not get being a developer.

DDJ: What is the most interesting bug you have seen?
AP: Better not talk about this in public :)

DDJ: How would you describe your testing philosophy?
AP: Code doesn’t work until proven otherwise. It makes me really mad when I see a product that doesn’t meet a certain bar. At the same time I keep in mind two things: there will always be bugs, and change happens, so be ready for both.

DDJ: What do you think is the most important thing for a tester to know?
AP: 1) Know what need your product solves. 2) Know your customer. 3) Know your technology.

DDJ: For developers to know and do about testing?
AP: For developers I would inverse the order. For a tester's relationship with your developer I would highly recommend: 1) Earn their respect. 2) Be friendly to them. 3) Bring them cookies every Friday.

DDJ: Is there something which is typically emphasized as important regarding testing that you think can be ignored, is unimportant?
AP: I think that there are things that are not mentioned regarding testing that we are ignoring. We focus so much on functional testing that we can miss the big picture. For example a part of my job is to create opportunities for the team to have the big picture and make sure we test real customer scenarios. Doing this helps developers understand what they are building, testers what we should be building and designers how to improve what we are building.

DDJ: What do you see as the biggest challenge for testers/the test discipline for the next five years?
AP: I think there are two big things that will happen in the next 5 years. The first is the technology of testing: The discipline started by doing manual tests, then moved to automation. I think the next big challenge is how to balance both and make full customer scenarios automated. Also products have become extremely interdependent on other products. Getting everything to work under one single, simple to use, and robust testing environment is going to be challenging. The second: We need to move from functional testing to more customer scenarios.

[See my Table Of Contents post for more details about this interview series.]

Posted by The Braidy Tester at 07:30 AM  Permalink |