Five Questions With Jon Bach

Jon Bach is Manager for Corporate Intellect at Quardev, a software testing and technical communications consulting firm in Seattle, Washington. I first met Jon at STAR East last year, where he tested an application before a live audience of hundreds of testers while his brother James provided running color commentary. Jon is president of this year’s Conference for the Association for Software Testing, and he runs Seattle's own QASIG which discusses all things testing. He'll be presenting at STAR East again this year, this time talking about the Session-Based Exploratory Testing method he and his brother invented some years ago. Here is what Jon has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
JB: It was an entry-level data processing job entering commercial real estate listings for an online database, but I was always running into bugs. There were only 8 people in the company, and no testers. I reported what I found to the developer, who seemed to appreciate that. That felt good. What also felt good was when he said it saved him a lot of headaches because he was tired of the tech support guy forwarding him all of the calls, when instead he could focus on fixing things before the customers ever found them. Call it my Myers-Briggs type (ESFJ), but I liked being of service. I also realized I was in service to the customers who might never need to call tech support if I found all of the bugs! It became a treasure hunt, a gauntlet thrown at my feet to see how many problems I could find. In my mind, I wasn’t making software better, I was saving lives! To this day (12 years later), it still feels that way – like I’m a secret service agent. The fact that the software seems to get better when my bugs are fixed is just icing on the cake.

DDJ: What has most surprised you as you have learned about testing/in your experiences with testing?
JB: What surprises me is that the people I respect most in this business – the people with years and years of varied project experience, speaking at international conferences, writing a million articles – they all bring something new to the table every year. You’d think after a certain point, they would know all there is to know and would become complacent or boring or pompous. I know many “experts” do go that route, but the people I respect always want to contribute something new and of value. That shouldn’t surprise me, but it always does. Maybe I’m surprised because these experts are so much like me? Could it be that they, too, are still getting over that “impostor syndrome” ever since their first testing job? My first official testing job (where I was actually hired as a tester, not a data processor) was with Microsoft, where suddenly I was working next to people with CS degrees. The fact that Microsoft took a chance on me in light of what they had to pick from was a surprise in itself. But I found good bugs for them, and they didn’t fire me, so I knew I must have been doing something right. What they called my “energy and enthusiasm” may actually just have been an eagerness to show them the different ways I could be qualified for the job I already had – to not let them down. Surprisingly, the people I respect most in this business seem to have the same attitude.

DDJ: What is the most interesting bug you have seen?
JB: I have to confess that in all the projects I’ve been on, the most interesting one is something I saw at a testing workshop, courtesy of instructor Grigori Melnik from the University of Calgary. You can try it right now. Run notepad.exe and type "this app can break". Save it as test.txt, close it, then open it again. You might see that it now shows all rectangles, as if the font couldn’t be loaded. Ironically, it was part of Grigori’s talk titled “Test-Infecting Future Engineers” to a group of 20 seasoned testing instructors invited to the Workshop on the Teaching of Software Testing and it wound up infecting all of us with curiosity and energy. My brother James seemed to be the most “infected” and decided to take on the challenge of investigating this behavior with Grigori looking on. He recorded his process in a 20-minute video during lunch that day. The cool part for me was that unlike everyone else who got rectangles when they opened the file, I got what looked like Simplified Chinese characters! One of the people in the workshop happened to be Chinese, so I asked him what it said. He said “it says garbage, nothing”. That was a big clue that inspired my own investigation, but James took this to a deeper level. It inspired me to think, how much of the day should testers spend investigating cool bugs? Could it be a waste of time when a programmer already has enough information?

DDJ: How would you describe your testing philosophy?
JB: As a service provider (working in an outsource lab), I have to find things that are of more value to my clients than my competitors can. I learned that from Robert Sabourin at Amibug, who works very hard to instill it in his students at McGill University and demonstrate it in his consulting. But value-driven testing is not just a business philosophy, it’s also a personal ethic. I’m self-critical and the projects I’m on are usually short and have small budgets, so that combination of factors leads me to have the mantras: “is this test valuable”, “is my strategy likely to reveal something valuable?” and “am *I* valuable”? It starts with a healthy sense of paranoia and a desire to learn more than just what the mission is for the projects I’m given. What about the Bigger-Picture Mission, for example? Once I know the mission, sometimes it’s useful to ask, “why is that my mission?” to begin to see a larger system in place that may impact my testing approach and techniques. Seeing systems is the most important philosophical thing I’ve learned in testing. I have my brother to thank for that – mainly for introducing me to Peter Senge’s book – The Fifth Discipline. Senge didn’t call this out by name, but his book was the first time I learned the difference between recognizing faults and recognizing failures. If you don’t consider system interactions or consider the software as a whole system of features, you may end up reporting failures, not their underlying causes (faults). It’s not the end of the world if you don’t find the fault – sometimes there’s not enough time -- but I prefer to build up my credibility bank account as soon as I can with the people to which I’m in service by investigating as much as I can into why the bug happened. The contract I want with them is: “don’t waste my time having me test things you already know are broken, and I won’t waste your time giving you bugs that are shallow.” Since I’m interested in quickly winning autonomy and trust with my clients, knowing the difference between fault and failure is one huge way to do that (and to me, it’s the most apparent skill that differentiates novice testers from veteran ones).

DDJ: What do you think is the most important thing for a tester to know? To do? For developers to know and do about testing?
JB: Let me answer this like I did the other day when speaking to college seniors: “Be cautious, critical, curious.” Technology can be taught much easier (in my experience) than how to make someone curious and engaged, yet have a healthy sense of self-critique. If they are curious and engaged, you can teach them all kinds of ways to harness that. I also say “take a risk and ask a question.” This is something I see the most seasoned testing experts do – well, at least the people *I* call experts. It takes practice (and courage at first) to ask questions, because when you ask a question, you’re making yourself vulnerable. You’re admitting you don’t know something and you risk someone thinking less of you, not more. That’s why there are a lot of “experts” out there that never seem to ask a single honest question. I assume that’s because they think they can’t afford to have people think less of them. But the people I respect most not only ask questions, they ask all *types* of questions, inspiring me to look just as hard at where answers may be hiding as where bugs may be hiding. After all, a test is just a question in disguise, and tests are my business. Now on the dev side, they should know that testers are trying hard to be of service. When testers celebrate a bug, they are not celebrating a developer’s mistakes, they’re celebrating the fact that the software did not fool them this time. This is an important celebration because the software so often fools testers into thinking it is working most of the time! All said and done, I think programmers should test for a day, and testers should program something at least once in their careers.

Posted by The Braidy Tester at 07:30 AM  Permalink