Five Questions With Kevin Beto

Kevin Beto was, once upon a time, a professional wrestler. And a cheerleader. Not at the same time, he claims. Seven years ago he traded bashing other people for bashing applications at Microsoft. Which is probably a good thing, as he is currently a test lead on Microsoft Office PerformancePoint, and bashing heads is generally frowned upon by all the best people managers today.

Here is what Kevin has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
KB: My first introduction to professional testing was at Microsoft. I had a CS degree and only had development experience, and didn’t realize I was interviewing for a test internship here at Microsoft until half way through the interview when I absolutely got my butt kicked by the “Test a toaster” question. I thought it was the most boring thing to do in the world when given a handful of manual test cases to run, until my mentor introduced me to a tool called “Genesis” that did genetic algorithm testing. I spent only a week implementing that tool for a zip code conversion library that we had and found all sorts of really juicy bugs. It would just run overnight and find bugs which would be sitting on my computer in the morning for me to pump into our bug reporting database. That is really what changed my opinion of testing from people who just click buttons until something breaks to a very challenging engineering discipline.

Ever since then, all I think about is how I can do things more innovatively or efficiently. Now I love the manual stuff too because I see so much opportunity in finding ways to make that faster and easier.

DDJ: What has most surprised you as you have learned about testing/in your experiences with testing?
KB: I was surprised how complex and exciting testing can be. I continually run into fresh challenges that stretch me, and always find some new way for me to grow myself professionally.

DDJ: What is the most interesting bug you have seen?
KB: We ran into a really interesting threading issue about a year ago. Our automated BVTs suddenly started randomly failing in the middle of execution. It was never the same test case, but it was always the same error. After about a month of investigation, arguing with development, proving older builds worked, and newsgroup postings, we were able to track down the issue to a product bug. Turns out the product was creating a whole bunch of threads, which overflowed the thread pool and exposed a really nasty synchronization issue if you had the right amount of data in your system. This of course would repro on the multiprocessor lab machines but never on the single processor development machines.

DDJ: How would you describe your testing philosophy?
KB: I really like to plan for the long term when doing testing. I am willing to cut short term goals down in size if I know that one or two releases from now I’ll be able to accomplish orders of magnitude more than I am doing today. I invest heavily in automation and long term projects (model based testing, etc.) that will allow us to at least keep, if not exceed, pace as the scope of the product grows without needing to add headcount.

DDJ: What do you think is the most important thing for a tester to know? To do? For developers to know and do about testing?
KB: That we’re all on the same team and working toward the same goal. It is very easy to draw a battle line and line up on opposing sides and make everything a contentious issue, but the really high performing teams I’ve been on had an unprecedented amount of cooperation between development and testing. Once that last feature is checked in, everyone should be looking for bugs. Test should be focusing on finding the new issues, and dev should be focusing on finding places where known bugs have close relatives in other parts of the code.

DDJ: Is there something which is typically emphasized as important regarding testing that you think can be ignored, is unimportant?
KB: Finding every last little bug, especially on new products. At some point, you need to label the quality as “good enough” and get it out in front of customers. You may ship buggy software, but given how agile we can be with the advent of ubiquitous connectivity and the internet it is easy to make patches to fix almost anything that makes it past the test team and gets found by a customer. Having that customer feedback to improve and adjust your testing practices and coverage in my opinion is much more efficient than trying to come up with it on your own.

DDJ: What do you see as the biggest challenge for testers/the test discipline for the next five years?
KB: With the introduction of managed code that enables rapid development practices, and testability taking a front seat in many platforms (IE, .NET Framework, Avalon, etc.), I believe we’re right at the brink of a huge wave of technical innovation in the testing world with both the tools and techniques we use to do our job. The challenge is going to be collaborating/leveraging existing solutions that are out there and avoiding the sometimes irrepressible urge to reinvent the wheel.

[See my Table Of Contents post for more details about this interview series.]

Posted by The Braidy Tester at 07:30 AM  Permalink