Five Questions With Alan Myrvold

I first met Alan Myrvold a few months ago when he emailed me a question about one of my MSDN blog posts. I looked him up in our internal address book to see who he was. "Security Senior SDET" it said. "Hmmm", I thought, "he must know his stuff." Some time later I met him in person at a meeting of our internal Test Architect Group and I learned that, indeed, he does know his stuff.

For the last two years Alan has been a security tester for Microsoft Office. The sixteen years before that he worked at several largish Canadian software companies doing testing, programming, and management. He's a tester again now because he's found he enjoys that most. Here is what Alan has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
AM: My first introduction was a summer job in 1987 helping with quality assurance, control charts and defect prevention at an injection molding plastics factory. Although it wasn’t software, It made me realize that there are many dimensions to product quality, and that even the best testers can miss important issues, like the grill we made that didn’t fit right on the car. I was always addicted to software, and in 1988 joined a software company initially working on quality assurance, including software metrics and policies and procedures for code reviews. Later I was a test automation toolsmith, creating tools for automated testing. I finally discovered the joy of testing while testing statistical software for data mining. I was strongly influenced by the book Testing Computer Software, which turned on a light and changed how I thought about testing.

DDJ: What has most surprised you as you have learned about testing/in your experiences with testing?
AM: Every time I arrived at a firm belief about software testing; that testing is all about verifying customer requirements, or that all tests should be automated, or that no tests should be automated, or that testing is all about finding bugs, or that defining expected results isn’t important, I was later surprised to realize that my belief wasn’t correct in all contexts, and was a dangerous assumption in some.

DDJ: What is the most interesting bug you have seen?
AM: There have been many. I still get very excited when I discover a cool bug. I get excited not only about bugs, but about how to best describe them clearly and accurately. I’ve been doing security testing, and some of most interesting bugs to me are subtle issues that allow cryptography to be bypassed. I found my most recent cool bug while stepping through code in a debugger, and then I just needed to create test data to make the suspect code path get hit. It was a well tested area, and a very subtle bug that allowed validation to be bypassed. I was euphoric when I found it. After logging it, I updated the repro steps and title a bunch of times to make the bug description clearer. It was late in the release, so the bug went through a formal triage. When the bug was being discussed in triage, my team lead pointed out how many times I updated the bug details, and that it must mean I felt strongly about it. The bug was approved for the release, but I had updated the repro steps yet again while they had it open, so they needed to refresh it to make the change.

DDJ: Is there something which is typically emphasized as important regarding testing that you think can be ignored, is unimportant?
AM: Code coverage tends to be overemphasized. Many cool bugs are errors of omission, not commission, or require interesting data or perceptive evaluation of the product’s behavior.

DDJ: What do you think is the most important thing for a tester to know? To do? For developers to know and do about testing?
AM: Learn about testing. When interviewing testers, I’m surprised by how few have ever read a book about testing. Testing is not yet well taught in most computer science programs, and testers need to learn how to test. Many developers don’t understand the role of testing, or toss code over the wall with obvious mistakes. Testing doesn’t ensure quality, and doesn’t find all the bugs. I’d like developers to be terrified of bugs escaping to customers. Many small software development companies produce great software without an dedicated testing group, and developers should be just as cautious when there is a test group.

DDJ: Going meta (to channel Jerry Weinberg), what else should I ask you? What would you answer?
AM: After moving here from Canada, what foods do you miss most?

DDJ: What would you answer?
AM> Timbits, poutine, and Montreal style bagels.

Posted by The Braidy Tester at 07:30 AM  Permalink