Five Questions With Tom Gallagher

Tom Gallagher got into security testing in perhaps the best way imaginable: by compromising an ISP's computer system! (With their permission, he says....) He joined Microsoft out of college and has been penetration (white-hat) testing Office ever since. These days he leads Office's security test team, continues to pen test, writes security testing tools, and educates folks in all things security. You might have seen him speaking at the UNCC Fall Cyber Security Symposium a few weeks ago or at last year's Black Hat USA security conference; you might also have noticed his recent (co-authored) book Hunting Security Bugs at your local bookstore. And, if you're into browsing notices of security defects, you likely have seen his name connected to various of those reports.

Here is what Tom has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
TG: Before I was formally introduced to testing, I was trying to explore and break things. I later found out I was really testing. At an early age I took a particular liking to automated phone systems. There are a lot more options than the automated voice tells you about! Even the dial tone has many more options than simply calling a phone number. In the beginning, I randomly tried things. As I spent more time I saw the need to be organized so I could be more efficient and reproduce things more reliably. I found much of the testing very enjoyable and like solving a puzzle or playing a game.

My first programming teacher used really good test cases to validate programming assignments. He didn’t tell the class his test input values ahead of time. This forced me to spend a lot of time thinking carefully about how things might go wrong. I knew other teachers used mostly expected values as input when grading assignments. At the time it seemed my teacher was just evil, but this forced me to test my programs more thoroughly.

DDJ: What has most surprised you as you have learned about testing/in your experiences with testing?
TG: The many different areas of testing. I had no idea how much work is involved to properly test an area. Things like accessibility and international testing were not immediately apparent to me. As I learn more, there continues to be more things involved in testing.

DDJ: What is the most interesting bug you have seen?
TG: If I told you, I’d have to kill you. =) Most of the interesting bugs I’ve seen are security issues that aren’t publicly known. I shouldn’t really talk about the details.

Since I can’t share the technical details, I’ll share the story around one of the bugs. Robert Fly and I were brainstorming around a new attack which might allow us to compromise the client application we were testing. We thought our idea was decent so we started looking for places in the application that might be vulnerable. After a little bit of time, I found a place that was vulnerable and was ready to log the bug. Luckily Microsoft takes security seriously, but I thought I might have to sell this bug a little. Tell a developer they have a buffer overrun, they get it and fix it. Tell someone they are vulnerable to a new attack, sometimes you have some explaining to do. I also personally enjoy writing exploit code so I thought I would do it in this case. My bug’s repro steps were something along the lines of “Visit this URL. Notice your name and annual salary are accessed and displayed by an attacker’s code.” We were at a time in the product cycle when all bugs/fixes requires heavy review (Microsoft’s triage process). First, individuals review the bug privately and then meet to discuss the issue. This meeting typically involves a room of feature owners and senior level developers, testers, and program managers reviewing the proposed change and if it is necessary. The bug details are displayed on big projection screen. I was not present for the review meeting, but the bug was deemed a showstopper for the product. I was also told during the meeting the presenter refused to visit my URL because he didn’t want his personal information displayed to the rest of the attendees. My proof of concept code seemed to sell the importance of the issue.

Recent research around exploiting uninitialized memory bugs is also very interesting to me.

DDJ: What do you think is the most important thing for a tester to know? To do? For developers to know and do about testing?
TG: In my opinion, it is essential to understand the technologies/languages leveraged by the product being tested and study the product’s implementation. I see a lot of testing happening from the user’s perspective by only using the product as a black box. Software isn’t really a black box. Source code is a valuable resource if available. Even when no source code is available, the software product being tested can tell you a lot about how it really works. Testing while using debugging and monitoring tools like Process Explorer, a packet sniffer, an API monitor, etc. are very helpful but often overlooked. With knowledge of how the product actually works, it is easier to identify places where bugs are likely to occur.

It is helpful if developers help testers by identifying areas of concern and explaining how the code works. Some of the better developers I know work closely with testers to ensure shipping quality code. These developers also understand what the testers plan to test, the tools that are being used, weaknesses that might be present in the plan or tools, and is able to assist the testers strengthen their approach.

DDJ: What do you see as the biggest challenge for testers/the test discipline for the next five years?
TG: Security. Feature level testers understand the way the product actually works because they use the product and try so many different things. With this knowledge feature level testers are in a good position to identify security bugs. Not enough feature testers include deep security testing as part of their testing. Often security testing isn’t easy. Many times the product behaves as intended, but there is still a security issue. Many feature testers haven’t had lots of training around security testing. I worry that if this doesn’t change we will continue to have many serious security bugs in critical pieces of software.

Just so I don’t sound all doom and gloom here, the team I work on at Microsoft attempts to solve this problem by training feature testers about security testing and requires testers to security test all of their features. This is also the focus of the book entitled Hunting Security Bugs from Microsoft Press.

[See my Table Of Contents post for more details about this interview series.]

Posted by The Braidy Tester at 07:30 AM  Permalink