Five Questions With Johanna Rothman

How might you know Johanna Rothman? Let me count the ways. She is the author of Hiring the Best Knowledge Workers, Techies & Nerds, which teaches manager types how to find people like you and me and convince us to work for them. She teamed up with Esther Derby on Behind Closed Doors: Secrets of Great Management, which helps manager types manage people like you me - and helps us understand our managers. And most recently she published Manage It! Your Guide to Modern, Pragmatic Project Management, which helps manager types - and people like you and me - keep our projects out of the Business Blow-Ups section of our local newspapers. If blogs are more your style, you can get Hiring Technical People and Managing Product Development that way as well.

Can you tell Johanna runs a management consulting business? <g/> Also why she co-hosts AYE and helps Jerry Weinberg teach PSL?

Johanna follows the Helpful Rule in everything she does, and her books, workshops, and consulting practice all help you understand how to best do the Good Job you want to do. Here is what Johanna has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
JR: I became a tester when I went to work for a new company where I didn't know the products, didn't know the language, and didn't know how the products were applied (Symbolics, the lisp machine manufacturer). I wanted to work with the smart people who were there, and I thought Symbolics was poised for greatness. They were; it was brief greatness :-(

Since I'd been a developer for the previous 9 years, I applied what I knew about development to testing. I did these things, just as I had when I had been a developer:

• Explored the application for a short while to decide where to automate.


• Used combinatorial testing techniques.


• Learned quickly to ask about the vital few scenarios that our customers would use, so I could test those in detail, looking for the places our developers had blind spots.

I'm still not an expert in Lisp, but at one point, I was good enough to write lots of little pieces of code that we could use for tests.

What impressed the developers was how "nasty" I was. I found problems no one imagined could exist. I didn't think I was all that nasty; I was using the product the way I thought a user might :-) The most valuable thing the tester brings to his or her work is the curiosity and skepticism about how the product should/does work. Developers could never believe all I did was try to approach the application with an open mind. "Why did you do that?" "It seemed reasonable to me." "It's *not* reasonable!" And then, if the developers didn't fix the problem, sure enough, a customer *would* find exactly that problem. A number of developers were surprised by how creative testers needed to be.

DDJ: What has most surprised you as you have learned about testing/in your experiences with testing?
JR: I am constantly amazed at how easy it is to break systems that provide an interface to the general public. For example, my local grocery store upgraded their software last Friday. On Sunday, when I did my grocery shopping, I whipped out my credit card to pay, and the cashier said, "Oh, if it's a <large-bank-name>, we can't take that today." "Why not?" "The debit part of it doesn't work. You need to use a credit card."

How is this possible? How could the vendor not test (or not realize there's a problem with) the software for a common banking partner? I don't know about the cost to my local grocery store, but you can bet I'll be checking the prices of what I buy. If the vendor can't get the payment details right, how will they get the cost details right? (You and I both suspect that the price lookup is different from the payment verification, but if their vendor got one thing that wrong, why wouldn't the vendor make other mistakes?)

I don't understand how producers can continually *not* test their most common scenarios. These common scenarios aren't just the happy paths; they're the common ways people interact with the system, whether that scenario is a common mistake or a typical job the user wants to accomplish. I'm surprised when testers (and developers) don't break those scenarios down from system level tests to smaller and smaller parts of tests. Knowing that your product works in its most common form gives me a feeling of confidence about the system. And, if I can't prove that the common scenarios work, I have areas I can address in more detail.

DDJ: How would you describe your testing philosophy?
JR: I want to make sure the testers are testing the areas that provide the most value to the customer and the areas where we have the most risk. That means the testers have to work with the requirements people to understand the value to the customer. And, the testers have to work with the developers to understand the areas of risk.

JR: What do you think is the most important thing for a tester to know? To do? For developers to know and do about testing?
JR: There are two important things for testers to know: Testers need to know about the systems they are testing and how to be friends with their developers. They need to learn the common product scenarios; the implementation language(s) and how developers can be tripped up by the language; what's most valuable to the customers; how to demand automation hooks from developers; and how to use those hooks to develop small pieces of automation. Testers who ask these kinds of questions also tend to be people who use their previous knowledge of other systems and apply that knowledge to this system.

Testers need to be friends with developers, so they can eat lunch with their colleagues and ask, "What was challenging for you? What should I look at? What should I avoid?" I don't necessarily believe the "what to avoid"--I'll ask why so I can think of ways to test.

Developers need to learn how to test as they go, so they can learn how their code could break. Developers have to learn that they can't just do unit tests; they need to look at larger chunks. There's a huge gap between unit tests and system tests, and most of that gap is the developer's responsibility.

Testers, along with developers, need to continually re-examine their approaches and ask themselves, "Am I learning something new from this testing approach? If not, what do I need to?"

DDJ: Is there something which is typically emphasized as important regarding testing that you think can be ignored, is unimportant?
JR: This whole business of measuring how many defects there are by severity. Oh please. Severity is irrelevant. If you have defects in the code, it's dirty. Seeing one defect means it's ok to have more defects. If testers showed the total remaining open number of defects by week, instead of anything else by week, the trend would have a lot more benefit to the project. Developers need to learn to clean up the code, and the best way to do that is to measure the total number of remaining open defects.

Severity and priority might be useful for organizing when and what to fix, but people start playing the defect promotion/demotion game as soon as they start counting by severity. That game allows people to put blinders on, and release broken software.

Now, it's always a business decision about when and what to release, but the people who make that decision need the data to make a good decision. They can't make good decisions without trend data.

DDJ: Going meta (to channel Jerry Weinberg), what else should I ask you? What would you answer?
JR: What should project managers know about testing?

Project managers need to know which testing should be done by which people, and if those people are actually performing those tests. A PM who waits for the testers to test the product (of course, in three weeks!) is an incompetent PM. Software is a design problem, not a manufacturing problem. You gotta test the design as you go, which means no matter what lifecycle you use, you have to start testing from the beginning.

Test driven development (TDD) is really about design, and good PMs should encourage their developers to use TDD. But even if the organization is wedded to waterfall, or if they think they need an architectural phase, the testers should know enough to participate in architectural reviews, and design reviews. Testers who bring curiosity and skepticism will help the developers create a much better product.

Sometimes, asking the "dumb" questions help people realize what's going on in the product. I once asked "How does that queue work? I understand how things get into it, but I don't understand how things get out." Turns out, there was a design problem in removing things from the queue. If I hadn't learned enough about systems to know that, or if I had been uninterested in the code, I would never have asked that question.

[See my Table Of Contents post for more details about this interview series.]

Posted by The Braidy Tester at 07:30 AM  Permalink