New Architect: Mozilla has done an admirable job of standards compliance—perhaps a better job than any other browser. Why is it so difficult to build a solid, standards-compliant Web browser?
Gervase Markham: Go and read the specs for HTML, XHTML, CSS 1, 2, and 3, the DOM, XML, and ECMAScript 1.5. Imagine for a moment how they all interact, and how many gray areas that produces. Then, go and examine several million broken Web pages, each of which caters to a different quirk of old browsers—their proprietary DOMs, workarounds, and hacks. Then see if that question still requires an answer!
NA: In our experience, QA is often the forgotten component of the development process, overlooked and undervalued. Was it difficult to get the Mozilla team to recognize the importance of a QA process?
GM: Because Mozilla was initially driven by Netscape, which had a QA department, QA has always been a feature of Mozilla development. Each component of the product has a QA contact who is (theoretically) responsible for quality in that area. The need has always been understood. Actually getting the resources is sometimes a harder thing.
NA: An open source project of Mozilla's scope is certainly ambitious, but crafting an organized QA process with so many far-flung volunteers must be nearly as challenging.
GM: The Mozilla Project got off to a slightly rocky start with regard to external involvement. It took the efforts of Eli Goldberg at Netscape and Asa Dotzler, then of Texas and now of mozilla.org, to pull in QA people and create a community. We're organized around our three communications tools: newsgroups, IRC, and Bugzilla, which is a bug system and a giant message board in one.
NA: On the Mozilla QA homepage, you quote Eric S. Raymond as saying, "Given enough eyeballs, all bugs are shallow." How does a QA process benefit from throwing more manpower at it, where many other types of projects don't?
GM: QA benefits from more bodies because, as Raymond also says, "debugging is parallelizable." Ten people triaging bugs is ten times as effective as one—more so, because they can communicate via IRC and hand off bugs to the person with the expertise in a particular area.
NA: What have been the biggest challenges along the long road to Mozilla 1.0?
GM: Bug triage. Our bug system is totally open. Anyone can file a bug, and some days it seems almost everyone has. We get several hundred bugs filed per day, fifty percent of which end up resolved as duplicates of other bugs. We've had to be very proactive about developing techniques to stop people from filing pointless, vague, or duplicate reports.
NA: The browser has taken a central role in computing today. Do you sense any special pressure or intensity surrounding QA for such a pivotal piece of software?
GM: The browser is under particular pressure to be secure, and this obviously reflects on the QA team, which tests it. Microsoft has a particularly poor track record in this area, and Mozilla has been leading the way with a better security architecture and response to problems. The first security hole was found in Mozilla recently, and it was fixed and patches were made available within 48 hours. By contrast, IE currently has fourteen open holes (see www.jscript.dk/unpatched/), with no news as to when they'll be closed.
NA: What can other companies learn from the Mozilla QA process?
GM: The lessons from Mozilla QA are open source lessons: Release early, release often, and let your customers bang on it. They are your best testers. But be sure they know it's not the finished product.
NA: With the release of Mozilla 1.0, what will be next for the Mozilla Organization, and the QA team in particular? Will you be winding down?
GM: Absolutely not. Mozilla 1.0 is a beginning, not an end. It's about API freezes, and saying to embedders and to our customers (OEMs) that we're now happy for them to take Mozilla and build on it in safety. Going forward, there will be both trunk builds and 1.0-branch builds, with QA going on for both. I don't think there's a particular focus; people file bugs on everything! We just do what needs doing at the time.
NA: So what's next on Mozilla QA's immediate agenda, post-1.0?
GM: Mozilla's future and goals aren't driven by the QA team. We'll just keep making the product rock more and more.
