I have been outed! Sorta.

If you follow the Google Testing Blog you might have read the latest post by Dr. James Whittaker's about 'antenna gate'.

The gist of the post was that Apple missed an important bug -- holding the phone too hard will ruin the antenna signal -- and what the testing community can learn from it. One part of the post of particular interest to me was:

Risk analysis is so important that we've built an internal tool to help guide testers in performing it. Code-named "Testify" this tool streamlines the process of risk analysis, at least the way we do it at Google. We're working on open-sourcing an early prototype in time for GTAC 2010 (I can hear my team cringing now ... "you promised it when?").

The mention of a mysterious risk analysis tool sounds interesting and I like the humorous anecdote about the team cringing about their newfound deadline. Well, I would have found that humorous if I wasn't on that team. (And too busy writing code for that newfound deadline.)

I'm one of the people working on that "internal test tool" for performing Risk Analysis, and am especially glad that James outed the tool so I can talk about it publicly. Risk Analysis is an interesting topic that needs to be mentioned more in software testing circles.

In my experience a classic mistake testers make is being too focused on finding bugs. That's right, you can be too focused on finding bugs. Because what really matters is answering the following questions:

• Is your software ready to ship?
• If not now, when?
• When it does ship, will it solve the problems it set out to solve?

You can find bugs until I'm blue in the face, but systematically finding and removing defects doesn't get you any closer to answering those questions. What matters is understanding where you are testing and what areas need that testing. Polishing the bottom of a banister isn't going to do you any favors.

Risk Analysis is the process of looking at the distribution of possible outcomes and understanding your acceptable risk. If the worst thing that can happen if component X breaks is that the user gets annoyed, then that might be acceptable. Certainly a $500 phone being rendered inoperable is not an acceptable risk.

Risk Analysis wouldn't have said, "Hey, Apple, make sure holding the phone doesn't impact the signal!" But it would have uncovered that there is a piece of the system that has potentially devastating risk, and therefore needs some more investigation or testing.

I don't think Risk Analysis or tool like Testify will revolutionize the way people develop software, but I do think that if used judiciously they can help prevent the next 'gate'.