Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing

Chapter 1. Defining What's on Your Plate: The Foundation of a Test Project

Testing requires a tight focus. It's easy to try to do too much. There is a virtually infinite number of tests that you could run against any nontrivial piece of software or hardware. Even if you try to focus on what you think might be "good enough" quality, you can find that such testing is too expensive or that you have trouble figuring out what "good enough" means for your customers and users.^[] Before I start to develop the test system—the testware, the test environment, and the test process—and before I hire the test team, I figure out what I might test, then what I should test, and finally what I can test. Determining the answers to these questions helps me plan and focus my test efforts.

^[] To my knowledge, James Bach first applied the phrase "good enough" to software quality. You can find his thoughts on the subject at www.satisfice.com.

What I might test are all those untested areas that fall within the purview of my test organization. On every project in which I've been involved, some amount of the test effort fell to organizations outside my area of responsibility. Testing an area that is already covered by another group adds little value, wastes time and money, and can create political problems for you.

What I should test are those untested areas that directly affect the customers' and user's experience of quality. People often use buggy software and computers and remain satisfied nevertheless; ...