A month ago Michael Bolton sent me an article that underscores the problem of “all tests being deterministic” and “we have to cover all the functions in this application.” Something I was thinking about while I read the article was how a checkup or physical at a doctor’s office works and how that compares to software testing.
Often stakeholders want testers to “test everything.” Can you imagine if instead of checking the usual health indicators and vital signs your doctor tried to test “everything.” Consider how long that would take, and how much it would cost. It would be ludicrous to do such a thing especially if you were in good health.
However the risk in not testing everything (and the primary fear of our stakeholders), is that the usual indicators and vitals don’t always turn up something in the examination room. A patient could leave and drop dead in the parking lot. So what can doctors, or in this case testers, do knowing this very real risk?
I think the key is having a medical history for your software. Having year’s worth of tribal knowledge about the application helps you decide what indicators and vitals matter. Having this history helps a tester perform tests that are likely to turn up something important about the application while it’s in the building. While there still is no guarantee that the application won’t drop dead in the parking lot, it does increase the chances of finding something significant before it does walk out the door.