Sometimes it’s hard to resist the temptation to just go for it, to run at the fresh build and leave long trails of filthy QA bootprints all over its virgin white snow. But hold on, Captain Scott! Sometimes you want to know it’s not ice before you start stamping.
Sometimes includes projects with significant risk outside of the software itself, like running over budget or headed for late delivery or even non-delivery. When that sometimes is now it’s reasonable (although, as always, not always) to try to shorten each iteration somehow, to limit effort until the likelihood of it being well spent is higher. That kind of somehow should look to fail a build early and at low cost and, to do that, somebody needs to find cheap and discriminatory tests.
Of course, the somebody who’ll have to do that is you and you want tests which, while they might not have the power to explain what the issue is, can say that there’s an important issue someplace. It’s more than a smoke test. You’re after sanity check oracles who’ll work cash-in-hand. Remember that they’ll be heuristic: a pass doesn’t mean that everything’s hunky dory but it does mean that it’s more likely to be worth continuing testing.
Some examples might be:
- the application can run in multiple modes and, regardless of mode, some values in or features of its output must be identical (or derivable from each other.)
- a grep of config files for inconsistencies or perhaps comparing values with the installed application under test.
- you have downstream processes that you could feed output to as a cheap validation test.
- there are performance constraints that can be easily monitored and tested for on some dummy runs.
And once you’ve got these you can start to automate them, and give them back to the developers to use, and start to build up your test suite. And maybe then you can pull the project back on track somewhat, somewise. Some chance…
Image: J Fry / FreeDigitalPhotos.net