Purely by chance I encountered three takes on perfection in the space of a couple of days.
“Perfection”, Run DMC said, “is quite essential”. A laudable aspiration but, as their later career showed handsomely, hard to achieve. Compromise has a lot of very persuasive arguments.
“This software is bug-free. It is perfect”, Fast Company said of the Space Shuttle’s control system, “as perfect as human beings have achieved … the last three versions of the program – each 420,000 lines long – had just one error each.” Skipping over how they knew that there was just one error, and why they didn’t fix it, NASA had obvious reasons to strive for perfection, including the need to certify “that the software will not endanger the shuttle” before launch. Their approach was to be meticulous in the meta-work that supports software production as well as the production itself. But they still had errors.
“We test”, Weinberg said, “because people are not perfect”. Weinberg’s book, Perfect Software And Other Illusions About Testing, is a classification of the ways in which the human factor – in the coding, the development process, the testing, the management or anywhere else in the creation of a piece of software – rules out the possibility of perfection.
So what lessons are there for us?
- Software should generally be tested (the extent and rigour determined by the context) because the people who made it are unlikely to have made it defect free.
- Testing is not just an engagement with the software but an implicit interaction with the people involved in the software.
- We often put ourselves in the mind of the user but we should also be putting ourselves in the mind of the software creators, the business development team, the people who designed the process, the GUI designer, the QA manager and the rest. They aren’t black boxes, so what do we know about them, their strengths, frailties, foibles, blindspots, that will give us an insight to exploit in tests?
- Stopping at Raising Hell would have been perfect.