Why ‘test’ rather than ‘prove’?
- The questioner had been asked by a developer why he wasn’t proving that the software worked.
- What is meant by proof? (The developer wanted to know that “it worked as expected”)
- What is meant by works?
- What would constitute proof for the developer in this situation?
- Do we tend to think of proof in an absolutist, mathematical sense, where axioms, assumptions, deductions and so on are deployed to make a general statement?
- … remember that a different set of axioms and assumptions can lead to different conclusions.
- In this view, is proof 100% confidence?
- There is formal research in proving correctness of software.
- In the court system, we might have proof beyond reasonable doubt
- … which seems to admit less than 100% confidence.
- Are we happier with that?
- But still the question is proof of what?
- We will prioritise testing
- … and so not everything will be covered (if it even could be)
- … and so there can’t be emprical evidence for those parts.
- How long is enough when trying to prove that a program never crashes?
What would you be if not a tester? What skills cross over?
- One of us started as a scientist and then went into teaching. Crossover skills: experimentalism, feedback, reading people, communicating with people, getting the best out of people.
- One of us is a career tester who nearly went into forensic computing. Crossover skills: exploration, detail, analysis, presentation.
- One of us was a developer and feels (he thinks) more sympathetic to developers as a result.
- One of us is not a tester (Boo!) and is in fact a developer (Boo! Hiss! etc)
- I am a test manager. For me, the testing mindset crosses over entirely. In interactions with people, projects, software, tools, and myself I set up hypothesis, act, observe the results, interpret, reflect.
- … is there an ethical question around effectively experimenting “on” others e.g. when trying some approach to coaching?
- … I think so, yes, but I try to be open about it – and I experiment with how open, when I say what I’m doing etc.
Pair Testing. Good or Bad?
- The question starts from Katrina Clokie’s pairing experiment.
- The questioner’s company are using pairing within the Test team and find it gives efficiency, better procedures, skill transfer.
- It’s good not to feel bad about asking for help or to work with someone.
- It helps to create and strengthen bonds with colleagues.
- It breaks out of the monotony of the day.
- It forces you to be explain your thinking.
- It allows you to question things.
- It can quickly surface issues, and shallow agreements.
- It can help you to deal with rejection of your ideas.
- Could you get similar benefits without formal pairing?
- Yes, to some extent.
- What do we mean by pairing anyway?
- … Be at the same machine, working on something.
- … Not simply a demonstration.
- … Change who is driving during the session.
- We have arranged pairing of new hires with everyone else in the test team.
- … and we want to keep those communication channels open.
- We are just embarking on a pairing experiment based on Katrina’s.
- Edward Tufte, Beautiful Evidence, Envisioning Information: data analysis is about comparison so find ways to make comparisons easier, more productive etc on your data.
- Michael Lopp, Managing Humans: we all have to deal with other people much of the time.
- Amir Alexandar, Inifinitesimal: experimental vs theoretical (perhaps related to the idea of proof we discussed earlier).
Edit: Karo Stoltzenburg has blogged about the same session and Sneha Bhat wrote up notes from her group too.