I had the pleasure of hosting the second Online Summit, delivered by Software Test Professionals: Achieving Business Value With Test Automation. The online summit format consists of 3 sessions each for 3 consecutive days. The sessions for this summit were:
- Software Test Automation, Where’s the Value? – Scott Barber
- Achieving Value with Automated Acceptance Tests – Linda Hayes
- Developer Level Automation Value from “Velcro” to “Velocity” – Rob Sabourin
- The Simple and Powerful Build Verification Tests (BVT) – Alan Page
- Functional Automation is the Gateway Drug to Continuous Delivery – Adam Goucher
- Test Automation at the API Layer…Why Bother? – BJ Rollison
- Experiences with Adding Value Through Test Automation – Mark Fewster
- Test Automation Misconceptions and Money Pits – Hans Buwalda
- Top Tips for Addressing the Challenge – Scott Barber
- Speaker Panel – Scott Barber, Linda Hayes, Alan Page, Adam Goucher, Mark Fewster & Guest Panelist Griffin Jones
One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a “top tips” list. This is what I came up with:
- Not every tip is specific to test automation
- Several tips are points made by more than one presenter
- Like every good “Top 10 List”, tips are presented in reverse order (according to me)
- I’ve paraphrased many of the tips to make them “quippy” 🙂
Tip #10: Pure record/playback => FAIL
- Record/playback is simple, seductive, and is (in most cases) not the most valuable answer to a question worth answering.
Tip #9: Learn automation in low risk situations
- Do not start out by automating tests against production
- Do get your skills “up to snuff” before making promises your automation can’t keep
Tip #8: Forget ROI, think Cost/Value
- Provide Business Value
- The ROI of testing can’t be a positive number (unless your business product is testing services)
- Determine what value automation can *reasonably* provide
- Estimate the cost of providing that value
- Ask “is that value worth the cost?”
Tip #7: Valuable automation is trustworthy
- To be trustworthy, automation must not be excessively complex
- Diligently avoid false positives.
- Err on the side of false negatives.
Tip #6: Design Build Verification Tests (BVTs) to tell you if additional testing is valuable at this time
- Do not try to design BVTs to tell you whether or not the build is release ready
Tip #5: Test at different levels to accomplish different goals — use the tools that work/makes sense
- Unit, Object, Integration, API, and/or GUI
- This is true for manual, automated, or other forms of testing
Tip #4: In the land of automation, test design is king
- Brainlessly automating manual tests typically leads to a whole lot of NO VALUE
- Executable Requirements, Keywords, Classes, & Actions
- Test Data, Test Data, Test Data
Tip #3: Collaborate … Just do it!
- There is no wall between developers and testers, there are only politics.
- Reject politics and collaborate to accomplish the mission
- Testing is not a union that prohibits others from testing or testers from doing other things.
- Enable the right people, with the right skills to add the most testing value at the right time in the most efficient/effective manner
Tip #2: Automate things that are better suited for computers to do than humans — only if they have value
- Application deployment is a great candidate for automation – whether or not any automated testing is included.
Tip #1: Do not automate what requires human judgement.
- Just don’t.
- ‘nuff said.
“If you can see it in your mind…
you will find it in your life.”