Track Sessions

Testing and Music: Parallels in Practice, Skills and Learning

Michael Bolton and Jonathan Kohl
Tuesday, July 15, 10:45am - 11:45am

Many metaphors--besides engineering--can value to learning about patterns and principles in software testing. Music affords such a metaphor. Both testing and music are performed in a variety of contexts, providing different values for different people. Both fields are suffused with traditions; both involve dynamics between scripted and exploratory processes. In this presentation we'll try to discover comparisons and contrasts that might help to advance learning and provide new metaphors for testing.

Testing Fuzzy Interfaces - Can We Learn From Biology And Wargaming?

Bart Broekman
Wednesday, July 16, 2:45pm - 3:45pm

We testers don't like fuzzy functionality—vague descriptions of situations and how the system responds—because we can't tell if we've "tested it all" or if we can "predict the result of our test cases".

This presentation explores other ways of designing test cases, besides logically deriving them from system specifications. From biology, Darwin's theory is applied to the process of designing test cases, whereby test cases evolve into better test cases. From wargaming, when acting as players of a wargame, we can find defects in situations that would remain undiscovered by 'classical' methods.

Measuring File Systems

Morven Gentleman
Tuesday, July 15, 2:45pm - 3:45pm

Performance testing almost always involves cross-disciplinary skills. These skills include domain knowledge to identify sensible questions, appropriate variables, instrumentation to measure them, and functional forms that they might follow; statistical knowledge to design experiments, choosing control variables and observations; and data visualization knowledge to recognize breakdowns in models, outliers, or censoring.

This talk will illustrate these ideas as they are relevant to measuring and understanding file systems, their design, and static and dynamic characteristics of the load. It will show how designers, system operators, and users can perform empirical studies to aid in implementation choices, purchasing decisions, system tuning, and operational practices.

Lessons in Team Leadership from Kids in Armor

Adam Goucher
Wednesday, July 16, 2:45pm - 3:45pm

Groups of testers within an organization are often given the label of a team and point person is often a 'lead' or 'manager.' In sports, the person who leads the team is the 'coach'. While leading a group of testers or 4-year old lacrosse players might seem at opposite ends of the spectrum the similarities are striking.

The Challenge of Testing Scientific Software

Diane Kelly and Rebecca Sanders
Tuesday, July 15, 10:45am - 11:45am

Though scientific software is an engine for scientific progress and provides data for critical decisions, the testing of scientific software is often anything but scientific. Our research examines the factors that complicate the testing of scientific software, including the complexity of the subject matter, inadequate validation criteria, a high demand for correctness, and the lack of testing expertise among scientists.

Sleight-of-Quality : A Magical Approach to Testing

Jeremy Kominar
Tuesday, July 15, 2:45pm - 3:45pm

Quality assurance requires a diverse set of skills, demonstrated in complex testing environments. The study of traditional magic principles can help software testers raise their awareness of discrepancies that can be found in these environments, leading to improved quality assurance. Software likened to a magical "trick" offers an interdisciplinary approach to the study of method and effect, enabling us as professionals to approach testing from new angles. Magic in this context will not only educate but will also entertain!

Seeking Data Quality: Using Agile Methods to Test a Data Warehouse

Steve Richardson and Adam Geras
Wednesday, July 16, 10:45am - 11:45am

The basic purpose of a data warehouse is to provide information that will help people make better choices, with the outcomes of those choices determining the value of the data warehouse. This value comes at the cost of building and maintaining the data warehouse and depends greatly on the data quality. Inspired by agile testing methods and value-based prioritization, we achieved great data accuracy at minor cost.

Visualization and Statistical Methods In High Volume Test Automation of Embedded Devices

Martin Taylor
Tuesday, July 15, 10:45am - 11:45am

This presentation is an industrial experience report from Texas Instruments Education Technology (TI Calculators) division. Martin will describe how his team combined high volume test automation techniques, data visualization, and statistical regression methods to detect memory leaks, device crashes, and performance problems in the new TI-Nspire(tm) math and science learning handhelds. Martin also reports how these techniques have been used at TI to compare results between various TI-Nspire(tm) software builds.

Software Testing To Improv

Adam White
Wednesday, July 16, 10:45am - 11:45am

Improvisational theatre is about exploring and extending your environment: you go with what you have in an attempt to make your team, product and company better. My experience testing on turbulent software projects is often informed by the same principles. I frequently ask myself "How do I make this the best test in the world?" "What tests will expose the best bugs?" "How am I going to explore my environment?" In this session, Adam White will explore improv concepts and give examples of how this relates to his experience as a software tester.

Lessons for Testing from Financial Accounting: Consistency in a Self-Regulated Profession

Doug Hoffman
Tuesday, July 15, 2:45pm - 3:45pm

Comparing software testing with accounting is a study in stark contrasts. There are valuable lessons for software testing from several millennia of evolution and refinements in accounting, ranging from underlying accounting principles to professional self-regulation techniques. This session draws parallels from accounting and points out some valuable lessons that might be applied in the software testing profession.

Testing Lessons From Civil Engineering

Scott Barber
Wednesday, July 16, 2:45pm - 3:45pm

When tasked with designing and building an aesthetically pleasing bridge, using new environmentally friendly materials, over a river that is likely to flood over the roadway of the bridge once every ten years, an engineering firm doesn't start testing when the bridge is mostly complete. They don't even start testing at the same time that they start building the bridge. Rather, they start testing as soon as someone proposes an initial design. And even though bridges do fail sometimes, they fail catastrophically significantly less often than software does.

While I agree that testing a bridge is fundamentally different from testing software, the general approach and thought process that I learned while earning a B.S. in Civil Engineering have turned out to be extremely useful to me as a software tester. What surprises me is how few software testers have been exposed to these approaches and thought processes.

Come to this session for an introduction to the ways in which Professional Engineers test their creations and what lessons we can learn and appy to our jobs as software testers that will make our testing more effective and help our developers make better applications more efficiently.

Testing Electronic Voting Equipment, the VVSG, and the AST eVoting SIG

Geordie Keitt
Tuesday, July 15, 10:45am - 11:45am

Why is the quality of software for electronic voting machines so poor? The testers are well-meaning professionals, but the system discourages finding and fixing bugs. The vendors have an incentive to build in lots of bugs, it makes them immediately obsolete and forces electoral districts to buy the next version. The vendors certify their systems using a small number of test labs, whom they hire and which answer to them. What good does it do for a test lab to bite the hand that feeds it and report bugs? The certification guidelines, aka the VVSG, was written originally by a committee led by a representative of the vendors, and used the latest and greatest milspec software development guidelines to institutionalize the vendor / cert lab relationship. More recent versions go much deeper into exactly how the software should be tested, yet still leave massive holes in the testability of the systems, especially by making it unnecessary to test COTS software that is included in the systems. The AST eVoting SIG is working to improve the VVSG by developing a real, useful test plan for all eVoting machines, refining the plan on real systems, and creating a failure mode taxonomy for the eVoting problem space. You can help.

Back to the Program page...

To the Registration page...