This is Part IV in a series of entries inspired by the following quote from the “about page” of hosted by Cem Kaner:

“…However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now…”

And James Bach’s blog update (Context-Driven Testing at a Crossroads):

“I’m the last of the founders of the Context-Driven School, as such, who remain true to the original vision. I will bear its torch along with any fellow travelers who wish to pursue a similar program.”

If you haven’t done so already, I recommend starting with:

So far I’ve established that I’m a Context-Driven guy. For completeness, I should also share that I’m a guy who is most comfortable operating as part of a healthy team that embraces Agile principles, but who recognizes that Agile is not the most appropriate or effective answer for all organizations, teams, or situations.

I’ve also noted that I find the notion of “product” in both Context-Driven and Agile principles to be too subtle of a reference to the fact that the propensity of software is developed in a business context for my tastes. This is mostly due to many, many personal observations of individuals involved in the process of developing and delivering software emphasizing some aspect of the software over business value — from individuals who self-identify as Context-Driven, Agile or neither.

The reality that I have lived in since beginning my career as a technologist is that, business is the primary context-driver behind the development of the propensity of software and that money is the primary context-driver behind business (yes, I know, that’s a broad generalization, with somewhat ambiguous qualifiers — I’m going to ask you trust that I’m happy to support and specify that statement if needed, but for the time being, please accept the premise… at least while reading the remainder of this post.)

If you accept that premise, I believe (and I’m sure folks will point out if and where my logic is flawed) the following is a fair and logical model for testing in the context of business:

  • Business is driven by money
  • Businesses develop & deliver products & services for the purpose of generating or protecting revenue (money)
  • Sometimes businesses also develop & deliver tools & processes for internal use for the purpose of making it faster/easier/cheaper to generate or protect revenue (subsequently “support revenue”)
  • The more cheaply and quickly that product/service/tool/process (subsequently “product”) can be developed, customized and/or implemented sufficient to generate/protect/support revenue, the better
  • Testing is the activity focused on finding and providing information of interest to people who matter (stakeholders) about the product
  • Therefore- The primary reason businesses pay for testing is because:
    • they want as much information as possible
    • for a reasonable cost
    • available to stakeholders involved with developing, customizing, implementing, assessing, managing, and/or making business decisions 
    • related to the relevant product
    • with the expectation of that information allowing the product to start generating, protecting, or supporting revenue more quickly and cheaply
    • than it would if the business had *not* paid for testing.

In other words, the primary reason that testing happens in a business context is to help the business achieve their primary goal, which is to make money. Or even more simply, businesses choose to pay for testing *only* because they believe it costs less to pay for it than it will cost to *not* pay for it in the long run.

See the graphic below for another way to think about this.

(Larger view, .pdf format)

I could follow the same logic (but I will spare you for the time being) related to what most businesses treat as their next most significant context-driver, Risk.

I’d like you to reflect for a moment and notice some things about what I just did:

  1. I made a case that testing is all about money in a business context
  2. I never mentioned “requirement”, “test case”, “exploratory”, “automated”, or “tester”
  3. In fact, I made my case without ever mentioning “software”

Needless to say, this was intentional. I submit that the logic above applies equally well (or equally poorly) to physical products, qualitative services, software tools, process templates, or most anything else businesses use to generate, protect or support revenue. That is because I believe that while  software is certainly different from physical products in some important ways, from a business perspective, the goals & decision making processes are very much the same as for other types of products; thus requiring the same kinds of information.

In fact, I’ll go a step further, software development is not all that different than physical product development *except* for what I have labeled in the graphic as “Productization”. In both cases, what we are really referring to prior to Productization is “Research and Development” (again, that’s an assertion deserving of far more discussion and support than makes sense for this post. Consider this a bookmark for a future post).

So, what’s my point? Testing in a business context is done to help the company make/keep money. Testing is not done to ensure (or assure) quality, to protect the customer, or to make the software “good”. Testing done with the intent of improving the quality of the product is *only* valuable until the product achieves a certain minimum quality as determined by the business to be acceptable for achieving business goals (and in actuality is, even then, only incidentally valuable). Any testing done that cannot be linked to generating, protecting or supporting revenue is, in effect, working against the goal of the business (since that testing cost money, but does not provide corresponding monetary value).

Ergo, all that study, advancement, & education about how to test better may have made you a better tester, but did it make you more valuable to your business? I say, only if you have managed to apply that knowledge in a manner that made more of your testing more focused on business value and as a result helped your business make or keep more money.

I’ll go out on a limb and say that for most of you, the training and education you received over the last 10 (or more) years was far more focused on things like test process, techniques, tools, and metrics than on providing business value with the skills and knowledge at your disposal.

And *that* is why testers haven’t gained more respect, why ridiculous statements like “Testing is Dead” get so much traction. It’s not that testING is dead, it’s that testERS are the only ones who seem to be interested in calling attention to testing as separate line item under “product development”. The reality is that testing gets done pretty much everywhere that research and development happens, products are made, or services are developed — physical, conceptual, or software. The value of testing to the business is being realized (to an acceptable degree to the business) in areas other than software even when the word “testing” is completely absent from the business radar. When it comes to software, on average, testing is on their radar, and testing is not providing the value they are looking for (or, at least is not being communicated in a way they understand).

That thought process is what has led myself and several collaborators (I’ll allow them to self-identify if/when they so choose) down this road of deeply exploring business value as it relates to testing (and software in general). What I’ve included in this post is little more than a condensed summary of the background to our research, but now seemed like as good a time as any to let the cat out of the bag a little and gauge the public reaction and counter-arguments before our research goes far enough that it would make us angry (as opposed to grateful, sad and/or confused) if/when folks find flaws or gaps in our thinking.

Oh, and if you’re wondering what the connection is between all of this and Context-Driven, it lies in the “generate, protect and/or support revenue” statement.  For example, testing related to revenue generation may focus on functionality or regulatory compliance; testing related to revenue protection may focus on maintainability or legal defense; testing related to supporting revenue may focus on business process improvement or cost reduction. Of course there are many, many other potential context-drivers, but that too is a discussion for another day.

Scott Barber
Chief Technologist, PerfTestPlus, Inc.
Director, Computer Measurement Group

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

“If you can see it in your mind…
     you will find it in your life.”