The “Test is Dead” story is not news anymore. A lot of the good testers out there have since tried to reframe the idea, to put in into context. See Mark Tomlinson’s blog post on the subject or listen to this TWiST podcast (requires basic membership; it’s free) for good examples.

To refresh your memory, here’s a short recap:

After posting an ominous article about the death of testing this June, Alberto Savoia finally got to explain those cryptic lines in his opening keynote for GTAC 2011. In short, he believes that building the right product – the right “it” – is far more important that building “it” right. Or, to put it another way:

Testing slows you down… by the time our competitors had a well tested product, we had millions of users.”

So he appears to claim that it is far more important to test the concept than the actual product. He even wrote a book about it.

I have to admit, I can see a lot of potential in this approach. The success of twitter is just one example of how users can easily overlook even serious bugs in the product as long as the “it” part feels like warm apple pie.

Then again…

Going by Scott Berkun’s awesome guide of how to call bullshit on a guru, let me ask this:

“When is the theory you are advocating wrong?”

Well, off the top of my head…

And I haven’t even mentioned airplanes, medical devices or military computerized weaponry.

Still. In the context of getting the next big “it” to market in Silicon Valley… yeah, sure. I cannot deny that it worked out really well for twitter. I’m pretty sure it will work for a lot of awesome new software in the future too. In their context, being on the market will probably far outweigh the risk of any and all bugs.

So is testing dead or not?

To quote Scott Barber’s hilarious mock interview blog post on the same subject:

Silicon Valley is not the only place where development is done these days; neither are awesome social networking solutions the only kind of software in existence. Heck, a lot of software doesn’t even HAVE a market, only a customer that is already paying for it anyway and wants his product to actually work once it’s done.

The examples above are just the tip of the iceberg. Most of the software I see in my work has to be tested, no matter what. As far as I am concerned, it’s only when the health, money and time of the customers are not at risk that a “test is dead” approach can be considered.

I could go on, but Tomlinson’s article covered the test-is-dead bashing part well enough already.

However, there are some aspects that haven’t been discussed as of yet.

Taking it further

Now: why is it that when Savoia gave his keynote, a great number of good people out there deemed it necessary to reframe his ideas?

This is a prelude to a longer series prompted by the test is dead issue and the general problem of saying something that people are bound to misinterpret. More to come.


One Response to Putting “Test is Dead” into context: a prelude

  1. Hi – and thanks for another great dive into the topic of “test mortality” – and for the links back to my demystification of Mr. Savoia. In reply to your question, “why a great number of good people out there deemed it necessary to re-frame his idea?” – allow me to share my perspective.

    You see, it’s not that a great number of good people decided to re-frame Alberto’s idea, it that Alberto tried to re-frame his own good idea for pretotyping into an anti-test plea that spits on the professions of a great number of good people.”


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>