“Agile Automation” an Oxymoron? Resolved

Test automation, it turns out, is a broad term. The tools with which to do it range from the massive, expensive, infrastructure intensive, to the open source download-and-run. The latter are easy to deploy and make quick use of in an agile environment, but even in this context some suggest the best outcome is achieved by assigning dedicated resources to it.

Inherent in our culture is the constant striving to find ways to do everything better. We tend to jump to technology for this, assuming that if we can do it, we should, be it food production, human reproduction, national defense, or of course, software testing. But should we always look to technology to make things better? Catapulting headlong into uncharted technological waters just because we can has been known to backfire. Consider DDT , DES , the atomic bomb. We’ve paid a high price to learn that in many situations it’s healthier to do things “the old fashioned way”, rather than act on the assumption that technology could somehow do them better.

It strikes me that the way Exploratory Testing is now defined is exactly how we tested when I was just starting in the field, before we attempted to formally define it, and before we decided we could test software better by writing more software to test it with. Automation seems like a positive (and harmless) technological advance because it makes highly efficient work of crawling through software, performing in mere hours several human-days’ worth of keystrokes and verifications. But its inherent limitations can place at risk precisely that which it’s designed to protect and improve. Automation only tests what we tell it to test. It lacks the human intuition that tells us when to stray from the path to look in a new corner, peel back a curtain, or lift up a rock. These are places bugs like to hide. Like the consumer of DDT-laden produce, software tested primarily by automation can end up sicker than it might have had it enjoyed the magic touch of old fashioned exploratory testers.

The question is not so much can we use automation in an agile environment, but should we? Certainly there is a place for automation in software testing, but it can’t do it all. We should use it with care lest it become the costly and potentially caustic stuff caked on otherwise healthy organic software testing.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s