Testing as a Creative Endeavor

Recently, I did a Google search on “agile test automation”. I’m struggling to figure out how, if at all, test automation fits in to a very agile and loosely structured software development model. I come from a world of three- to twelve-month software release cycles. Where I work now, we release new code to production at least once a week. It’s exciting and invigorating.  What I most enjoyed about writing test automation was the challenge of building in extensibility. Without extensibility, automation is one-dimensional: it does one test or set of tests and reports what it finds. In agile development, software changes in small and not so small ways almost constantly. As the software changes, simplistic automation can’t change with it (at least not without more human intervention), and as such has limited value. By the time we make the test code robust enough to add sustained value, the product it’s designed to test has probably morphed two or three times over. Is robust automation being fast-paced into obsolescence, and if so, is there a new, nimble automation approach to replace it? Or is my search string an oxymoron?

In pursuing these questions, I stumbled upon articles slightly off topic but decidedly more interesting, about the mysterious process we call “exploratory testing”. This is the farthest thing from automation. It involves creative thinking about how you’ll approach the software, some knowledge of the target market, a notebook to record findings, and a chunk of uninterrupted time. In addition to these basics, skilled testers bring to the table some knowledge of the risk areas, and my favorite, intuition. The thrill of the hunt is on: you don’t know exactly what you’re looking for, but you’ll know it when you find it.

You may have heard software testing referred to as an art. I was delighted to discover Jonathan Kohl’s article likening exploratory testing to music. He offers that key both to successful software testing and great music is the proper balance of tension and resolution. Without this balance, he says, neither testing nor music is at its best, and both are boring. James Bach defines exploratory testing as “simultaneous learning, test design, and test execution.” Remove the word “test” from this definition, and it sounds a whole lot like musical improvisation. Testing — like music — as art. Nice. I agree with my test junkie colleagues that exploratory testing, rather than “don’t color outside the lines” scripted testing, keeps us engaged and nets better results: more bug finds, in shorter test sessions.

So the search didn’t lead to what I thought I was looking for. I still haven’t satisfied my question about the place of automation in agile development. But, like exploratory testing, following the unintended path led to the richer, more interesting discovery of like-minded thinkers in the field.