5 reasons why exploratory testing may be better for your software project

There are lot of disputes about scripted vs. exploratory testing. Here is a modest attempt to make more obvious some of advantages of later tasting approach.

1.  ET is more agile

Even if Agile manifesto is nearing its first decade, “Agile” is word which still sounds like a buzzword in IT business. It sounds so fresh and promising. It is more obvious than ever that modern software development is focused to users rather than code, functionality rather than documentation, customer rather than contracts, response to change rather than rigorous plan.

Since exploratory testing approach liberates the testers from the focus on pre-scripted plan and underlines the effort on productivity – providing the quality information – rather than completing the “hardcoded” test set, this approach is agile in nature. It allows testers to be valuable members of team which can bring in fresh quality related information quickly and efficiently.

2. ET is more cost effective

Exploratory testing uses so called “Charts” where specific assignment for tester is noted but with no designated step by step procedures. In the phase of creating test cases, this eliminates the need for test lead (or one who creates the tests) to focus in detail on steps of performing each test. This is often actually impossible because software test team is oftenly involved in test planning before the actual program for software under test is yet ready to be used. Test creator is depending on user stories and mockups to create a charter. It is not unusual that in this phase no one can predict how software will look and operate in detail so detailed planning of test steps in this phase is actually not possible (even if those tasks are performed because you have to do what you are ordered to ;))

Much more use of valuable time of test creator would be to carefully read the documentation, user stories and mockups and to determine actual usecases of application and write those down. Because exploratory testing is constant test creation, those charters created at the beginning of the project  can be instantly used and extended when application becomes testable so no additional efforts are invested in redesigning test scripts which obviously can missing steps or having wrong steps.

3. With ET, tester is involving more time in actual testing

As former chapter suggests, with scripted approach of testing, it is not unusual testers are spending much time in correcting those steps to affect actual application under test. Testcases are written before UI, or testcases are written for previous version of the application and migrated to new version but application undergone lot of changes, there are custom builds but you have to temper with original copied test scripts to fit the custom, branched, builds.

It is not unusual that lot of effort goes to scripts maintenance but it is also not unusual that in some point no one cares of test steps and just switch to some kind of edgy exploratory mode by keeping information of changes in head (popup window is replaced by inline frame in version 2.0, terms & conditions are no longer checked with checkbox but noted with asterisk…) , so there are misleading information in actual test scripts (login form is still in pop-up, terms & conditions still having checkbox…) which are confusing testers that are new to the project. Also lot of unnecessary defects are raised. Not to mention lot of frustration among developers. The worse happens if   those misleading information are unselectively copied to new testsets for new versions. Bottom line -  you can end up with testsuites which does not suite anything. Efforts (and costs) needed to make such unmaintained suite operative are huge.

4. With exploratory testing “pesticide paradox” of testing is very low

As James Bach colorfully put it  “Highly repeatable testing can actually minimize the chance of discovering all the important problems, for the same reason that stepping in someone else`s footprints minimizes the chance of being blown up by a land mine.”  If test cases are designed to be rigorously repeated step by step, big bugs can come unnoticed because testers are so focused on going through the steps watching for expected results.

Following the steps, testers can easily went pass the landmines in software simply because step never implied “click this after you did that” or “enter this information after you did that” and maybe the bug just lies in that, not scripted, use case. By executing the scripted testcase, that hidden bug will never be exposed and we have a “pesticide paradox” – old pesticide does not kill new bugs.

By not obliging to following the steps, testers can freshly start with each ET session because charter do not put the designated procedure in front of testers. It just reminds him what to be tested leaving to the exploratory skill of tester to find the hidden mine.

5. ET charters are much more robust than classic test scripts

By definition found in Wikipedia “Robustness is the quality of being able to withstand stresses, pressures, or changes in procedure or circumstance. A system, organism or design may be said to be “robust” if it is capable of coping well with variations (sometimes unpredictable variations) in its operating environment with minimal damage, alteration or loss of functionality.”

As already stated in this article, scripts are very little tolerant to change. And change is not unusual in modern software. Login box is moved elsewhere. Buttons are no longer buttons but images, purchase card is now having one “are you sure” instead of two… If we have scripted testcases, those are immediately defectable upon every minor change.

Scripted testing is often praised as a way of having reliable regression tests. However, what is regression test in high level UI testing? We are not talking about low level unit tests where you test a method which is taking two strings and returning boolean, but about  functional testing of user interface. Charter can just take into consideration that login button form is with new version of software under test is in inline frame instead in pop up window. Functionality is not broken if UI has changed with change request. But scripted test is broken. One have to redesign it and this is task for which there is never time, so you have unnecessary flaw in your login script which will be unselectively copied to new version test suite or someone will have to redesign testcase and spend valuable time on maintenance rather than testing.