Exploratory Testing - Its importance in an Agile environment

Software development process is witnessing an era of agility and a lot has changes in how software is developed, tested, and delivered. Agile and DevOps strategies have become the core mantra for delivering quality products in limited time span. The development cycles and release cycles have shortened, and today releases happens weekly, daily or even every now and then whenever a developer commits a code change. Modern teams achieve this speed with various agile practices, continuous Integration/Continuous delivery, Continuous testing and so on. These practices have changed the testing culture as well. There is more stress for automation in the agile and DevOps environments. Of course, test automation is necessary to keep in pace with the fast delivery approach. But, does more automation always ensure better quality software? Well, sadly the answer is “No”.

Of course, there has been lot of developments in artificial intelligence and machine learning too. We know that AI and machine learning can help build machines capable of doing tasks that humans can do multiple times faster and without errors and there are many automation testing tools as well these days utilizing AI which makes test automation easier and fast taking a lot of load off the testers. But, there is no machine yet developed that can beat the thinking power of humans as of date. Strong AI is not a reality yet. That is where exploratory testing still becomes one of the crucial techniques for quality assurance of software. Exploratory testing, rather a very old technique had almost lost its value in the times when automation wave swept over the testing industry. But with the agile movement, it has got a new life and is gaining acceptance as part of a broader “Agile Testing” skill set.

In an agile or DevOps project, testers often write automated tests based on the acceptance criteria and ensure it is working as expected. But you tend to forget about the big picture mainly because of time constraints and we may miss some scenarios. Remember ‘automating tests’ also takes time.

The solution to above challenges is manual testing. What? How can manual testing be a time saver and fit into an agile/DevOps environment when the whole world is behind automation and artificial intelligence? When I say manual testing, it is not the old school manual testing, where you follow a script and execute the mentioned steps. I mean real testing has to be performed by exercising interesting and thoughtful scenarios. That is called exploratory testing. Exploratory testing is a great way to actually “explore” the application and catch bugs that your automation test suite would have missed.

In this blog we will be exploring more about what is exploratory testing, how is it done, its benefits and some tools that can help for the same.

What is Exploratory Testing and how is it different from traditional manual testing?

Exploratory testing as the name implies is all about exploring the application under test. It is about discovery, investigation and learning. An exploratory tester needs to explore the application to discover what the software does and doesn’t do. There is no need for the testers to follow a test script or pre determined input definitions. This is the main difference between traditional manual testing and exploratory testing. Unlike in traditional testing, where the entire testing process happens in different phases (test analysis, test design, test execution, etc) in exploratory testing, learning, test design and test execution happens simultaneously.  The tester just learns about the application and uses that information to design and execute tests. In this process, the tester will be able to respond to each test in a way that would maximize focus on the relevant risks.

Thus exploratory testing relies more on tester’s knowledge of application, his mindset, etc. Instead of following the ‘usual path’ covered in scripted testing, the tester could imagine alternate paths where users do unexpected things.

Benefits of exploratory testing in Agile Environment

Exploratory testing depends on less documentation, emphasizes individuals and interactions over tools and processes, and values the tester as an integral part of the test process. These principles if you note are matching with the principles of agile methodology. Here are some benefits of exploratory testing that makes it ideal for testing in an agile environment where the focus is on delivering quality software at a fast pace.

• Exploratory testing requires less test preparation.
• Exploratory testing enables real time decision making and enables continuous and faster feedback which really helps in meeting agile schedule pressures.
• Exploratory testing helps find bugs that automation tests miss as the tester explores the application and thinks differently in order to identify critical bugs.
• Exploratory testing is helpful in testing dynamic product environment when product is unstable, undergoing changes or at early development stages. Since the approach doesn’t require much documentation, testers can focus on the testing activities based on the changes easily.

Tools for Exploratory Testing

Many often relate exploratory testing to ad-hoc testing which is ‘last minute testing of the application’ which is often carried out when the release date approaches and considers it to be an unorganized way of testing. But exploratory testing is really different. It is an approach which gives importance to tester’s creativity and gives him freedom to explore the software differently in order to identify any potential bugs or risks. It is not a purely unstructured way of testing as many think. It can be done in a structured way and managed using charters, time boxes or sessions.

Even though, exploratory testing does not demand too much documentation and test scripts, the tester has to be clear on the testing objectives and need to document the test steps to recreate an issue or the test steps. This could be done by different mechanisms like taking notes, taking screenshots or even screen recording. There are many cool tools available these days to help you plan and manage your exploratory testing sessions as well as record and generate reports easily. Given below is a list of few of the tools that comes handy for exploratory testing.

JIRA Capture: This tool from Atlassian allows you to create and record exploratory testing sessions for a faster and more efficient review process.

Rapid Reporter : Rapid Reporter is a free Session Based Test Management (SBTM) testing tool. It is basically a note taking application that aids in the reporting process of SBTM – Session Based Test Managements.

qTest  Explorer: qTest is one of the most sophisticated exploratory testing tools these days built for agile software development. It helps you by automatically documenting your results while you can focus on testing and provides detailed bug reports with high impact screen captures, environment details, and steps to reproduces. Furthermore, qTest Explorer’s intelligent capturing engine allows testers to instantly turn a test session into an automated Selenium or Protractor test script.

Bug Magnet: Bug magnet is a web based extension available for Chrome and Firefox browser and works as an Exploratory testing assistant. It adds common problematic values and edge cases to the context menu for editable elements, so you can keep them handy and access them easily during exploratory testing sessions.

Defect Scribe:Defect Scribe is an automated screen capture tool compatible with Microsoft Windows, Java, Qt, Flash, and browser- based applications. This enables testers to perform exploratory testing without having to stress over documenting their every action.

That was all about exploratory testing! It definitely is an approach which esters need to give more importance. When implemented in the right way and with usage of right tools and skilled testers who is capable of thinking in the right direction, exploratory testing can take testing to higher levels and assure better quality. Coupling automation testing with exploratory testing in agile environments will be really beneficial for achieving your testing goals.