Exploratory testing – Building skills through science, creativity, and intuition

QA testers must continuously learn and build their testing skills. The testing process changes constantly with ongoing speed to release pressure. The problem remains that defects find their way into every release seemingly regardless of the amount of time spent testing with formal manual and automated test scripts.

Traditional testing has changed with Agile, but Agile testing still needs improvement in overall release quality. How can a tester be fast, efficient, and find bugs before the release to customers? How best to continuously find bugs and protect customers from ever experiencing even minor, yet annoying defects? In the modern software application development world customer experience is critical. Defects whether severe, minor, or the simply annoying aren’t tolerated long with a customer’s ability to switch apps nearly at will. So, how best to build testing skills using creativity, science, and testing intuition?

Key Takeaways:

• What does exploratory testing entail?
• Learn the skills needed to be productive and efficient at exploratory testing.
• Is it simply random testing, or is there a method to the madness?
• Find out how to get started and what techniques to use.
• How to blend science with creativity and intuition to find defects.
• Build automated and manual testing skills using exploratory testing.

Think about enhancing quality and QA skills by embracing exploratory testing. Using exploratory testing techniques, QA testers use their technical experience and understanding of human behavior to test. Exploratory testing works perfectly with Agile development as well as other methodology types – be brave and embrace creative testing.

Exploratory testing works well with Agile because testing starts as soon as the code is in a testable-enough state. Beginning with design, and requirements or acceptance criteria review, and through to the theoretical end of the development cycle, exploratory testing is continuously effective.

What is exploratory testing exactly?

Exploratory testing involves combining planning, testing, and execution into a rapidly occurring set of simultaneous events. The exploratory testing effort may be planned out by creating a team “charter.” The team charter outlines what areas or functions are tested and by whom. Regardless of whether you are a part of a small or large testing team or are the lone tester creating a charter, it essentially defines the test coverage. Charters provide test coverage documentation for management, and other development team members, or serve as release test documentation for regulatory requirements or basic release tracking.

The charter also defines the general style of testing. For example, are you using testing tours, or simply taking each major function and creating tests as you go – in other words, exploring the application as a new user, or an experienced user looking to try a new feature.

Exploratory tests may be developed into written scripts, checklists, done on the fly, or executed based on design, user story description, requirements, or the development documentation. The many beauties or benefits of exploratory testing are flexibility, speed, efficiency, effectiveness, and a do-as-you-will approach.

Exploratory testing requires the will to explore, literally, an application in totality. The only requirements are access to the application and functions for testing. An understanding of the application’s purpose, design, and expected results are helpful but not necessary. Testers simply need a sense of adventure and willingness to think outside the script.

Exploratory testing – Getting started

A tester’s skill with exploratory testing comes from experience, understanding of the application, and a knack for asking questions of developers. I call it investigative testing.

The tester investigates design and function by:

• Talking with developers and asking where code has changed.
• Ask where developers believe high-risk or complex code lives.
• Get actively involved early, attending design reviews, debates, and developer code reviews if feasible.
• Essentially, understand as much as possible about the code design and intended functions from the developer’s view.
• Discuss the requirements or user story acceptance criteria with the product manager to gain a full understanding of the intended application function from the customers’ and product team’s point of view.

Hold on to the results of the investigation. Next, map out the application functionality needing testing keeping the following key points in mind:

• Go beyond the “happy path” – look for the unhappier paths through the application.
• Don’t simply confirm the application’s functions – test to break the code.
• Review your testing map. Discover gaps in functionality, otherwise known as “holes” or points of no return.

Once you’ve done your investigation and understand how the application intends to function, edit the test map or mind map for accuracy.

Mind mapping application functionality effectively generates a brainstorm of ideas on where to test. You’ll discover back-end processes, and often forgotten vulnerabilities in third-party applications, database connections, and integrated messaging systems. Even the simplest applications be it mobile or web, connect with another API or a database.

If mind mapping seems non-technical, the same information may be captured effectively with a workflow diagram. Whichever tool helps to best visualize the application, use it and compare it to any documentation from the development or the product team. Find any differences? Typically, gaps occur as well as contradictions. Mapping or workflow charting makes gaps, contradictions, and missing functions visible for effective testing.

Give yourself time to dig around and find the gaps. Analyze where and how the application pulls or pushes data to the database, or other connected applications. Check when and how the page data refreshes. If the application allows a user to generate email or SMS messages externally or internally, test if the functionality works as expected. Connection issues with third-party messaging systems are plentiful — see how many you can find that are not documented.

Many testers continue by writing out stories or testing tours through the application functionality. If time exists, writing out testing flows works. I prefer to save time and test from my mind map or flow diagram. The important part is to identify variable paths for single users through the application’s functions in expected and unexpected ways.

Use your own experience to test beyond the Happy Path

Exploratory testers find defects by using their own app experiences. We’ve all found defects in banking, retail, and game apps. How did you find them? You find them by trying to perform a function in an unpredicted way. Use the ability to mess up apps to your advantage and look for similar defects when testing applications.

For example, in my experience I find many documentation software applications handle bulleted and numbered lists poorly. I have struggled more with editing lists than any other feature. Often it’s easier to wipe out the list, save the document, and then put a new one in rather than try to edit the existing list. Typically, the text is off a line somehow, a carriage return or space throws off the entire list making editing impossible.

Another example, logging into a banking application and checking the box next to Remember me or the Remember this device for 30 days. These login options are a bad idea for security reasons, and most of these features simply don’t remember anything even when cookies are enabled. I use several banking and HR applications that simply never remember me, even before or after I clear the cache and cookies.
Collect defects found when using other apps and leverage the knowledge by testing them in the application under test. If friends or family bring up defects in apps, try those as well. Each individual’s negative experience with an app represents a gold mine of exploratory test information.

Introducing Test At Scale — a test intelligence and observability platform that shortens the testing durations & provides faster feedback. It enables devs to get early insights into flaky tests and enhances their productivity to ship code frequently with confidence. Start FREE Testing

Building testing skills

As a QA tester, the more one explores the entire depth and breadth of the applications, the faster testing skills and instincts improve. The science of exploratory testing revolves around learning, exploring, and fearlessly disturbing all the code aspects one can access.

Over time, testers learn the exploratory testing approach that helps them visualize missing functions, gaps in functionality or spot application design weaknesses. By continuously exploring, QA testers discover defects both superficial and those deeply embedded and persistent in the backend processing code the application relies on.

Up to now, the focus has been on manual testing methods for exploratory testing. What about building skills in test automation while exploratory testing? There is no reason test automation tools cannot be leveraged for exploratory testing. For example, codeless automation tools require no programming knowledge and are useful for creating automated test suites.

One could take their mind map, flow chart, or written descriptions of application functionality and create automated tests using them. Test automation works best in small functional groups or modular-style testing. However, one can simply break the visual tool into functional modules and automate each.

Prepare for frequent failures in test execution. When using automation for exploratory testing, expect frequent failures and re-work. Remember the purpose of exploratory testing is to find defects. Instead of manually walking through tests, use the automation tool and record modular tests and execute them repeatedly as tests are completed.

If tours or longer workflows are used for exploratory testing then attempt to automate full end-to-end paths through the functionality. See how far through the full workflow the automated tool gets. Best case scenario, the result is an automated test suite covering certain workflow paths. Such tests are handy for smoke testing execution or adding to automated test suites executed against each release or code build.

Adding in science, creativity & intuition to build testing techniques

Creativity in test execution is an underrated QA tester skill, but extremely valuable in any type of testing. For success in exploratory testing, creative testing becomes essential. How can a tester use creativity, science, and intuition to build exploratory testing skills?

As previously discussed, don’t disregard using personal application experiences and gleaning them for testing options. An annoying app, login defect, or failed functionality from personal use becomes a gold mine of information when exploratory testing applications. Fearlessly test outside the expected, or happy path and create creative test scenarios. The beauty of creative testing is not having to execute the same test over and over again. Creative tests find more defects that customers won’t need to experience.

What about science? Use psychology to analyze developer and product team strengths and weaknesses. Analyze team members to understand common behaviors. When working with a development team over time, learn the mistakes team members tend to make. Some may work fast and get a lot of code checked in, but did they test it? Is there a unit test for it? When testing a code of theirs, how often does testing reveal a similar type of defect? Developers that don’t test their code, or after merging to the main code branch produce defects not seen in their local environment. Experience with developers allows a tester to use exploratory testing to quickly find defects without digging deeply.

Analyze yourself as well. Do you tend to test quickly? Or do you test too much? Understanding personal tendencies, strengths, and weaknesses assist in finding defects.

For example, development teams experience communication issues frequently especially when application code stretches between integration application systems. When one team changes a function in the backend, it breaks another team’s code. Integration testing using exploratory methods yields frequent defects. In another example, one team changes API code, and the application fails in another team’s area. Integration issues are a boon for finding defects by exploring the UI and backend processing.
Exploratory testing methods depend on fearless tester creativity. An exploratory tester finds the alternative answer, goes the wrong way, performs functions incorrectly, goes backward, and clicks buttons repeatedly. Exploratory testing tests outside the box and uncovers problems in areas otherwise untested. Be fearless, like an end-user who knows no bounds and doesn’t read instructions, but simply goes for it to get their work done.

Also, check out this tutorial to learn about Pair Testing and its role in improving teamwork and testing quality!

Amy E Reichert

Amy E Reichert is a Freelance writer for a variety of topics focused mainly on QA testing, Agile, and technology trends. Amy has 23 years of professional experience as a QA Engineer/Analyst within the ERP, healthcare, and business management sectors. Many years of developing test process, leading diverse and inclusive teams as well as testing on mobile and web applications.

See author's profile