Test strategy and how to communicate it

I routinely come across test strategy documents when working with customers. They are lengthy—100 pages or more—and packed with monotonous text that is routinely reused from one project to another. Yawn once more— the test halt and resume circumstances, the defect management procedure, entrance and exit criteria, unnecessary generic risks, and in fact, one often-used model replicates the requirements of textbook testing, from stress to systems integration.

I have a number of issues with test documents like this. Mandatory papers are often mixed up with the things they are meant to represent in many corporate cultures. A test strategy document is a document, not a test strategy. It may or may not be used to communicate a plan. The larger and heavier a document is, and the more repetitive material it includes, the less likely it is to transmit anything meaningful from which readers may derive meaning. In actuality, most true test procedures may be properly stated in a handful of diagrams or a page or two of plain text.

There is a more fundamental issue at hand. A large test strategy document rarely contains a coherent strategy. That makes me wonder if we, as testers, truly grasp what “strategy” implies. I’ll return to the documentation concerns later. For the time being, accept that you must convey your test plan to project stakeholders in some form and negotiate their consent.

First and foremost, what is a test strategy?

It’s critical to begin by understanding what a strategy is and why you need one. This is sometimes a stumbling hurdle since few individuals truly understand what strategy is. Strategy is frequently mixed up with tactics, goals, and even actions. It derives from the French stratégie, the Greek stratgia “generalship,” and the Greek stratgos, which allude to military maneuvers and troop deployment.

The Oxford Dictionary defines strategy as:

“A plan of action designed to achieve a long-term or overall aim” “The art of planning and directing overall military operations and movements in a war or battle”

There are several definitions of what constitutes a test “strategy,” but here is the one I often choose when describing it. As the name implies, test strategy refers to the approach used to test a certain software application. A Test Strategy, in other terms, is an overview or strategy for how testing will be carried out across the software development life cycle. Its purpose is to specify the precise procedure that the testing team will use to fulfill the corporate goals from a testing standpoint.
If you want to go and check for one of the formal definitions, here is how it is described on wiki:

A test strategy is an outline that describes the testing approach of the software development cycle. The purpose of a test strategy is to provide a rational deduction from organizational, high-level objectives to actual test activities to meet those objectives from a quality assurance perspective. The creation and documentation of a test strategy should be done in a systematic way to ensure that all objectives are fully covered and understood by all stakeholders. It should also frequently be reviewed, challenged and updated as the organization and the product evolve over time. Furthermore, a test strategy should also aim to align different stakeholders of quality assurance in terms of terminology, test and integration levels, roles and responsibilities, traceability, planning of resources, etc.

Test strategies describe how the product risks of the stakeholders are mitigated at the test-level, which types of testing are to be performed, and which entry and exit criteria apply. They are created based on development design documents. System design documents are primarily used, and occasionally conceptual design documents may be referred to. Design documents describe the functionality of the software to be enabled in the upcoming release. For every stage of development design, a corresponding test strategy should be created to test the new feature sets.

Types of testing strategies

The following testing methodologies may be used as part of an organization’s testing strategy (there are a few more, but I will focus on the important ones):

• Standards or process-compliant strategy – Medical systems that adhere to US Food and Drug Administration (FDA) guidelines are ideal examples of this method. In this situation, the testers adhere to the protocols or recommendations set by the standards committee or a panel of industry experts to select test circumstances, develop test cases, and organize a testing team. In the case of a Scrum Agile project, testers will construct the entire test strategy, beginning with establishing test criteria, creating test cases, executing tests, reporting status, and so on, centered on each user story.
• Methodical approach– This method is essentially based on employing a pre-defined collection of testing methodologies that may be applicable to a certain sort of application testing. In this case, test teams adhere to an established quality standard (such as ISO 25000), checklists, or just a set of test circumstances. Specific types of testing (such as security) and application domains may have standard checklists.
• Analytical strategy – This strategy is focused on risk assessments or requirements-based testing on project needs and feedback from various stakeholders. A test strategy is designed based on the risk analysis to plan, design, and prioritize the testing activities. In the instance of requirements-based testing, the requirements are examined to determine the test circumstances. Then, to fulfill those criteria, tests are written, built, and performed. Even the outcomes of requirements are documented, such as those that were tested and passed, those that were tested but failed, and those that were not fully tested, and so on.
• Model-based approach – This method develops a test strategy using a variety of statistical models. The testing team selects a real-world or hypothetical scenario and models it, taking into consideration all relevant processes, inputs, outputs, and potential outcomes. Additionally, current software, hardware, data rates, infrastructure, etc. are taken into account while developing the models. Let’s think about the case of testing a mobile application. Models may be created to simulate incoming and outgoing traffic on mobile networks, the number of active/inactive users, predicted growth, etc. in order to carry out its performance assessment.

Overarching Goals

As you gain knowledge about the project and product, test-related concepts begin to form. What strategies will you employ, and do you already have a viable test design in mind? It’s improbable that you’ll be able to rigorously test everything in every method. Therefore, deciding how and on what basis to define the coverage boundaries is a crucial strategic choice. (This is arguably the most crucial concept that stakeholders need to comprehend.)

For instance, will you prioritize testing types and coverage depending on risk? What exactly do you mean by risk, and how are you going to recognize and evaluate those that testing might be able to reduce? Who are your key players? Practically speaking, what does product quality mean to them? How do they define the benefit they anticipate from the product? In most companies, the interests of certain stakeholders are more significant than those of others. Organizations and people have different risk appetites and risk perceptions. These are important factors in selecting what and how your test will cover.

Overarching facts, ethics, and emotions can either drive or dramatically reduce test priorities. As an example:

• The product will perform in a highly controlled environment. Certain sorts of bugs may result in the company’s executives being prosecuted.
• The product is a site that will be accessed by millions of visitors on a specified date as part of an event-based promotional campaign. If it breaks or crawls, your company’s reputation will deteriorate, and it will lose its most important and respected provider.
• The product is absolutely vital to the organization’s growth. Risks to the company’s publicly declared launch date will be disastrous for its reputation and share price.

Such value statements give the “why” for test strategy selections. Although it may seem apparent, defining project values upfront allows stakeholders to express whether you’ve based your strategies on the factors that are most important to them. Other strategy suggestions will come to you as you investigate the project context and identify risks, limits, and possible outcomes. It would make logical sense, for example, to utilize one or more high-level models, such as “revenue cycle” or “user experience lifecycle”? Will you utilize exploratory testing, pre-designed tests, or a combination of the two? How should major test procedures be sequenced for maximum test value?

How should the test strategy be communicated?

Negotiating consensus on important choices is essential for controlling stakeholder expectations through most projects. Stakeholders must be aware of any hurdles that may impede or prohibit vital testing since they will have to accept or eliminate them. Assume you discover that there is no option to do year-end period testing on a banking system, or you discover that the firm’s new restrictions prevent you from conducting Database queries on the central product database. If project schedules dictate that you can only undertake a cursory once-over feature confirmation on a complicated application, state this clearly and allow stakeholders time to reply.

Instead of adopting a template for a strategy document to inform stakeholders, look for a simple format that will effectively communicate the key points without being overly detailed. It might be the method you’ve been using to arrange your thoughts. A mind map, sticky notes, and presentation slides were all used. The organization with whom I was working and the nature of the test influenced my decision. Whatever format you use, it should serve as a portable solution for engaging stakeholders or stimulating conversation in meetings.

This is not to suggest that your current template is meaningless, but it most likely demands things you don’t need while ignoring the ones you need. You may be asked to complete the blanks in order to secure it in the vault. However, don’t begin there. A large document template is an ineffective tool for encouraging discussion and achieving consensus on a strategy.

Thinking is more important than writing.

The complexity and risks of what you have to deal with, as well as how much has already been understood and accepted by the project, will determine what you have to do to develop your test strategy. Whether it takes a couple of hours or four hundred, you will know your stakeholders and which values and ideals are crucial to your test at the conclusion of this process. You will have specified the high-level boundaries and methodologies for the test, and you will be able to attract the attention of stakeholders to any risks, challenges, or restrictions that would prevent testing within those boundaries, in addition to any assertions you feel comfortable expressing.

But don’t get overly attached to your strategy or foster the notion among your stakeholders that it’s fixed in stone. A test strategy might alter dramatically as the project evolves and you learn more about the product. “Thinking” is the essential word here. Consider each test project as a new adventure to complete, and understand that you will be learning and solving obstacles until the project is completed. Even if you’re testing a well-known product on a familiar road, try taking a step back and adopting a fresh strategic perspective. It might spark innovative thinking that can enhance your testing and save you money and effort for your business.

Don’t miss out on our comprehensive Analytical Test Strategy Tutorial: Your ultimate resource for mastering analytical testing. Discover best practices, from basics to advanced methods, ensuring precise data-driven decisions.

David Tzemach

The founder and owner of the Agile Quality Made Easy blog (25K followers). A platform that he uses for teaching and coaching others, sharing knowledge with people, and guiding them towards success while giving them the inspiration and tools to discover their own path.

See author's profile