Without a crystal clear understanding of the processes that take place when a team works on a software product, it can be tempting to think that all the problems stem from under-qualified QA engineers who click around randomly and ruin the hard work of the whole team. The value and purpose of the quality assurance process is not transparent without documentation. That's where a test plan and test strategy can help.
Even for those who are well aware of the processes, there's still the problem of measuring the quality QAs provide. If you don't measure quality, you don't have any control over the testing process or any ability to anticipate the results. How should you know what exactly you paid for?
A test plan is an estimate subcontract, which allows you to plan expenses and resources over a long period, plan budgets, analyze performance, and increase product value. Having QA documents contributes to a structured process, which is widely accepted as a best practice. Still, many teams skip this step in their products.
In this article, I’d like to discuss the benefits of maintaining a test plan and test strategy, as well as the most useful elements of each document for the team.
Test plan and test strategy: why you may need them
I have seen dozens of test plans and test strategies and can declare with confidence that there is no single correct, universal document that can be taken as a standard and applied to all types of projects.
The difference between test planning and test strategy is that test planning is focused on defining how testing will be performed for a specific project, while test strategy is focused on defining the overall approach to testing. Test planning is a tactical activity that happens at the project level, while test strategy is a more strategic activity that happens at the organizational level.
Test strategy vs test planning
Test strategy and test planning can exist either as separate documents that refer to each other, or the test strategy can be incorporated into the test plan. Since the test plan is updated frequently throughout the project, and the test strategy generally remains stable, it is often beneficial to keep them as separate documents.
Below is a list of sections that should be included in both the test plan and test strategy documents to ensure the team gets the most out of them.
Test plan
- Scope of work
- Quality & acceptance criteria
- Resources (team, tools, environments)
- Test documentation and deliverables
- Risk assessment
- Process description
Test strategy
- Testing approach
- Testing levels
- Testing types
- Compatibility testing priorities
- Impediment mitigation
- Testing phases
- Release verification
- Reporting
- Hotfix
- CI/CD testing pipeline
The value from the implementation of these documents affects the whole team in the broadest sense of the word, meaning everyone involved in product development. With the help of a test plan and strategy, it is easier to understand what a QA team does and coordinate work with other teams.
You need to grow your team and don't want to tie all product knowledge to individual team members
With a well-written test plan and test strategy, the team has a unified understanding of all testing processes on the project and can perform work efficiently even in the absence of management for some time. Additionally, high-level documentation helps to quickly get newbies up to date and synchronize the distributed team.
For example, onboarding normally includes the number of non-production hours (rate multiplied by hours), the manager's time for introductory meetings, and the time of other specialists who bring the newcomer up to date. Test documents allow you to reduce the number of hours that other departments' specialists spend on onboarding by prescribing parts of the processes in an industry-accepted format, which also saves time reading such documentation.
You want to design project timing
QA documents allow PMs to get a complete picture of what QA engineers are testing and how, the expected quality, and how much work will be done by the testing team at each phase of product development without having in-depth knowledge of testing.
You want to better control risks
The test plan unveils what the team is responsible for and what is not under its control. For example, it lists 3rd-party services and products, boundary cases that cannot be captured in the test environment, etc. This allows for management not only of risks but of expectations as well.
You want to predict product quality
Several times in my professional life, I’ve come across situations where the product I was managing became a partner with other major financial or medical products. Such organizations often request documentation that fully regulates product development (including risk management, business continuity plans, product development roadmaps, etc.) They also request to be shown what set of measures the team takes to obtain the predicted quality of the product. A well-written test plan and test strategy fully cover this request in most cases.
You want to access new markets and customers
The requirements for data safety and security of the entire process are very high in such industries as finance and healthcare. That's why they often require audits and certifications, which involve formalization of all processes of both the partner's team and the product development team.
Audits and certifications directly affect business expansion as they provide access to more clients. Investors can also request such audits in preparation for an IPO, as well as to conduct a Know-Your-Customer (KYC) procedure that ensures the company was never involved in crime or is not subject to any restrictions.
A test plan or test documentation may be required to cover the following gaps: development process, employees (resources), data integrity, business continuity, incident response, third-party reliance, operational resilience, infrastructure network, etc. (according to the Financial Planning Alliance).
You want to set up a CI/CD process and sync up with the development team
For setting up a CI/CD process, the test plan has a dedicated section that regulates the quality stages each feature accomplishes before it is deployed to production. This process is invisible to stakeholders and runs in an automatic mode, but it's crucial to ensure that everything goes smoothly. Using this test plan section, developers can find answers to questions about which cases the developed functionality will be tested in, return to development, specify the stage and environment, what level of unit test coverage is expected, and what quality gates will be configured when code flows from one environment to another.
Testing documentation: frequent mistakes
What (should be tested), when, who (will test), and with the help of what — if you cannot answer any of these questions, there are certainly gaps and a high risk to overlook bugs.
Here I will consider examples of test plan and test strategy of how incorrectly selected items or absence of correctly selected items can harm the project and what to look for in the test plan and test strategy.
Example #1
Your team has grown and it has become difficult to transfer information about processes. The number of issues has grown and all the development processes have slowed down. Despite the lack of project time, you will save a lot more time by introducing test documentation: newly joined QA engineers can read about your processes autonomously and come back to their description when necessary.
Example #2
Imagine the following situation: the project had vague exit criteria. Each team member interpreted them their own way, which resulted in major bugs crashing into the release. As a result, the product release had to be postponed for a few weeks, because the team logged the wrong severity level.
This was only an example of how incomplete quality criteria can affect product development. If you have already encountered such an issue, it’s time to think about formalizing quality and acceptance criteria via the test plan section.
Good release criteria examples:
- “All tickets in the release backlog are implemented, deployed, and tested in accordance with the acceptance criteria and ticket description.”
- “Complete set of manual and automated tests passed after code freeze.”
- “All E2E scenarios are completed.”
- “All failed scripts have been analyzed with bug reports added to the bug tracker.”
- “Compatibility tests passed in accordance with the first priority list of browsers and OS described in the test strategy.”
- “No open bugs with Critical or Major priority.”
Example #3
During testing, the team may encounter various problems that can affect timing or quality: dropped testing environments, a team member taking sick leave, unexpected code freeze, poor requirement details, changes in requirements when they are half-finished, and so on. The team should have an emergency plan to minimize the damage from the triggered risk and a plan of countermeasures to prevent such risks. This is why I think risk assessment is one of the most important sections of the test plan. It consists of a list of risks, their likelihood, impact on the testing process, priority, and a response plan.
Summary
While it may seem obvious, both a test plan and a test strategy are crucial—even if you believe you're performing well without them. Incorporating QA documents into your workflow will promptly reveal their significance in minimizing errors and misunderstandings. It's essential to grasp the difference between test plan and test strategy document.
The main difference between test strategy and plan is that a test plan is a detailed outline of the testing approach and scope for a specific project, while a test strategy is a higher-level document that outlines the overall testing approach and methodologies for an organization. The test plan includes information such as testing objectives, test environment, test schedule, and test deliverables, while the test strategy provides a broader view of the organization's testing approach.
Our teamwork at Techstack became more transparent and faster when we implemented test plans and test strategies in all our product development teams. You will always know responsibility areas and where to direct questions. You'll have a transparent process that’s easy to analyze and manage. Certainly, you’ll have to spend time preparing each software testing document, but they will save you many working hours.
Let's talk in more detail about managing QA teams! Contact us to learn the best practices our Software Testing Consulting Team can implement to ensure the smooth processes that will make your product a success.
