Without testing documentation, measuring QA quality remains challenging even for seasoned professionals. The need for more documentation is even more apparent when onboarding new QA engineers and scaling your product. With documentation, the value and purpose of quality assurance remain clear.
When quality is measured correctly, you can predict test outcomes and influence the testing process. High-level documentation, such as a test strategy and test plan, allows you to measure the quality of the QA effectively, and thereby also the quality of your product in general. A test strategy gives you greater control over the testing process and the testing budget.
In this post, we will define a test strategy as opposed to a test plan, discuss its components, give tips on how to write a solid test strategy document, and provide an example of a good test strategy document.
What is a Test Strategy?
A test strategy is high-level testing documentation that confirms the types or levels of tests that will be run on the product and describes the testing methodology used in the Software Development Life Cycle. It defines how a product will be tested, specifying the precise procedure or approach that QA specialists will use to achieve their objectives.
Test Strategy vs Test Plan
Some people may need clarification on test plans and test strategies. A test plan addresses test coverage, features that should and shouldn't be checked, as well as estimations, scheduling, and resource management. In a nutshell, the test plan is your goal-oriented vision, and the test strategy is your strategy for getting there.
A test plan is a written description of the scope and various steps involved in the testing procedure. A test plan document is dynamic and can be changed during development. Determining how to test a product, what to test it on, when to test it, who will test it, and who will verify the results is the main objective here.
A test strategy is a high-level document that covers test objectives, methodologies, environments, automation techniques, tools, and risk analysis with a contingency plan. Once the test strategy has been written and approved by the project manager and development team, it usually doesn’t change. There are following types of test strategies.
- Model-Based Strategy
- Methodical Strategy
- Standard Compliant or Process-Compliant Strategy
- Reactive Strategy
- Consultative Strategy
- Regression-Averse Strategy
Components of a Test Strategy
There isn't one perfect test strategy document to use as a standard and apply to all kinds of products. Therefore, the components of a test strategy may vary from company to company. We suggest looking at the Techstack test strategy document's components developed based on decades of experience in providing QA as a service.
- Testing approach
- Testing levels
- Testing types
- Compatibility testing priorities
- Impediment mitigation
- Testing phases
- Release verification
- Reporting
- Hotfix
- CI/CD testing pipeline
Testing Approach
This part describes the types of tests that will be used during product testing, the testing pyramid, and its stack. Also, it prioritizes such types of tests as compatibility testing, installation testing, etc. For convenience, we will break it into subsections.
Testing Levels
This is a pyramid-shaped description of the testing levels.
Testing Types
This includes a list of all types of testing that the team plans to conduct, their goals, process features for each type, and acceptance criteria. For example, for Smoke Testing, the goal would be to make sure that the core features are free of critical defects and to determine that the application is ready for subsequent testing phases.
Features of the process: smoke tests should take no more than 30 minutes, run after each new build, and be covered as much as possible by autotests.
Compatibility Testing Priorities
This part contains a list of components, or parts of the application, to which this type of test is applicable. It also includes a matrix of these same priorities, which may look like this for various browsers and operating systems.
If the testing process has nuances for other types of tests listed in the Testing Type (which need additional details), they should also be in a separate subsection.
Impediment Mitigation
It is a section that describes a list of potential problems a product may have with its quality, as well as the types of testing that aim to reduce these risks and their priority.
The real table will be larger than in the example above. Based on this table, we can break down our test types into priorities:
At Techstack, we classify the priorities according to the volume of running the corresponding tests in case of lack of time or other risks:
High - testing must be carried out in full.
Medium - testing can be carried out partially.
Low - testing will be performed if there is time left.
Testing Phases
Testing can be carried out at different phases depending on the development process. For example, when working with Scrum, you can break down the testing phases into those that occur before the sprint, during the sprint, acceptance testing, and after the production release.
For this process to be of predictable quality, each phase is formalized and described as input criteria, the testing process during the phase, and output criteria.
Release Verification
- Post-release testing is carried out in production as needed and under the guidance and supervision of the test lead.
- Only smoke testing is carried out without creating/updating or deleting any data.
- The team reports on the testing results, including issues found during the testing of the release for production.
Reporting
The team sends a test report to higher management and stakeholders, informing them about the outcomes of the testing process.
Hotfix
The issues discovered during the testing process should be fixed according to their criticality and SLA if they are defined as critical for the production environment. (Next, you can list the steps for working with a hotfix, testing it, and releasing it to production).
Exit Criteria:
- The smoke test was passed.
- Found defects are listed in the bug tracker.
- Critical defects are fixed as part of the hotfix process.
CI/CD Testing Pipeline
Continuous integration and continuous deployment (CI/CD) methodology is not applicable everywhere, but in some cases, QA as a part of your CI/CD pipeline can be useful. This diagram describes quality gates and serves as a starting point for configuring CI pipelines.
How to Prepare a Good Test Strategy Document
Instead of blindly following the templates, consider what will work best for your product. Every product has different specification, so you need to stick to what you know works best for you. Do not slavishly follow any standard or organization. Make sure it is always benefitting you and your procedures. Here are a few tips you might find helpful when preparing a test strategy document.
- When drafting the test strategy document, answer the question, "Why do stakeholders wish to build this product?" This will make it easier to understand and rank issues quickly.
- Make a list of all the critical components you plan to test. If you believe that a certain feature is not included in this version, list it under the "Features not to be tested" category.
- Write down a test approach for your product. Indicate the intended type of testing in clear terms. Examples include functional testing, user interface testing, integration testing, load/stress testing, security testing, etc.
- Describe your strategy for performing functional testing, for example. Manual testing or automation? Do you intend to run each test case in your test?
- Set clear entry and exit criteria for your test. When exit criteria are ambiguous, team members will interpret them differently, which will cause significant problems to be released. It follows that the product release will be delayed.
- Choose how you will monitor the results of your testing and what metrics you will employ to monitor test completion.
- Decide on the task distribution, specifying each team member's responsibilities and functions.
- What records will you generate both during and following the testing phase?
- Ensure that the entire team has a unified understanding of testing processes and can work effectively even when management is away for some time.
- The test strategy should provide PMs with a thorough understanding of what QA engineers are testing and how, the desired quality, and how much work will be performed by the testing team at each stage of product development without having in-depth knowledge of testing.
- A good test strategy helps predict product quality if partner organizations request documentation with a set of measures that the team takes to assess the predicted quality of the product.
- Meet data safety and security requirements to pass any audits and certifications required to access new markets.
- Set up a CI/CD process as a dedicated section of the test strategy document to sync up with the development team and regulate the quality stages each feature accomplishes before it is deployed to production.
- Take into account QA metrics that show you need a test strategy, such as defect detainment, reopen rate, estimation accuracy, decline rate, and mean time to defect.
Example of a Good Test Strategy Document
As we mentioned earlier, there is no single standard on how a good test strategy document should look. While every product might have various specifications that should be displayed in your test strategy document, we suggest you look into Techstack’s Test Strategy Document developed, taking into account many years of experience in software testing.
The Techstack Test Strategy Document includes all the components of the test strategy that we covered above. It consists of four major parts:
01. Testing Approach
02. Impediments and Mitigations
03. Testing Phases
04. CI/CD Testing Pipeline
Testing Approach
The Testing Approach section includes four tables (testing levels, test types, compatibility testing requirements for browsers and OS, and screen resolution for compatibility testing) and a subsection for performance testing.
In the Testing Levels table, we have two columns where we write down testing levels and descriptions, respectively. The Test Types table has four columns: test type, test objective(s), process details, and acceptance criteria.
In the Browsers and OS for Compatibility Testing table, the number of columns may vary based on the number of OS taken for compatibility testing. In Screen Resolution for Compatibility Testing, we write down the resolution, screen size, display ratio, and priority.
Impediments and Mitigations
In this section, there are two tables. In the first one, we have three columns where we write down product impediment, impact, and mitigation, respectively. The second table assigns priority to different testing types.
Testing Phases
In this section, the testing activities take place during the following phases:
3.1 Testing Activities During Sprint
3.1.1 Entry Criteria
3.1.2 Process Details
3.1.3 Exit Criteria
3.2 UAT and Post-UAT Activities
3.2.1 Entry Criteria
3.2.2 Process Details
3.2.3 Exit Criteria
3.3 Production and Post-Production activities
3.3.1 Entry Criteria
3.3.2 Process Details
3.3.3 Exit Criteria
CI/CD Testing Pipeline
The last section of the Techstack Test Strategy Document is where CI/CD Testing Pipeline methodology is applied optionally.
Control Your Product Quality with a Solid Test Strategy
A good test strategy gives you an instrument for storing and sharing the QA team’s knowledge, allowing you to grow your team and onboard newcomers without relying on particular team members. It gives a simple understanding of how to manage the testing process.
With a well-written test strategy document, you can make sure that your QA team understands what deliverables should be so that they provide value for a product. It allows you to control risks better and predict the quality of your product. Test strategy, along with other product documentation, helps to pass any audits and certifications seamlessly and expand your business by accessing new markets.
When Techstack implemented a Test Strategy Document across our product development teams, our teamwork became more visible and efficient. This allows Techstack team members to be aware of different areas of responsibility and where to send inquiries.
Contact us to find ways to ensure your product quality.
