Test Management Unit 04

Test Plan

 A Test Plan is a detailed document that describes the test strategy, objectives, schedule, estimation,
deliverables, and resources required to perform testing for a software product. Test Plan helps us determine
the effort needed to validate the quality of the application under test.
 The test plan serves as a blueprint to conduct software testing activities as a defined process, which is
minutely monitored and controlled by the test manager.
 As per ISTQB definition: “Test Plan is A document describing the scope, approach, resources, and schedule of
intended test activities.”
A TEST PLAN is a document describing software testing scope and activities. It is the basis for formally testing
any software / product in a project.
Test Plan is a dynamic document. The success of a testing project depends upon a well-written Test Plan
document that is current at all times. Test Plan is more or less like a blueprint of how the testing activity is
going to take place in a project.
Given below are a few pointers on a Test Plan:
#1) Test Plan is a document that acts as a point of reference and only based on that testing is carried out
within the QA team.
#2) It is also a document that we share with the Business Analysts, Project Managers, Dev team and the other
teams. This helps to enhance the level of transparency of the QA team’s work to the external teams.
#3) It is documented by the QA manager/QA lead based on the inputs from the QA team members.

What is the Importance of Test Plan?

 Making Test Plan document has multiple benefits
 Help people outside the test team such as developers, business managers, customers understand the details of
testing.
 Test Plan guides our thinking. It is like a rule book, which needs to be followed.
 Important aspects like test estimation, test scope, Test strategy are documented in Test Plan, so it can be
reviewed by Management Team and re-used for other projects.

Components of a Test Plan

Scope: Details the objectives of the particular project. Also, it details user scenarios to be used in tests. If
necessary, the scope can specify what scenarios or issues the project will not cover.
Schedule: Details start dates and deadlines for testers to deliver results.
Resource Allocation: Details which tester will work on which test.
Environment: Details the nature, configuration, and availability of the test environment.
Tools: Details what tools are to be used for testing, bug reporting, and other relevant activities.
Defect Management: Details how bugs will be reported, to whom and what each bug report needs to be
accompanied by. For example, should bugs be reported with screenshots, text logs, or videos of their
occurrence in the code?
Risk Management: Details what risks may occur during software testing, and what risks the software itself may
suffer if released without sufficient testing.
Exit Parameters: Details when testing activities must stop. This part describes the results that are expected
from the QA operations, giving testers a benchmark to compare actual results to.

How to create a Test Plan?

Creating a Test Plan involves the following steps:
1. Product Analysis
2. Designing Test Strategy
3. Defining Objectives
4. Establish Test Criteria
5. Planning Resource Allocation
6. Planning Setup of Test Environment
7. Determine test schedule and estimation
8. Establish Test Deliverables

What is Lifecycle
 Lifecycle in the simple term refers to the sequence of changes from one form to other forms. In a similar
fashion, Software is also an entity. Just like developing software involves a sequence of steps, testing also has
steps which should be executed in a definite sequence.

Software Testing Life Cycle (STLC)

Software Testing Life Cycle refers to a testing process which has specific steps to be executed in a definite
sequence to ensure that the quality goals have been met. In the STLC process, each activity is carried out in a
planned and systematic way. Each phase has different goals and deliverables. Different organizations have
different phases in STLC.
Software Testing Life Cycle (STLC) is a sequence of different activities performed during the software testing
process.

STLC Phases
There are different phases in STLC which are given below. The testing activities start from the Requirements
analysis phase and goes through all the phases one by one before completing with the Test cycle closure
phase.
 There are 6 STLC Phases in the STLC Lifecycle
 The entry criteria must be fulfilled before each phase can start
 The exit criteria should be fulfilled before exiting a phase
 Every phase has one or more deliverables that are produced at the end of the phase
 The phases are executed in a sequence

The 6 STLC Phases are given below:

1. Requirement analysis
2. Test Planning
3. Test case development
4. Environment Setup
5. Test Execution
6. Test Cycle Closure

 Each of the step mentioned above has some Entry Criteria (it is a minimum set of conditions that should be
met before starting the software testing) as well as Exit Criteria (it is a minimum set of conditions that should
be completed in order to stop the software testing) on the basis of which it can be decided whether we can
move to the next phase of Testing Life cycle or not.

Each of these stages has a definite Entry and Exit criteria, Activities & Deliverables associated with it.
What is Entry and Exit Criteria in STLC?
Entry Criteria: Entry Criteria gives the prerequisite items that must be completed before testing can begin.
Exit Criteria: Exit Criteria defines the items that must be completed before testing can be concluded
 You have Entry and Exit Criteria for all levels in the Software Testing Life Cycle (STLC)

Phase 1: Requirement Analysis

During this phase, feature requirements collected in the SDLC process are evaluated to identify testable
aspects. If necessary, testing teams may need to consult with stakeholders to clarify requirements. These
requirements can either be functional or non-functional, defining what a feature can do or it’s characteristics
respectively. The ability to automate testing is also evaluated during this phase.

Activities to be done in Requirement analysis phase are given below:

 Analyzing the System Requirement specifications from the testing point of view
 Preparation of RTM that is Requirement Traceability Matrix
 Identifying the testing techniques and testing types
 Prioritizing the feature which need focused testing
 Analyzing the Automation feasibility
 Identifying the details about the testing environment where actual testing will be done
 Deliverables (Outcome) of Requirement analysis phase are:
 Requirement Traceability Matrix (RTM)
 Automation feasibility report

Phase 2: Test Planning

 During this phase, the test strategy is outlined in a test plan document. This strategy includes tools needed,
testing steps, and roles and responsibilities. Part of determining this strategy is a risk and cost analysis and an
estimated timeline for testing.

Activities to be done in Test Planning phase are given below:

 Estimation of testing effort
 Selection of Testing Approach
 Preparation of Test Plan, Test strategy documents
 Resource planning and assigning roles and responsibility to them
 Selection of Testing tool

Deliverables (Outcome) of Test Planning phase are:

 Test Plan document
 Test Strategy document
 Best suited Testing Approach
 Number of Resources, skill required and their roles and responsibilities
 Testing tool to be used

Phase 3: Test Case Development

During this phase, test cases are created. Each case defines test inputs, procedures, execution conditions, and
anticipated results. Test cases should be transparent, efficient, and adaptable. Once all test cases are created,
test coverage should be 100%. Any necessary automation scripts are also created during this phase.

Activities to be done in Test Case Development phase are given below:

 Creation of test cases
 Creation of test scripts if required
 Verification of test cases and automation scripts
 Creation of Test Data in testing environment

Deliverables (Outcome) of Test Case Development phase are:

 Test cases
 Test scripts (for automation if required)
 Test Data

Phase 4: Test Environment Setup

During this phase, testing environments are configured and deployed. This phase may include a variety of
testing tools, including TestComplete, Selenium, Appium, or Katalon Studio. Sometimes, this phase also
includes setting up test servers. Once environments are deployed, smoke tests are performed to ensure that
environments are working as expected with all intended functionality.

Activities to be done in Test Environment Setup phase are given below:

 As per the Requirement and Architecture document the list of required software and hardware is prepared
 Setting up of test environment
 Creation of test data
 Installation of build and execution of Smoke testing on it

Deliverables (Outcome) of Test Environment Setup phase are:

 Test Environment setup is ready
 Test Data is created
 Results of Smoke testing

Phase 5: Test Execution

During this phase, features are tested in the deployed environment, using the established test cases. Expected
test results are compared to actual and results are gathered to report back to development teams.

Activities to be done in Test Execution phase are given below:

 Execution of Test Cases
 Reporting test results
 Logging defects for the failed test cases
 Verification and retesting of the defect
 Closure of defects

Deliverables (Outcome) of Test Execution phase are:

 Test execution Report
 Updated test cases with results
 Bug Report

Phase 6: Test Cycle Closure

This is the last phase of the STLC, during which a test result report is prepared. This report should summarize
the entire testing process and provide comparisons between expected results and actual. These comparisons
include objectives met, time taken, total costs, test coverage, and any defects found.

Activities to be done in Test Cycle Closure phase are given below:

 To evaluate the test completion on the basis of Test Coverage and Software Quality
 Documentation of the learning from the project
 Analyzing the test results to find out the distribution of severe defects
 Test Closure Report preparation

Deliverables (Outcome) of Test Cycle Closure phase are:

 Report of Test Closure

How to write a Test Plan

This means that the test plan conveys how testing will be performed at a particular level (such as system
testing or user acceptance testing), or for a particular type of testing (such as performance testing or security
testing).
The Test Plan (sometimes also referred to as a QA Test Plan) can be seen as the instruction manual or guide for
your testing effort. It describes the objectives of testing (what are you planning to verify and/or validate), the
scope of testing (what will and will not be tested), together with the general and sometimes detailed schedule
of the activities you want to perform (how and when are you testing).

You already know that making a Test Plan is the most important task of Test Management Process. Follow the
seven steps below to create a test plan as Analyze the product
1. Design the Test Strategy
2. Define the Test Objectives
3. Define Test Criteria
4. Resource Planning
5. Plan Test Environment
6. Schedule & Estimation
7. Determine Test Deliverables
 Step 1) Analyze the product
 How can you test a product without any information about it? The answer is Impossible. You must learn a
product thoroughly before testing it.
 The product under test is ABC banking website. You should research clients and the end users to know their
needs and expectations from the application
 Who will use the website?
 What is it used for?
 How will it work?
 What are software/ hardware the product uses?

Step 2) Develop Test Strategy

 Test Strategy is a critical step in making a Test Plan in Software Testing. A Test Strategy document, is a high-
level document, which is usually developed by Test Manager. This document defines:
 The project’s testing objectives and the means to achieve them
 Determines testing effort and costs
 For project, you need to develop Test Strategy for testing that banking website. You should follow steps below

Step 2.1) Define Scope of Testing

Before the start of any test activity, scope of the testing should be known. You must think hard about it.
 The components of the system to be tested (hardware, software, middleware, etc.) are defined as "in scope"
 The components of the system that will not be tested also need to be clearly defined as being "out of scope."
Defining the scope of your testing project is very important for all stakeholders. A precise scope helps you
 Give everyone a confidence & accurate information of the testing you are doing
 All project members will have a clear understanding about what is tested and what is not
How do you determine scope your project?
To determine scope, you must –
 Precise customer requirement
 Project Budget
 Product Specification
 Skills & talent of your test team
Now should clearly define the "in scope" and "out of scope" of the testing.
 As the software requirement specs, the project ABC Bank only focus on testing all the functions and external
interface of website ABC Bank (in scope testing)
 Nonfunctional testing such as stress, performance or logical database currently will not be tested. (out of
scope)

Step 2.2) Identify Testing Type

A Testing Type is a standard test procedure that gives an expected test outcome.
Each testing type is formulated to identify a specific type of product bugs. But, all Testing Types are aimed at
achieving one common goal “Early detection of all the defects before releasing the product to the customer”
The commonly used testing types are described as following figureCommonly Used Testing Types
 There are tons of Testing Types for testing software product. Your team cannot have enough efforts to handle
all kind of testing. As Test Manager, you must set priority of the Testing Types
 Which Testing Types should be focused for web application testing?
 Which Testing Types should be ignored for saving cost?

Step 2.3) Document Risk & Issues

Risk is future’s uncertain event with a probability of occurrence and a potential for loss. When the risk actually
happens, it becomes the ‘issue’.

Risk Mitigation

Team member lack the required skills for website Plan training course to skill up your members
testing.

The project schedule is too tight; it's hard to complete Set Test Priority for each of the test activity.
this project on time

Test Manager has poor management skill Plan leadership training for manager

A lack of cooperation negatively affects your Encourage each team member in his task, and
employees' productivity inspire them to greater efforts.

Wrong budget estimate and cost overruns Establish the scope before beginning work,
pay a lot of attention to project planning and
constantly track and measure the progress
Step 2.4) Create Test Logistics
In Test Logistics, the Test Manager should answer the following questions:
 Who will test?
 When will the test occur?
Who will test?
You may not know exact names of the tester who will test, but the type of tester can be defined.
To select the right member for specified task, you have to consider if his skill is qualified for the task or not,
also estimate the project budget. Selecting wrong member for the task may cause the project to fail or delay.
Person having the following skills is most ideal for performing software testing:
 Ability to understand customers point of view
 Strong desire for quality
 Attention to detail
 Good cooperation
In your project, the member who will take in charge for the test execution is the tester. Base on the project
budget, you can choose in-source or outsource member as the tester.
When will the test occur?
Test activities must be matched with associated development activities.
 You will start to test when you have all required items shown in following
 Test Specification
 Human Resources
 Test Environmnet

Step 3) Define Test Objective

 Defining the test objectives should be the ultimate goal of achieving the test executions. The main goal of
testing is to find as many software bugs as possible, to ensure that the software is free from all the bugs. To
interpret and document test objective, you will need to follow 2 steps;
1. List down all the features and functionality of the system including its performance and user interface.
2. Identify the target or the end result based on the above features.

Step 4) Define Test Criteria

 Test Criteria is a standard or rule on which a test procedure or test judgment can be based. There’re 2 types of
test criteria as following
Suspension Criteria
Specify the critical suspension criteria for a test. If the suspension criteria are met during testing, the active
test cycle will be suspended until the criteria are resolved.
Test Plan Example: If your team members report that there are 40% of test cases failed, you should suspend
testing until the development team fixes all the failed cases.

Exit Criteria
It specifies the criteria that denote a successful completion of a test phase. The exit criteria are the targeted
results of the test and are necessary before proceeding to the next phase of development. Example: 95% of all
critical test cases must pass.
Example: 95% of all critical test cases must pass.
Some methods of defining exit criteria are by specifying a targeted run rate and pass rate.
 Run rate is ratio between number test cases executed/total test cases of test specification. For example, the
test specification has total 120 TCs, but the tester only executed 100 TCs, So the run rate is 100/120 = 0.83
(83%)
 Pass rate is ratio between numbers test cases passed / test cases executed. For example, in above 100 TCs
executed, there’re 80 TCs that passed, so the pass rate is 80/100 = 0.8 (80%)
This data can be retrieved in Test Metric documents.
 Run rate is mandatory to be 100% unless a clear reason is given.
 Pass rate is dependent on project scope, but achieving high pass rate is a goal.

Step 5) Resource Planning

 Resource plan is a detailed summary of all types of resources required to complete project task. Resource
could be human, equipment and materials needed to complete a project
 The resource planning is important factor of the test planning because helps in determining the number of
resources (employee, equipment…) to be used for the project. Therefore, the Test Manager can make the
correct schedule & estimation for the project.
 This section represents the recommended resources for your project.
Human Resource
The following table represents various members in your project team
No. Member Tasks

1. Test Manager Manage the whole project

Define project directions
Acquire appropriate resources
 2. Tester Identifying and describing appropriate test techniques/tools/automation architecture
Verify and assess the Test Approach
Execute the tests, Log results, Report the defects.
Tester could be in-sourced or out-sourced members, base on the project budget
F20or the task which required low skill, I recommend you 0choose outsourced members
to save project cost.

3. Developer Implement the test cases, test program, test suite etc.

4. Test Administrator Builds up and ensures Test Environment and assets are managed and maintained
SupportTester to use the test environment for test execution

5. SQA members Take in charge of quality assurance

Check to confirm whether the testing process is meeting specified requirements

Step 6) Plan Test Environment

What is the Test Environment?
A testing environment is a setup of software and hardware on which the testing team is going to execute test
cases. The test environment consists of real business and user environment, as well as physical environments,
such as server, front end running environment.
How to setup the Test Environment

You should ask the developer some questions to understand the web application under test clearly. Here’re
some recommended questions. Of course, you can ask the other questions if you need.
What is the maximum user connection which this website can handle at the same time?
What are hardware/software requirements to install this website?
Does the user's computer need any particular setting to browse the website?

Step 7) Estimation and Scheduling

 In the test environment phase, the test manager has already used techniques to come to the conclusion of
estimating the project. Now along with the estimate its necessary you bind to the schedule of test planning.
 In the Test Estimation phase, suppose you break out the whole project into small tasks and add the estimation
for each task as below
 Then you create the schedule
 To create the project schedule, the Test Manager needs several types of input as below:
Employee and project deadline: The working days, the project deadline, resource availability are the factors
which affected to the schedule
Project estimation: Base on the estimation, the Test Manager knows how long it takes to complete the
project. So he can make the appropriate project schedule
Project Risk : Understanding the risk helps Test Manager add enough extra time to the project schedule to
deal with the risks

Step 8) Test Deliverables

 Test Deliverables is a list of all the documents, tools and other components that has to be developed and
maintained in support of the testing effort.
 There are different test deliverables at every phase of the software development lifecycle.

Test deliverables are provided before testing phase.

 Test plans document.
 Test cases documents
 Test Design specifications.
Test deliverables are provided during the testing
 Test Scripts
 Simulators.
 Test Data
 Test Traceability Matrix
 Error logs and execution logs.
Test deliverables are provided after the testing cycles is over.
 Test Results/reports
 Defect Report
 Installation/ Test procedures guidelines
 Release notes

Test Management
Test Management is a process of managing the testing activities in order to ensure high quality and high-end
testing of the software application.
The method consists of organizing, controlling, ensuring traceability and visibility of the testing process in
order to deliver the high quality software application.
It ensures that the software testing process runs as expected.
As suggested by its name, test management is a process of managing testing activities, such as planning,
execution, monitoring, and controlling activities. Test management involves several crucial activities that
caters to both manual and automation testing. With the assistance of this process, a team lead can easily
manage the entire testing team, while monitoring their activities, as well as paying close attention to various
details of SDLC.
 Test management, process of managing the tests. A test management is also performed using tools to manage
both types of tests, automated and manual, that have been previously specified by a test procedure.
Test Management Responsibilities:
 Test Management has a clear set of roles and responsibilities for improving the quality of the product.
 Test management helps the development and maintenance of product metrics during the course of project.
 Test management enables developers to make sure that there are fewer design or coding faults.

Test Management Phases

Test Management Process

Test Management Process is a procedure of managing the software testing activities from start to the end. The
test management process provides planning, controlling, tracking and monitoring facilities throughout the
whole project cycle. The process involves several activities like test planning, designing and test execution. It
gives an initial plan and discipline to the software testing process.

There are two main Parts of Test Management Process: -

 Planning
Risk Analysis
Test Estimation
Test Planning
Test Organization
 Execution
Test Monitoring and Control
Issue Management
Test Report and Evaluation

Planning

Risk Analysis and Solution

Risk is the potential loss (an undesirable outcome, however not necessarily so) resulting from a given action or
an activity.
Risk Analysis is the first step which Test Manager should consider before starting any project. Because all
projects may contain risks, early risk detection and identification of its solution will help Test Manager
to avoid potential loss in the future & save on project cost.

Test Estimation
 An estimate is a forecast or prediction. Test Estimation is approximately determining how long a task would
take to complete. Estimating effort for the test is one of the major and important tasks in Test Management.
 Benefits of correct estimation:
1. Accurate test estimates lead to better planning, execution and monitoring of tasks under a test manager's
attention.
2. Allow for more accurate scheduling and help realize results more confidently.

Test Planning
 A Test Plan can be defined as a document describing the scope, approach, resources, and schedule of
intended Testing activities.
 A project may fail without a complete Test Plan. Test planning is particularly important in large software
system development.
 In software testing, a test plan gives detailed testing information regarding an upcoming testing effort,
including:
 Test Strategy
 Test Objective
 Exit /Suspension Criteria
 Resource Planning
 Test Deliverables

Test Organization
Test Organization in Software Testing is a procedure of defining roles in the testing process. It defines who is
responsible for which activities in testing process. Test functions, facilities and activities are also explained in
the same process. The competencies and knowledge of the people involved are also defined however
everyone is responsible for quality of testing process.

Execution
1. Test Monitoring and Control
Monitoring
 Monitoring is a process of collecting, recording, and reporting information about the project activity that the
project manager and stakeholder needs to know.
 To Monitor, Test Manager does following activities
 Define the project goal, or project performance standard
 Observe the project performance, and compare between the actual and the planned performance
expectations
 Record and report any detected problem which happens to the project

Controlling
 Project Controlling is a process of using data from monitoring activity to bring actual performance to planned
performance.
 In this step, the Test Manager takes action to correct the deviations from the plan. In some cases, the plan has
to be adjusted according to project situation.
 While testing work is in progress or while the testers are executing the test plan, all these work progress must
be monitored. One should keep track of all this testing work. If test monitoring is done, then the test team and
test manager will get feedback on how the testing progress is?
 Using this feedback, the test manager can guide the team members to improve the quality of further testing
work. With the help of test monitoring, the project team will get visibility on the test results. It also helps to
know about test coverage.


2. Issue Management
In the life cycle of any project, there will be always an unexpected problems and questions that crop up. For an
example:
 The company cuts down your project budget
 Your project team lacks the skills to complete project
 The project schedule is too tight for your team to finish the project at the deadline.
Risk to be avoided while testing:
 Missing the deadline
 Exceed the project budget
 Lose the customer trust

3. Test Report & Evaluation

 The project has already completed. It’s now time for look back what you have done
 Test report helps to identify test coverage, quality of the developed product and the required process
improvements. We can decide ‘how much testing is required?’
 If enough testing is done, then we can submit this test report to the stakeholders or clients. So that they will
also get to know the quality of the product and have an idea of how much testing is performed on the product.
 Infrastructure testing
What Is Infrastructure?
 IT Infrastructure Ecosystem includes Operating Systems platforms (such as Windows, UNIX, Linux, macOS),
Computer Hardware platforms (such as Dell, IBM, Sun, HP, Apple), Internet platforms (such as Apache, Cisco,
Microsoft IIS, .NET), Data Management and Storage (such as IBM DB2, Oracle, SQL Server, MySQL) and
Enterprise Software Applications (such as SAP, Oracle, Microsoft).

What Is Infrastructure Testing?

Every software requires an infrastructure to perform its actions. Infrastructure testing is the testing process
that covers hardware, software, and networks. It involves testing of any code that reads configuration values
from different things in the IT framework and compares them to intended results.

It reduces the risks of failure. This testing incorporates testing exercises, procedures to guarantee that IT
applications and the fundamental infrastructure are tuned to deliver on execution, adaptability, unwavering
quality, accessibility, performance, and scalability. The aim is to test infrastructure between test environments,
test tools, and office environments.

#Infrastructure Testing Team

The infrastructure-testing team has a good bunch of knowledge related to this testing. They are also involved
with the Quality Assurance team. This team knows how to test IT infrastructure. This team knows how to
design test cases for this type of testing.

 When To Perform Infrastructure Testing?

There is an urgent need to perform this testing whenever any infrastructure-related changes are introduced.
Examples of such changes are:
1. Any new patch in the system is developed.
2. Any new system updates are experienced.
3. Any update in Operating System.
4. The database version/structure is upgraded.
5. When there is memory up-gradation for servers.
6. Implementation of the new tool.
7. Security fixes.
8. Software update.

What Are The Benefits Of Infrastructure Testing?

 The planned and exhaustive approach of Infrastructure Testing gives many benefits to a software product as
well as organizations.
 Few of the benefits are enlisted below:
1. Reduction in Production failures.
2. Improvement in defect identification before production execution. Upgrade the quality of infrastructure with
zero defect slippage to production.
3. Quickened test execution, empowering early go live.
4. It helps in annual cost savings in operations as well as in business.
5. Confirm that software works in a systematic and controlled procedure.
6. Reduction in downtime.
7. Improvement in quality of service.
8. Availability of stable environments.
9. Reduction in the cost involved in risks.
10. Better user experience.

Test people management

 A managers most important, and most difficult job is to manage people. You must lead, motivate, inspire and
encourage them.
 Sometimes manager will have to hire, fire, discipline or evaluate employees.
 Your job as a manager is to get things done. However it also means getting things done through others.
 People management is an integral part of any project management.
 People management also require the ability to hire, motivate and retain the right people.
 Testing projects present several additional challenges.

Test Process
Testing is not a single activity instead it’s a set of number of processes.
It include following activities.
[Link] Lining a Test plan
[Link] Case Specification
[Link] of Traceability Matrix

Test processes are a vital part of Software Development Life Cycle (SDLC) and consist of various activities,
which are carried out to improve the quality of the software product. From planning to execution, each stage
of the process is systematically planned and require discipline to act upon them. These steps and stages are
extremely important, as they have their own entry criteria and deliverable, which are combined and evaluated
to get expected results and outcomes.
Therefore, we can say that the quality and effectiveness of the software testing is primarily determined by the
quality of the test processes used by the software testers. Moreover, by following a fundamental test process,
testers can simplify their work and keep a track of every major and minor activity.

Fundamental test process divided into following steps

1) Test Planning and Control
2) Test Analysis and Design
3) Test Implementation and Execution
4) Evaluating exit criteria and Reporting
5) Test Closure activities

Base Lining a Test plan

 A test combines all the points into a single document that acts as an anchor point for the entire testing project.
 An organization normally arrives at a template that is to be used across the board. Each testing project puts
together a test plan based on the template.
 The test plan is reviewed by a designated set of component people in the organization.
 After this, test plan is baselined into configuration management repository.
 From then on, the baselined test plan becomes the basis for running the testing project.
 In addition, periodically any changes needed to the test plan templates are discusses among the different
stakeholders and this is kept current and applicable to the testing teams.

Test Case Specification

 Test Case Specification document described detailed summary of what scenarios will be tested, how they will
be tested, how often they will be tested, and so on and so forth, for a given feature. It specifies the purpose of
a specific test, identifies the required inputs and expected results, provides step-by-step procedures
for executing the test, and outlines the pass/fail criteria for determining acceptance.
 Test case specification has to be done separately for each unit. Based on the approach specified in the test
plan first the feature to be tested for this unit must be determined.
 The overall approach stated in the plan is refined into specific test techniques that should be followed and into
the criteria to be used for evaluation. Based on these the test cases are specified for testing unit.
 The two basic reasons test cases are specified before they are used for testing. It is known that testing has
severe limitations and the effectiveness of testing depends very heavily on the exact nature of the test case.
Even for a given criterion the exact nature of the test cases affects the effectiveness of testing.
 Constructing good test case that will reveal errors in programs is still a very creative activity that depends a
great deal on the tester. Clearly it is important to ensure that the set of test cases used is of high quality. As
with many other verification methods evaluation of quality of test case is done through “test case review” And
 for any review a formal document or work product is needed. This is the primary reason for having the test
case specification in the form of a document.

What is Traceability Matrix? (TM)

 A Traceability Matrix is a document that co-relates any two-baseline documents that require a many-to-many
relationship to check the completeness of the relationship.
 It is used to track the requirements and to check the current project requirements are met.

Requirement Traceability Matrix (RTM)

Requirement Traceability Matrix (RTM) is a document that maps and traces user requirement with test cases.
It captures all requirements proposed by the client and requirement traceability in a single document,
delivered at the conclusion of the Software devlopement life cycle. The main purpose of Requirement
Traceability Matrix is to validate that all requirements are checked via test cases such that no functionality is
unchecked during Software testing.

In Requirement Traceability Matrix or RTM, we set up a process of documenting the links between the user
requirements proposed by the client to the system being built. In short, it’s a high-level document to map and
trace user requirements with test cases to ensure that for each and every requirement adequate level of
testing is being achieved.

The traceability matrix is typically a worksheet that contains the requirements with its all possible test
scenarios and cases and their current state, i.e. if they have been passed or failed. This would help the testing
team to understand the level of testing activities done for the specific product.

Which Parameters to include in Requirement Traceability Matrix

Requirement ID
Requirement Type and Description
Test Cases with Status

Test Report
 Test Report is a document which contains a summary of all test activities and final test results of a testing
project. Test report is an assessment of how well the Testing is performed. Based on the test report,
stakeholders can evaluate the quality of the tested product and make a decision on the software release.
 For example, if the test report informs that there are many defects remaining in the product, stakeholders can
delay the release until all the defects are fixed.
 A test report is an organized summary of testing objectives, activities, and results. It is created and used to
help stakeholders (product manager, analysts, testing team, and developers) understand product quality and
decide whether a product, feature, or a defect resolution is on track for release.
 Test reporting is essential for making sure your web or mobile app is achieving an acceptable level of quality.
 test report as a document containing information about the performed actions (run test cases, detected bugs,
spent time etc.) and the results of this perfomance (failed/passed test cases, the number of bugs and crashes
etc.).

What does a test report contain?

 Project Information
All information of the project such as the project name, product name, and version should be described in the
test report. For example, the information of XYZBank project

Test Objective
As mentioned in Test planning , Test Report should include the objective of each round of testing, such as Unit
Test, Performance Test, System Test …Etc.

Test Summary
 This section includes the summary of testing activity in general. Information detailed here includes
 The number of test cases executed
 The numbers of test cases pass
 The numbers of test cases fail
 Pass percentage
 Fail percentage
 Comments

Defect
One of the most important information in Test Report is defect. The report should contain following
information
 Total number of bugs
 Status of bugs (open, closed, responding)
 Number of bugs open, resolved, closed
 Breakdown by severity and priority
 Like test summary, you can include some simple metrics like Defect density, % of fixed defects.
The project team sent you the Defect information as followin
 Defect density is 20 defects/1000 lines of code average
 90% defects fixed in total

You can represent the data as following graph

Tips to write a good test report

Test report is a communication tool between the Test Manager and the stakeholder. Through the test report,
the stakeholder can understand the project situation, the quality of product and other things.
The following scenario shows you why we need a good Test Report
You co-operate with outsourcing company, its tester after having performed Performance Testing of the
website Guru99 Bank, sends you a test report like this
The information of that report is too abstract. It does not have any detailed information. The stakeholder who
will read it might be slightly puzzled when they get it. They might ask or have following sets of questions: -
 Why did they not execute 30 TCs that remains
 What are these failed Test Cases
 Doesn't have any bugs description
To solve that problem, a good Test Report should be:
 Detail: You should provide a detailed description of the testing activity, show which testing you have
performed. Do not put the abstract information into the report, because the reader will not understand what
you said.
 Clear: All information in the test report should be short and clearly understandable.
 Standard: The Test Report should follow the standard template. It is easy for stakeholder to review and ensure
the consistency between test reports in many projects.
 Specific: Do not write an essay about the project activity. Describe and summarize the test result specification
and focus on the main point.

Recommending Product Release

 A test report is a description or explanation or justification the status of a test project.
 Based on test summary report, an organization can take a decision on whether to release a product or not.
 Ideally, an organization would like to release a product with zero defects.
 However, market pressures may cause the product to be released with the senior management is convinced
that there is no major risk of customer dissatisfaction.
 Such a decision should be taken by the senior manager after consultation with customer support team,
development team and testing team.

Executing Test Cases

 The prepared test cases have to be executed at the appropriate times during a project. As test cases are
executed during a test cycle, the defect repository is updated with,
1. Defect from earlier test cycles that are fixed in the current build.
2. New defect that get uncovered in the current run of tests.
[Link] test procedure to execute test suite and test cases.
4. confirm test, re test re – execution to fix
[Link] the result of test execution, status of test pass and fail.
6. Comparison of actual results with expected results.
[Link] occurrences and differences should be updated in report
[Link] suspension and resumption of further test cases also update traceability matrix
 During test design and execution, the traceability matrix should be kept current.
 As when test gets designed and executed successfully, the traceability matrix should be updated.

Collecting and Analyzing Metrics:

 When test are executed, information about test execution gets collected in test logs and other files. The basic
measurements from running the tests are then converted to meaningful metrics by the use of appropriate
transformation and formulae
 Percentage test cases executed= (No of test cases executed/ Total no of test cases written) X 100
 Passed Test Cases Percentage = (Number of Passed Tests/Total number of tests executed) X 100
 Failed Test Cases Percentage = (Number of Failed Tests/Total number of tests executed) X 100
 Blocked Test Cases Percentage = (Number of Blocked Tests/Total number of tests executed) X 100
 Fixed Defects Percentage = (Defects Fixed/Defects Reported) X 100
 Accepted Defects Percentage = (Defects Accepted as Valid by Dev Team /Total Defects Reported) X 100
 Defects Deferred Percentage = (Defects deferred for future releases /Total Defects Reported) X 100
 Critical Defects Percentage = (Critical Defects / Total Defects Reported) X 100
 Number of tests run per time period = Number of tests run/Total time
 Test design efficiency = Number of tests designed /Total time
 Test review efficiency = Number of tests reviewed /Total time
 Bug find rote or Number of defects per test hour = Total number of defects/Total number of test hours

Test Summary Report

 Test Summary Report is an important deliverable which is prepared at the end of a Testing project, or rather
after Testing is completed. The prime objective of this document is to explain various details and activities
about the Testing performed for the Project, to the respective stakeholders like Senior Management, Client,
etc.
 As part of Daily Status Reports, daily testing results will be shared with involved stakeholders every day. But
the Test Summary Report provides a consolidated report on the Testing performed so far for the project.
 test summary report is a formal document that summarizes the results of all testing efforts for a particular
testing cycle of a project / module or a sub module. Generally, test leads or test managers prepare this
document at the end of testing cycle. Some test managers prepares it at the end of project.

Contents of a Test summary report: