Software Testing
Standard Operating Procedures
Software Testing
Software Testing
Standard Operating Procedures
Proprietary Information
This document contains confidential information relating to FUSION Alliance, Inc., and FUSION Technologies, LLC (FUSION)
which is provided for the sole purpose of permitting the recipient to evaluate the document submitted herewith. In consideration
of receipt of this document, the recipient agrees to maintain such information in confidence and to not reproduce or otherwise
disclose this information to any person outside the group directly responsible for evaluation of its contents, except that there is
no obligation to maintain the confidentiality of any information which was known to the recipient prior to receipt of such
information from FUSION, or becomes publicly known through no fault of recipient, or is received without obligation of
confidentiality from a third party owing no obligation of confidentiality to FUSION.
Software Testing
Standard Operating Procedures
Revision History
Date
Version
Revision Description
Author
04/19/2002
1.0
Initial document
Mike Goempel
05/18/2002
1.1
Revised all sections and added Defect Management
Mike Goempel
2002 FUSION Alliance
ii
Software Testing
Standard Operating Procedures
Table of Contents
1.
2.
3.
4.
5.
6.
Discipline: Software Testing
1
1.1
Purpose......................................................................................................................... 1
1.2
Scope............................................................................................................................. 1
1.3
Workflow Detail.............................................................................................................. 1
SOP: Test Planning
2
2.1
Purpose......................................................................................................................... 2
2.2
Scope............................................................................................................................. 2
2.3
Workflow Detail.............................................................................................................. 2
2.4
Roles.............................................................................................................................. 2
2.5
Activities......................................................................................................................... 3
2.5.1 Activity: Develop Test Guidelines
3
2.5.2 Activity: Plan Test
4
SOP: Test Design
5
3.1
Purpose......................................................................................................................... 5
3.2
Scope............................................................................................................................. 5
3.3
Workflow Detail.............................................................................................................. 5
3.4
Roles.............................................................................................................................. 5
3.5
Activities......................................................................................................................... 6
3.5.1 Activity: Design Test
6
SOP: Test Implementation
7
4.1
Purpose......................................................................................................................... 7
4.2
Scope............................................................................................................................. 7
4.3
Workflow Detail.............................................................................................................. 7
4.4
Roles.............................................................................................................................. 7
4.5
Activities......................................................................................................................... 8
4.5.1 Activity: Implement Test
8
SOP: Test Execution and Evaluation
9
5.1
Purpose......................................................................................................................... 9
5.2
Scope............................................................................................................................. 9
5.3
Workflow Detail.............................................................................................................. 9
5.4
Roles............................................................................................................................ 10
5.5
Activities....................................................................................................................... 11
5.5.1 Activity: Execute Test
11
5.5.2 Activity: Evaluate Test
12
SOP: Defect Management
13
6.1
Purpose....................................................................................................................... 13
6.2
Scope........................................................................................................................... 13
6.3
Workflow Detail............................................................................................................ 13
6.4
Roles............................................................................................................................ 13
6.5
Activities....................................................................................................................... 14
6.5.1 Activity: Manage Defects
14
2002 FUSION Alliance
iii
Software Testing
Standard Operating Procedures
1.
Discipline: Software Testing
1.1
Purpose
The purpose of these SOPs is to guide software testing activities in order to ensure a standard
level of quality and repeatability on all software projects.
1.2
Scope
The scope of these SOPs is all software testing activities for all FUSION Alliance software
engineering projects.
1.3
Workflow Detail
Software Testing
2002 FUSION Alliance
Software Testing
Standard Operating Procedures
2.
SOP: Test Planning
2.1
Purpose
The purpose of this SOP is to guide the Test Manager through planning the testing activities on
software projects.
The purpose of test planning is to identify and describe software testing that will be conducted. This
purpose is accomplished by generating a Test Plan, which contains the requirements for test and
test strategies.
2.2
Scope
The scope of this SOP is all test planning activities for Fusion Alliance software engineering
projects.
2.3
Workflow Detail
Test Planning
2.4
Roles
Role
Test Manager
2002 FUSION Alliance
Responsibilities
Develops Test Guidelines
Software Testing
Standard Operating Procedures
2.5
Activities
2.5.1
Activity: Develop Test Guidelines
Purpose
The purpose of this activity is to develop software testing guidelines for a project.
Role: Test Manager
Timing/Frequency
Test Guidelines should be available before software testing begins. This activity is normally performed in the
Inception phase.
Input Artifacts
Output Artifacts
Test Guidelines (Organizational level)
Development Case
Test Guidelines (revised)
Steps
1.
Review the Development Case
2.
Review the organizational level Test Guidelines
3.
Tailor the Test Guidelines
Artifacts
Test Guidelines
2002 FUSION Alliance
Templates/
Examples
Tools
Microsoft Word
Ver.
When
Class.
1.0
Inception
Should
x.x
Elaboration
Could
x.x
Construction
Could
x.x
Transition
Could
Yes
Review
Type
Informal
Internal
Informal
Internal
Informal
Internal
Informal
Internal
Software Testing
Standard Operating Procedures
2.5.2
Activity: Plan Test
Purpose
The purpose of this activity is to identify the appropriate focus of the test effort for each iteration, and to gain
agreement with stakeholders on the corresponding goals that will direct the test effort.
Role: Test Manager
Timing/Frequency
A Test Plan must be created during the Inception phase and should be updated by the end of each iteration in the
Elaboration and Construction phases. The initial version of the Test Plan provides an overview of the test effort over
the life of the project, providing foresight into when resources will be required and when important quality dimensions
and risks will be addressed.
Input Artifacts
Output Artifacts
Iteration Plan
Software Development Plan
Use-Case Model
Design Model
Deployment Model
Test Plan
Steps
1.
Identify Requirements for Test
2.
Assess Testing Risks
3.
Develop Test Strategy
4.
Identify Testing Resources
5.
Create Test Schedule
6.
Define Test Environment Configuration
7.
Create the Test Plan
8.
Review the Test Plan
Artifacts
Test Plan
2002 FUSION Alliance
Templates/
Examples
Tools
Microsoft Word
Ver.
When
Class.
1.0
Inception
x.x
Elaboration
Should
x.x
Construction
Should
x.x
Transition
Could
Must
Yes
Review
Type
Formal
Internal
Formal
Internal
Formal
Internal
Formal
Internal
Software Testing
Standard Operating Procedures
3.
SOP: Test Design
3.1
Purpose
The purpose of this SOP is to guide the Test Manager through designing Test Cases. The purpose
of test design is to identify, describe and generate the test model and its reported artifacts (test
procedures and test cases). Test design is performed so test implementation and execution efforts
are efficient and effective.
3.2
Scope
The scope of this SOP is all test design activities for FUSION Alliance software engineering
projects.
3.3
Workflow Detail
Test Design
3.4
Roles
Role
Test Manager
2002 FUSION Alliance
Responsibilities
Designs test cases
Evaluates test coverage
Software Testing
Standard Operating Procedures
3.5
Activities
3.5.1
Activity: Design Test
Purpose
The purpose of design test is to identify a set of verifiable Test Cases for each build and test procedures that show
how the Test Cases will be realized.
Role: Test Manager
Timing/Frequency
The initial Test Cases are identified during in the Elaboration phase as soon as there are some use cases or
scenarios defined. The Test Cases are refined throughout the remainder of the lifecycle during each iteration.
Input Artifacts
Output Artifacts
Test Plan
Use Cases
Supplementary Specifications
Test Guideline
Iteration Plan
Software Architecture Document
Design Guidelines
Test Cases
Steps
1.
See RUP version 2001A.05.00
Artifacts
Test Cases
2002 FUSION Alliance
Templates/
Examples
Tools
Microsoft Word,
Rational
TestManager
Ver.
When
Class.
1.0
Inception
Could
x.x
Elaboration
Must
x.x
Construction
Must
x.x
Transition
Could
Yes
Review
Type
Informal
Internal
Informal
Internal
Informal
Internal
Informal
Internal
Software Testing
Standard Operating Procedures
4.
SOP: Test Implementation
4.1
Purpose
The purpose of this SOP is to guide the Test Analyst through implementing Test Scripts.
The purpose of implementing tests is to record, or generate the Test Scripts for the Test Cases that
were defined in Design Test. The generation of Test Scripts can be completed either with a test
automation tool or manually using the Test Case template.
4.2
Scope
The scope of this SOP is all test implementation for FUSION Alliance software engineering projects.
4.3
Workflow Detail
Test Implement
4.4
Roles
Role
Test Analyst
2002 FUSION Alliance
Responsibilities
Creates executable and reusable Test Scripts (automated or manually)
Creates Test Data
Performs Smoke Testing
Creates Test Suite
Software Testing
Standard Operating Procedures
4.5
Activities
4.5.1
Activity: Implement Test
Purpose
The purpose of implementing test is to create or generate reusable Test Scripts and to maintain traceability of the
test implementation artifacts back to the associated Test Cases and Use Cases or requirements for test.
Role: Test Analyst
Timing/Frequency
The initial Test Scripts are created during the test implementation activity as soon as there are some use cases or
scenarios implemented. The Test Scripts are modified and / or added to throughout the remainder of the lifecycle
during each iteration.
Input Artifacts
Output Artifacts
Test Cases
Test Scripts
Test Procedures
Test Cases (Revised)
Steps
1.
Record or generate test scripts
2.
Identify test-specific functionality in the design and implementation models
3.
Create test data
4.
Create test build for execution
5.
Perform smoke testing
6.
Trace test cases back to use case requirements
Artifacts
Test Scripts
2002 FUSION Alliance
Templates/
Examples
Tools
Microsoft Word,
Rational Robot,
Rational
TestManager
Ver.
When
Class.
1.0
Inception
Could
x.x
Elaboration
Should
x.x
Construction
Should
x.x
Transition
Could
Yes
Review
Type
Informal
Internal
Informal
Internal
Informal
Internal
Informal
Internal
Software Testing
Standard Operating Procedures
5.
SOP: Test Execution and Evaluation
5.1
Purpose
The purpose of this SOP is to guide the Test Manager and Test Analyst through executing Test
Scripts and evaluating Test Results.
The purpose of executing tests is to ensure that the new functionality (Integration Testing) or
complete system (System Testing) functions as intended. The system integrator compiles and links
the system in increments. Each increment needs to go through testing of the functionality that has
been added, as well as all tests the previous builds went through (regression tests).
Within an iteration, you will execute integration and system testing several times until the whole
system (as defined by the goal of the iteration) functions as intended and meets the test's success
or completion criteria.
The purpose of evaluating tests is to generate and deliver the Test Evaluation Summary. This is
accomplished by reviewing and evaluating the Test Results, identifying and logging software
defects, and calculating the key measures of test. The Test Evaluation Summary presents the test
results and key measures of test in an organized format and is used to assess the quality of the
target-of-test and the quality of the test process.
5.2
Scope
The scope of this SOP is includes executing the test for both integration and system testing for all
FUSION Alliance software engineering projects.
5.3
Workflow Detail
Test Execution and Evaluation
Execution
2002 FUSION Alliance
Evaluation
Software Testing
Standard Operating Procedures
5.4
Roles
Role
Test Manager
Test Analyst
2002 FUSION Alliance
Responsibilities
Evaluates test results, and test effectiveness
Generates the Test Evaluation Summary
Evaluates test coverage and test effectiveness
Manages testing risks
Executes test cases and scripts
Evaluates test results
Logs defects
Creates test log
10
Software Testing
Standard Operating Procedures
5.5
Activities
5.5.1
Activity: Execute Test
Purpose
The purpose of executing tests is to execute and capture test results. Those results should be verified and defects
logged as necessary.
Role: Test Analyst
Timing/Frequency
The test results are captured during the execution of test. Since test execution may occur many times during the
development lifecycle, test results should be captured and stored in such a way that they can be reviewed and
evaluated individually for each instance of test execution.
Input Artifacts
Test Build
Test Scripts
Output Artifacts
Test Results
Steps
1.
Execute Test Build (scripts and procedures)
2.
Evaluate Test Execution
3.
Verify Test Results
4.
Log new defects
5.
Perform regression testing when necessary
Artifacts
Test Results
2002 FUSION Alliance
Templates/
Examples
Tools
Microsoft Word
Rational
TestManager
Ver.
When
Class.
1.0
Inception
Could
x.x
Elaboration
Must
x.x
Construction
Must
x.x
Transition
Could
No
Review
Type
Informal
Internal
Informal
Internal
Informal
Internal
Informal
Internal
11
Software Testing
Standard Operating Procedures
Activity: Evaluate Test
5.5.2
Purpose
The purpose of evaluating tests is to evaluate the test results and log change requests, calculate and deliver the key
measures of test and generate the Test Evaluation Summary.
Role: Test Manager
Timing/Frequency
The Test Evaluation Summary is created in the test evaluation activity. This activity may occur several times during an
iteration.
Input Artifacts
Output Artifacts
Test Results
Test Evaluation Summary
Test Log
Steps
1.
Generate Test Log
2.
Analyze Test Results
3.
Manage defects to completion
4.
Evaluate Test Coverage
5.
Analyze Defects
6.
Determine if Test Completion and Success Criteria have been achieved
7.
Generate Test Evaluation Summary
Artifacts
Test Log
Test Evaluation
Summary
2002 FUSION Alliance
Templates/
Examples
Tools
Microsoft Word
Microsoft Excel
Microsoft Word
Ver.
When
Class.
1.0
Inception
Could
x.x
Elaboration
Must
x.x
Construction
Must
x.x
Transition
Could
1.0
Inception
Could
1.0
Elaboration
Must
x.x
Construction
Must
x.x
Transition
Could
Yes
Yes
Review
Type
Informal
Internal
Informal
Internal
Informal
Internal
Informal
Internal
Formal
Internal
Formal
Internal
Formal
Internal
Formal
Internal
12
Software Testing
Standard Operating Procedures
6.
SOP: Defect Management
6.1
Purpose
The purpose of this SOP is to define how FUSION Alliance manages defects. This procedure is the
FUSION Alliance standard process for tracking and managing defects on projects.
Rational ClearQuest, Compuware TrackRecord or Microsoft Excel is used to track all software defects.
6.2
Scope
The scope of the Defect Management SOP is all FUSION Alliance software projects.
6.3
Workflow Detail
Defect Management
6.4
Roles
Role
Test Manager
Test Analyst
2002 FUSION Alliance
Responsibilities
Manages defects
Assigns defects
Closes defects
Logs defects
13
Software Testing
Standard Operating Procedures
Role
Responsibilities
2002 FUSION Alliance
Re-test defects
14
Software Testing
Standard Operating Procedures
6.5
Activities
6.5.1
Activity: Manage Defects
Purpose
The purpose of managing defects is to identify, resolve and measure defects or incidents found in the software
product.
Role: Test Manager
Timing/Frequency
At the conclusion of evaluating test results, all defects should be submitted into a defect management tool.
Input Artifacts
Output Artifacts
Test Results
Software Defect
Steps
1.
2.
3.
4.
5.
Submit Defect
Assign Defect
Fix Defect
Resolve Defect
Close Defect
Artifacts
Software Defect
2002 FUSION Alliance
Templates/
Examples
Tools
Rational
ClearQuest
Compuware
TrackRecord
Microsoft Excel
Ver.
When
Class.
N/A
Inception
Could
N/A
Elaboration
Should
N/A
Construction
Should
N/A
Transition
Could
Yes
Review
Type
Informal
Internal
Informal
Internal
Informal
Internal
Informal
Internal
15
