STATISTICAL SOFTWARE QUALITY ASSURANCE
Statistical quality assurance reflects a growing trend throughout industry to
become more quantitative about quality. For software, statistical quality assurance
implies the following steps:
1.Information about software defects is collected and categorized.
2.An attempt is made to trace each defect to its underlying cause (e.g., non-
conformance to specifications, design error, violation of standards, poor
communication with the customer).
3. Using the Pareto principle (80 percent of the defects can be traced to 20 percent
of all possible causes), isolate the 20 percent (the "vital few").
4. Once the vital few causes have been identified, move to correct the problems
that have caused the defects.
This relatively simple concept represents an important step towards the creation o:
an adaptive software engineering process in which changes are made to improve
those elements of the process that introduce error.
To illustrate this, assume that a software engineering organization collects
information on defects for a period of one year. Some of the defects are uncovered
as software is being developed. Others are encountered after the software has been
released to its end-users.
Although hundreds of different errors are uncovered, all can be tracked to
one (or more) of the following causes:
•incomplete or erroneous specifications (IBS)
•misinterpretation of customer communication (MCC)
•intentional deviation from specifications (IDS)
•violation of programming standards (VPS)
•error in data representation (EDR)
•inconsistent component interface (ICI)
•error in design logic (EDL)
•incomplete or erroneous testing (IET)
•inaccurate or incomplete documentation (IID)
•error in programming language translation of design (PLT)
•ambiguous or inconsistent human/computer interface (HCI)
•miscellaneous (MIS)
To apply statistical SQA, Table 1.1 is built. The table indicates that IES, MCC, and
EDR are the vital few causes that account for 53 percent of all errors. It should be
noted, however, that IES, EDR, PLT, and EDL would be selected as the vital few
causes if only serious errors are considered. Once the vital few causes are
determined, the software engineering organization can begin corrective action.
For example, to correct MCC, the software developer might implement facilitated
application specification techniques to improve the quality of customer
communication and specifications.
To improve EDR, the developer might acquire CASE tools for data modeling and
perform more stringent data design reviews. It is important to note that corrective
action focuses primarily on the vital few. As the vital few causes are corrected,
new candidates pop to the top of the stack.
Statistical quality assurance techniques for software have been shown to provide
substantial quality improvement [ART97]. In some cases, software organizations
have achieved a 50 percent reduction per year in defects after applying these
techniques.
In conjunction with the collection of defect information, software developers can
calculate an error index (El) for each major step in the software process [IEE94].
After analysis, design, coding, testing, and release, the following data are
gathered:
E i = The total number of errors uncovered during the ith step in the software
engineering process
Table 1.1: DATA COLLECTION FOR STATISTICAL SQA
Si = the number of serious errors
Mi = the number of moderate errors
Ti = the number of minor errors
PS = size of the product (LOG, design statements, pages of documentation) at
the ith step
Ws, Wm, Wt = weighting factors for serious, moderate, and trivial errors, where
recommended values are Ws = 10, Wm = 3, Wt = 1. The weighting factors for each
phase should become larger as development progresses. This rewards an
organization that finds errors early.
At each step in the software process, a phase index, PIi, is computed:
Pli = Ws (Si/Ei) + Wm (Mi/Ei) + Wt (Ti/Ei)
The error index is computed by calculating the cumulative effect on each PI i,
weighting errors encountered later in the software engineering process more
heavily than those encountered earlier:
EI= Σ(I x PIi)/PS = (PIi + 2PI2 + 3PI3 + .. . iPIi)/PS
The error index can be used in conjunction with information collected.in Table
1.1.To develop an overall indication of improvement in software quality.
