2/19/2020 Software Quality Metrics - Tutorialspoint

Software Quality Metrics

Software metrics can be classified into three categories −

Product metrics − Describes the characteristics of the product such as size,
complexity, design features, performance, and quality level.

Process metrics − These characteristics can be used to improve the

development and maintenance activities of the software.

Project metrics − This metrics describe the project characteristics and

execution. Examples include the number of software developers, the staffing
pattern over the life cycle of the software, cost, schedule, and productivity.

Some metrics belong to multiple categories. For example, the in-process quality
metrics of a project are both process metrics and project metrics.

Software quality metrics are a subset of software metrics that focus on the quality
aspects of the product, process, and project. These are more closely associated with
process and product metrics than with project metrics.

Software quality metrics can be further divided into three categories −

Product quality metrics
In-process quality metrics
Maintenance quality metrics

Product Quality Metrics

This metrics include the following −
Mean Time to Failure
Defect Density
Customer Problems
Customer Satisfaction

Mean Time to Failure

It is the time between failures. This metric is mostly used with safety critical systems
such as the airline traffic control systems, avionics, and weapons.

Defect Density

[Link] 1/6
2/19/2020 Software Quality Metrics - Tutorialspoint

It measures the defects relative to the software size expressed as lines of code or
function point, etc. i.e., it measures code quality per unit. This metric is used in many
commercial software systems.

Customer Problems

It measures the problems that customers encounter when using the product. It
contains the customer’s perspective towards the problem space of the software, which
includes the non-defect oriented problems together with the defect problems.

The problems metric is usually expressed in terms of Problems per User-Month

(PUM).

PUM = Total Problems that customers reported (true defect and non-defect orient
orien
problems) for a time period + Total number of license months of the software d
the period

Where,

Number of license-month of the software = Number of install license of the soft

sof
Number of months in the calculation period

PUM is usually calculated for each month after the software is released to the market,
and also for monthly averages by year.

Customer Satisfaction

Customer satisfaction is often measured by customer survey data through the five-
point scale −
Very satisfied
Satisfied
Neutral
Dissatisfied
Very dissatisfied

Satisfaction with the overall quality of the product and its specific dimensions is usually
obtained through various methods of customer surveys. Based on the five-point-scale
data, several metrics with slight variations can be constructed and used, depending on
the purpose of analysis. For example −
Percent of completely satisfied customers

[Link] 2/6
2/19/2020 Software Quality Metrics - Tutorialspoint
Percent of satisfied customers
Percent of dis-satisfied customers
Percent of non-satisfied customers

Usually, this percent satisfaction is used.

In-process Quality Metrics

In-process quality metrics deals with the tracking of defect arrival during formal
machine testing for some organizations. This metric includes −
Defect density during machine testing
Defect arrival pattern during machine testing
Phase-based defect removal pattern
Defect removal effectiveness

Defect density during machine testing

Defect rate during formal machine testing (testing after code is integrated into the
system library) is correlated with the defect rate in the field. Higher defect rates found
during testing is an indicator that the software has experienced higher error injection
during its development process, unless the higher testing defect rate is due to an
extraordinary testing effort.

This simple metric of defects per KLOC or function point is a good indicator of quality,
while the software is still being tested. It is especially useful to monitor subsequent
releases of a product in the same development organization.

Defect arrival pattern during machine testing

The overall defect density during testing will provide only the summary of the defects.
The pattern of defect arrivals gives more information about different quality levels in
the field. It includes the following −

The defect arrivals or defects reported during the testing phase by time
interval (e.g., week). Here all of which will not be valid defects.

The pattern of valid defect arrivals when problem determination is done on the
reported problems. This is the true defect pattern.

The pattern of defect backlog overtime. This metric is needed because

development organizations cannot investigate and fix all the reported
problems immediately. This is a workload statement as well as a quality
statement. If the defect backlog is large at the end of the development cycle
and a lot of fixes have yet to be integrated into the system, the stability of the

[Link] 3/6
2/19/2020 Software Quality Metrics - Tutorialspoint

system (hence its quality) will be affected. Retesting (regression test) is

needed to ensure that targeted product quality levels are reached.

Phase-based defect removal pattern

This is an extension of the defect density metric during testing. In addition to testing, it
tracks the defects at all phases of the development cycle, including the design reviews,
code inspections, and formal verifications before testing.

Because a large percentage of programming defects is related to design problems,

conducting formal reviews, or functional verifications to enhance the defect removal
capability of the process at the front-end reduces error in the software. The pattern of
phase-based defect removal reflects the overall defect removal ability of the
development process.

With regard to the metrics for the design and coding phases, in addition to defect rates,
many development organizations use metrics such as inspection coverage and
inspection effort for in-process quality management.

Defect removal effectiveness

It can be defined as follows −

D
Deef
feec
cttr
reem
moov
veed
dddu
urri
inng
gaad
deev
veello
oppm
meen
nttp
phha
asse
e
D
DRRE
E =
= ×
× 100
100%%
D
Deef
feec
ctts
s lla
atte
ennt
tiin
ntth
heep
prro
oddu
ucct
t

This metric can be calculated for the entire development process, for the front-end
before code integration and for each phase. It is called early defect removal when
used for the front-end and phase effectiveness for specific phases. The higher the
value of the metric, the more effective the development process and the fewer the
defects passed to the next phase or to the field. This metric is a key concept of the
defect removal model for software development.

Maintenance Quality Metrics

Although much cannot be done to alter the quality of the product during this phase,
following are the fixes that can be carried out to eliminate the defects as soon as
possible with excellent fix quality.
Fix backlog and backlog management index
Fix response time and fix responsiveness
Percent delinquent fixes
Fix quality

[Link] 4/6
2/19/2020 Software Quality Metrics - Tutorialspoint

Fix backlog and backlog management index

Fix backlog is related to the rate of defect arrivals and the rate at which fixes for
reported problems become available. It is a simple count of reported problems that
remain at the end of each month or each week. Using it in the format of a trend chart,
this metric can provide meaningful information for managing the maintenance process.

Backlog Management Index (BMI) is used to manage the backlog of open and
unresolved problems.

N
Nuum
mbbe
erro
off p
prro
obblle
emms
sccllo
osse
eddd
duur
riin
nggt
thhe
emmo
onnt
thh
B
BMMI
I =
= ×
× 100
100%%
N
Nuum
mbbe
erro
off p
prro
obblle
emms
saar
rrri
ivve
eddd
duur
riin
nggt
thhe
emmo
onnt
thh

If BMI is larger than 100, it means the backlog is reduced. If BMI is less than 100, then
the backlog increased.

Fix response time and fix responsiveness

The fix response time metric is usually calculated as the mean time of all problems
from open to close. Short fix response time leads to customer satisfaction.
The important elements of fix responsiveness are customer expectations, the agreed-
to fix time, and the ability to meet one's commitment to the customer.

Percent delinquent fixes

It is calculated as follows −

P
Peer
rcce
ennt
tDDe
elli
innq
quue
ennt
tFFi
ixxe
ess =
=

N
Nuum
mbbe
err o
off f
fiix
xees
s t
thha
att e
exxc
ceee
edde
edd t
thhe
e r
rees
sppo
onns
see t
tiim
mee c
crri
itte
erri
iaa b
byy c
ceev
veer
riit
tyy lle
evve
ell
×
× 100
100%%
N
Nuum
mbbe
err o
off f
fiix
xees
s d
deelli
ivve
erre
edd i
inn a
a s
sppe
ecci
iffi
ieed
d t
tiim
mee

Fix Quality

Fix quality or the number of defective fixes is another important quality metric for the
maintenance phase. A fix is defective if it did not fix the reported problem, or if it fixed
the original problem but injected a new defect. For mission-critical software, defective
fixes are detrimental to customer satisfaction. The metric of percent defective fixes is
the percentage of all fixes in a time interval that is defective.
A defective fix can be recorded in two ways: Record it in the month it was discovered
or record it in the month the fix was delivered. The first is a customer measure; the

[Link] 5/6
2/19/2020 Software Quality Metrics - Tutorialspoint

second is a process measure. The difference between the two dates is the latent
period of the defective fix.

Usually the longer the latency, the more will be the customers that get affected. If the
number of defects is large, then the small value of the percentage metric will show an
optimistic picture. The quality goal for the maintenance process, of course, is zero
defective fixes without delinquency.

[Link] 6/6