Module 5

Software Quality
Software Engineering & Project Management
 1. Introduction

 While quality is generally agreed to be “a good thing”

 We need to define precisely what qualities we require of a system

 Judge objectively whether system meets quality requirements

 Quality is not only the final system but also focusing on best development
methods
 2. The Place of
Software Quality
in Project
Planning
3. Importance Increasing criticality of Software –
of Software Reliability
Quality

The Intangibility of Software –

Deliverables that meets quality
requirements

Accumulating errors during software

development –
Don’t delay in resolving errors instead of
accumulating and handling in the end
4. Defining Software Quality

The quality Some qualities of a

The external qualities
requirements state software reflect the
have to be mapped
how well this system external view by
with internal factors
is to operate users - usability

When quality
Conduct program
measurements
inspection to find
identified – must set
errors and eradicate
targets for project
them at early stage
team members
 4. Defining Software Quality contd.,
There is need Definition/Description – definition of quality characteristics
for specific
quality Scale – the unit of measurement
characteristics
then a quality Test – the practical test of the extent to which the attribute quality exists
specification
Minimally acceptable – worst value may be acceptable if other characteristics
compensated for it
Target range – quality measurement should lie between range of values

Now – the value that applies currently

 • Several measurements applicable to quality
characteristic:

• In case of Reliability
1) Availability
4. Defining 2) Mean time between failures

Software 3)
4)
Failure on demand
Support activity
Quality
contd., • Associated with Reliability is maintainability (quick
response for correcting errors on detection)

• Changeability – at what ease the software is modified

• Analysability – at what ease the cause of failure can

be identified
5. Software Quality
Models

IT IS HARD TO IT CAN BE THE QUALITY

DIRECTLY MEASURE EXPRESSED IN MODELS GIVE A
THE QUALITY OF A TERMS OF SEVERAL CHARACTERIZATION
SOFTWARE ATTRIBUTES OF A (OFTEN
SOFTWARE HIERARCHICAL) OF
SOFTWARE QUALITY
IN TERMS OF A SET
OF
CHARACTERISTICS
OF THE SOFTWARE.
 1) Performance - how well jobs are performed

2) Features – supports required features

5. Software 3) Reliability – satisfactory deliverables in specified time

Quality Models 4) Conformance – degree to which product meets requirements

– Garvin’s
Quality 5) Durability – Product life

Dimensions 6) Serviceability – speed and effectiveness

7) Aesthetics – the look and feel

8) Perceived quality – User’s opinion

 Correctness – product satisfies its specifications
Reliability
Efficiency
Integrity – Data validation

5. Software Usability

Quality Models Maintainability

– McCall’ Model Flexibility – Product adapting in changing environment

Testability – ensuring it performs efficiently and error free
Portability – transferring from h/w to other or s/w to other
Reusability
Interoperability – integration with other software
5. Software Quality Models – Dromey’s model
5. Software
Quality
Models –
Boehm’s
Model
 6. ISO 9126

The ISO 9126 standards documents are very lengthy due to

people with differing motivations in Software Quality, namely:

1) Acquirers – who are obtaining software from external

suppliers

2) Developers – who are building a software product

3) Independent Evaluators – who are assessing the quality of

software product
 6. ISO 9126 Contd.,

• Effectiveness
ISO 9126 also
introduces • Productivity
another type of
quality – • Safety
“quality in use”
• Satisfaction
 6. ISO 9126 Contd.,
ISO 9126 identifies six major external software quality characteristics:

External Software
Quality
Characteristics

Maintainabilit
Functionality Reliability Usability Efficiency Portability
y

Suitability Maturity Understandability Time Behavior Analyzability Adaptability

Resource
Accuracy Fault Tolerance Learnability Changeability Installability
Utilization
Efficiency
Interoperability Recoverability Operability Stability Coexistence
Compliance
Functionality Reliability
Attractiveness Testability Replaceability
Compliance Compliance
Usability Maintainability Portability
Security
Compliance Compliance Compliance
6. ISO 9126 Contd.,
 7. Product and process Metrics
• The users assess the quality of a software product based on its external attributes

• Whereas during development, the developers assess the product’s quality based on various
internal attributes.

• Let us understand the product and process metrics:

Product Metrics Process Metrics

Help measure the characteristics of a product Help measure how a development process is
being developed performing
Ex: Review Effectiveness, average no. of defects
Ex: Function point metrics, Effort required to found per inspection, average defects correction
develop the product, Time required to develop time
the product
8. Product versus Process Quality Management
• The system development process comprises several activities linked
so that the o/p of one activity is input to the next

• Errors can enter the process at any stage

• Errors not removed at early stages become more expensive to

correct later

• One way of doing this is by having the following process

requirements for each step:
1) Entry Requirement – before an activity starts (plan and prepare
the test data)
2) Implementation Requirement – all test runs must be repeated
whenever an error is found and corrected
3) Exit Requirement – all tests must run successfully with no
outstanding errors
 9. Quality Management System (QMS)
Various International and National standard bodies including British standards Institution (BSI)
engaged in creation of standards for QMS
The British Standard is now called BS EN ISO 9001:2000, which is identical to the Internation
standard ISO 9001:2000
ISO 9000 series try to ensure that a monitoring and control system to check quality is in place.

ISO 9000 series relate to quality systems in general and not just those in software development

ISO 9000 – describes the fundamental features of QMS and its terminology

ISO 9001 – describes how QMS can be applied to the creation of products and provision of
services
ISO 9004 applies to process improvement
 The standard is built on a foundation of the following principles:
1) Understanding the requirements of customers (met or
exceeded)

2) Leadership to provide the unity of purpose and direction to

achieve quality objectives
An Overview of
3) Involvement of staff at all levels
BS EN ISO
9001:2000 4) A focus on the individual processes w.r.t. deliverable products
and services
QMS
Requirements 5) A focus on systems of interrelated processes w.r.t. deliverable
product and services

6) Continuous Improvement of processes

7) Decision making based on factual evidence

8) Building mutually beneficial relationships with suppliers

An Overview of BS EN ISO 9001:2000 QMS Requirements Contd.,
These principles are applied through cycles which involve the following activities:
1) Determining the needs and expectations of the customer
2) Establishing a quality policy – organization’s objectives in relation to quality
3) Design the processes which will have the qualities implied in organization’s quality objectives
4) Allocation of responsibilities for meeting these requirements
5) Ensuring the resources are available to execute process properly
6) Design the methods for measuring the effectiveness and efficiency of each process that
contribute to organization’s quality objectives
7) Gathering the measurements
8) Identification of any discrepancies between the actual measurements and the target values
9) Analysis and elimination of the causes of discrepancies
 An Overview of BS EN ISO 9001:2000 QMS Requirements Contd.,
The Procedures above should be designed and executed so that there is continual improvement,
carried out properly, lead to effective QMS:

1) Documentation of objectives, procedures, plans, and records relating to the actual operation of
processes

2) Management responsibility – organization need to show QMS and processes conforming to the
quality objectives

3) Resources – must ensure adequate resources, trained staff and appropriate infrastructures
applied to processes

4) Production should be characterized by –

Planning, Determination & review, Effective communication, Design & Development,
Requirement & other information, Design outcomes tested & documented, Changes to the
designs, Adequate measures for evaluation, Controlled conditions for production,
Measurement to demonstrate and confirm to standards
10. Process capability Models
 The process metrics are more meaningfully measured during product
development

 Popular process-based techniques: SEI CMM, ISO 15504 and Six Sigma

 A Historical Perspective: Quality paradigm shifted from Product

assurance to process assurance

 Modern quality assurance techniques focus on providing sufficient

guidance for recognizing, defining, analyzing, and improving the
process

 A good documented process enables good quality system: TQM

includes documentation, process, tools and product

 A term related to TQM is Business Process Reengineering (BPR)

Level 1: Initial – Practice haphazard activities by
staff
Software
Engineering
Level 2: Repeatable – Practice some basic project Institute (SEI)
management practices - Capability
Level 3: Defined – Both management & Maturity Model
development activities are defined and (CMM)
documented
SEI-CMM classifies
Level 4: Managed – Quantitative goals are set for software development
products and processes organizations into the
following 5 maturity
Level 5: Optimizing – collect metrics, analyze, levels:
improve and optimize
Capability Maturity Model
Integration(CMMI)
ISO 15504 Process
Assessment

Allocate one of the following score

 Personal Software Process (PSP)
• It helps in developing personal skills and methods and provides a defined process
which can be tuned by individuals

• Time Measurement – PSP advocates & developers should rack how they spend time

• PSP Planning – Individuals must plan their project

 Six Sigma
• Six sigma is essentially a disciplined, data-driven
approach to eliminate defects in many process.

• The statistical representation of six sigma describes

quantitatively how a process is performing

• Six sigma methodology is the implementation of a

measurement-based strategy that focus on process
improvement and variation reduction

• Two six sigma sub-methodology:

DMAIC (Define, Measure, Analyze, Improve, Control)
DMADV (Define, Measure, Analyze, Design, Verify)
 11. Techniques to help
enhance Software Quality
Three main themes emerge in this discussion of
software quality:

1) Increasing Visibility – “egoless programming” –

simple practice of programmers looking at each
other’s code

2) Procedural Structure – Growth of methodologies

where every process in SDLC has carefully laid
down steps

3) Checking Intermediate – Debugging at every

stage for the correctness of work at its earlier
 • Inspections can be applied to documents produced at
any development stage.

• Based on experience of using this technique has been

that:

1) It is very effective way of removing superficial errors

Inspections
2) It motivates developers to produce better structured
and self-explanatory software

3) It helps spread good programming practices as the

participants discuss specific piece of code

4) It can enhance team spirit

General Principles behind the Fagon Method
• Inspections are carried out at all major deliverables

• All types of defect are noted – not just logic or function errors

• Inspections can be carried out by colleagues at all levels except the

very top

• Inspections are carried out using a preferred set of steps

• Inspections meetings do not last for more than two hours

• Inspections is led by moderator who has specific training

• Other participants have defined role (ex. Recorder, Reader)

• Checklists are used to assist the fault – finding process

• Material is inspected at an optimal rate about 100 lines an hour

• Statistics are maintained so that effectiveness is monitored

 With this type of development there are
separate teams:

Structured 1) Specification Team – which obtains the

Programming user requirements & also a usage profile
estimating the volume of use for each
and Clean- feature in system

room 2) Development Team – which develops the

Software code, but which does no machine testing
of the program code produced
Development
3) Certification Team – which carries out
testing
Formal Methods: It uses an unambiguous but non-mathematical notations
which developers are familiar with programming languages

Software Quality Circles (SWQC): A team of volunteers who meet to identify,

analyze, and solve their work-related problems

Lesson Learnt Reports: Project managers are required to write a Lessons

Learnt Report at the end of project. This should be distinguished from a Post
Implementation Review (PIR)
12. Testing
 Testing: Verification and Validation
Verification:
It is the process of determining whether the output of one phase of S/W development
conforms to that of previous phase.

Verification is carried out during development process.

Validation:
It is the process of determining whether fully developed software conforms to its
requirements specifications.

Validation is carried out towards the end of the development process.

 Testing: Test Case Design

1 3
2

There are essentially In white-box

two main approaches to In black-box approach: approach: Requires
systematically design test cases are designed analysis if source code,
test cases: Black-box using only the also known as structural
approach and White-box functional specification testing or structure-
approach of software (I/O or driven testing
functional behaviour) –
functional testing also
known as requirements-
driven testing
 • A Software product is normally
Levels tested at different stages or levels.
These three testing stages are:

of 1) Unit Testing
2) Integration Testing

Testing 3) System Testing

Testing Activities
Testing involves performing the following main activities:

• Test planning
• Test Suite Design
• Test Case Execution and Result Checking
• Test Reporting - A review body adjudicates these issues:
The issue is dismissed on the grounds that there has been misunderstanding of
requirement by tester
The issue is identified as a fault which developers need to correct
It is recognized that the software is behaving as specified but requirement agreed
in fact incorrect
The issue is identified as a fault but is treated as an off-specification (app can
be made operational with error still in place)
Testing Activities Contd.,
Testing involves performing the following main activities:

• Debugging
• Error Correction
• Defect Retesting
• Regression Testing
• Test Closure
 Who performs Testing
Many organizations have separate system testing groups to provide
independent assessment of the correctness of software before release
Test Automation
• Testing
Estimation of Latent Errors
13. Software Reliability
13. Software Reliability
13. Software Reliability
13. Software Reliability
13. Software Reliability
13. Software Reliability
14. Quality Plans
Some organizations produce quality plans for each project, a quality plan might have
entries for:
• Purpose – scope of plan
• List of references to other documents
• Management arrangements, including organization, tasks, & responsibilities
• Documentation to be produced
• Standards, practices and conventions
• Reviews and Audits
• Testing
• Problem Reporting & Corrective Action
• Tools, Techniques and Methodologies
• Code, Media and Supplier control
• Records collection, maintenance and retention
• Training
• Risk Management – the methods of risk management that are to be used
End of Module 5