Software Quality Assurance
Muhammad Azeem Akbar
LUT University, Finland
Software Quality Assurance
2
Topics covered
Software quality
Software standards
Reviews and inspections Ch. 24
Sommerville
Quality management and agile development SW Engineering
Software measurement
Software Quality Assurance
3
Software Quality
Software Quality Assurance
4
Software Testing
Software Quality Assurance
5
Quality and Testing
Software Quality Assurance
6
Errors, Faults and Failures
Software Quality Assurance
7
Goal of Quality Assurance
Software Quality Assurance
8
QA Technique Classification
Software Quality Assurance
9
Defect Prevention
Software Quality Assurance
10
Defect Reduction
Software Quality Assurance
11
Issues with Testing
Software Quality Assurance
12
Quality planning
• A quality plan sets out the desired product qualities and how these are
assessed and defines the most significant quality attributes.
• The quality plan should define the quality assessment process.
• It should set out which organisational standards should be applied and,
where necessary, define new standards to be used.
Software Quality Assurance
13
Quality plans
• Quality plan structure
• Product introduction;
• Product plans;
• Process descriptions;
• Quality goals;
• Risks and risk management.
• Quality plans should be short, succinct documents
• If they are too long, no-one will read them.
Software Quality Assurance
14
Scope of quality management
• Record progresses and support continuity of development as the
development team changes.
• For smaller systems, quality management needs less documentation and
should focus on establishing a quality culture.
• Techniques have to evolve when agile development is used.
Software Quality Assurance
15
Software quality
• Quality, simplistically, means that a product should meet its specification.
• This is problematical for software systems
• There is a tension between customer quality requirements (efficiency, reliability,
etc.) and developer quality requirements (maintainability, reusability, etc.);
• Some quality requirements are difficult to specify in an unambiguous way;
• Software specifications are usually incomplete and often inconsistent.
• The focus may be ‘fitness for purpose’ rather than specification
conformance.
Software Quality Assurance
16
Software fitness for purpose
• Has the software been properly tested?
• Is the software sufficiently dependable to be put into use?
• Is the performance of the software acceptable for normal use?
• Is the software usable?
• Is the software well-structured and understandable?
• Have programming and documentation standards been followed in the
development process?
Software Quality Assurance
17
Non-functional characteristics
• The subjective quality of a software system is largely based on its non-
functional characteristics.
• This reflects practical user experience – if the software’s functionality is not
what is expected, then users will often just work around this and find other
ways to do what they want to do.
• However, if the software is unreliable or too slow, then it is practically
impossible for them to achieve their goals.
Software Quality Assurance
18
Software quality attributes
Safety Understandability Portability
Security Testability Usability
Reliability Adaptability Reusability
Resilience Modularity Efficiency
Robustness Complexity Learnability
Software Quality Assurance
19
Quality conflicts
• It is not possible for any system to be optimized for all of these attributes –
for example, improving robustness may lead to loss of performance.
• The quality plan should therefore define the most important quality
attributes for the software that is being developed.
• The plan should also include a definition of the quality assessment
process, an agreed way of assessing whether some quality, such as
maintainability or robustness, is present in the product.
Software Quality Assurance
20
Process and product quality
• The quality of a developed product is influenced by the quality of the
production process.
• This is important in software development as some product quality
attributes are hard to assess.
• However, there is a very complex and poorly understood relationship
between software processes and product quality.
• The application of individual skills and experience is particularly important in
software development;
• External factors such as the novelty of an application or the need for an
accelerated development schedule may impair product quality.
Software Quality Assurance
21
Process-based quality
Develop Assess product
Define process
product quality
No Yes
Improve Quality Standardize
process OK process
Software Quality Assurance
22
Quality culture
• Quality managers should aim to develop a ‘quality culture’ where everyone
responsible for software development is committed to achieving a high
level of product quality.
• They should encourage teams to take responsibility for the quality of their
work and to develop new approaches to quality improvement.
• They should support people who are interested in the intangible aspects of
quality and encourage professional behavior in all team members.
Software Quality Assurance
23
Software standards
• Standards define the required attributes of a product or process. They play
an important role in quality management.
• Standards may be international, national, organizational or project
standards.
Software Quality Assurance
24
Importance of standards
• Encapsulation of best practice- avoids repetition of past mistakes.
• They are a framework for defining what quality means in a particular
setting i.e. that organization’s view of quality.
• They provide continuity - new staff can understand the organisation by
understanding the standards that are used.
Software Quality Assurance
25
Product and process standards
• Product standards
• Apply to the software product being developed. They include document
standards, such as the structure of requirements documents, documentation
standards, such as a standard comment header for an object class definition,
and coding standards, which define how a programming language should be
used.
• Process standards
• These define the processes that should be followed during software
development. Process standards may include definitions of specification,
design and validation processes, process support tools and a description of the
documents that should be written during these processes.
Software Quality Assurance
26
Product and process standards
Product standards Process standards
Design review form Design review conduct
Requirements document Submission of new code for
structure system building
Method header format Version release process
Java programming style Project plan approval process
Project plan format Change control process
Change request form Test recording process
Software Quality Assurance
27
Problems with standards
• They may not be seen as relevant and up-to-date by software engineers.
• They often involve too much bureaucratic form filling.
• If they are unsupported by software tools, tedious form filling work is often
involved to maintain the documentation associated with the standards.
Software Quality Assurance
28
Verification vs Validation
Chapter 8 Software Testing
29
V & V confidence
• Aim of V & V is to establish confidence that the system is ‘fit for purpose’.
• Depends on system’s purpose, user expectations and marketing
environment
• Software purpose
• The level of confidence depends on how critical the software is to an organisation.
• User expectations
• Users may have low expectations of certain kinds of software.
• Marketing environment
• Getting a product to market early may be more important than finding defects in the
program.
Chapter 8 Software Testing
30
Inspections and testing
• Software inspections
• Concerned with analysis of
the static system representation to discover problems (static verification)
• May be supplement by tool-based document and code analysis.
• Discussed in Chapter 15.
• Software testing
• Concerned with exercising and observing product behaviour (dynamic
verification)
• The system is executed with test data and its operational behaviour is observed.
Software Quality Assurance
31
Reviews and inspections
• A group examines part or all of a process or system and its documentation
to find potential problems.
• Software or documents may be 'signed off' at a
review which signifies that progress to the next development stage has
been approved by management.
• There are different types of review with different objectives
• Inspections for defect removal (product);
• Reviews for progress assessment (product and process);
• Quality reviews (product and standards).
Software Quality Assurance
32
Quality reviews
• A group of people carefully examine part or all of a software system and
its associated documentation.
• Code, designs, specifications, test plans, standards, etc. can all be
reviewed.
• Software or documents may be 'signed off' at a review which signifies that
progress to the next development stage has been approved by
management.
Software Quality Assurance
33
Phases in the review process
• Pre-review activities
• Pre-review activities are concerned with review planning and review
preparation
• The review meeting
• During the review meeting, an author of the document or program being
reviewed should ‘walk through’ the document with the review team.
• Post-review activities
• These address the problems and issues that have been raised during the
review meeting.
Software Quality Assurance
34
The software review process
Error
Planning
correction
Individual Review Follow-up
preparation meeting checks
Group Improvement
preparation
Pre-review activities Post-review activities
Software Quality Assurance
35
Distributed reviews
• The processes suggested for reviews assume that the review team has a
face-to-face meeting to discuss the software or documents that they are
reviewing.
• However, project teams are now often distributed, sometimes across
countries or continents, so it is impractical for team members to meet face
to face.
• Remote reviewing can be supported using shared documents where each
review team member can annotate the document with their comments.
Software Quality Assurance
36
Program inspections
• These are peer reviews where engineers examine the source of a system
with the aim of discovering anomalies and defects.
• Inspections do not require execution of a system so may be used before
implementation.
• They may be applied to any representation of the system (requirements,
design,configuration data, test data, etc.).
• They have been shown to be an effective technique for discovering
program errors.
Software Quality Assurance
37
Inspection checklists
• Checklist of common errors should be used to
drive the inspection.
• Error checklists are programming language dependent and reflect the
characteristic errors that are likely to arise in the language.
• In general, the 'weaker' the type checking, the larger the checklist.
• Examples: Initialisation, Constant naming, loop termination, array bounds,
etc.
Software Quality Assurance
38
An inspection checklist (a)
Fault class Inspection check
Data faults • Are all program variables initialized before their values are
used?
• Have all constants been named?
• Should the upper bound of arrays be equal to the size of the
array or Size -1?
• If character strings are used, is a delimiter explicitly assigned?
• Is there any possibility of buffer overflow?
Control faults • For each conditional statement, is the condition correct?
• Is each loop certain to terminate?
• Are compound statements correctly bracketed?
• In case statements, are all possible cases accounted for?
• If a break is required after each case in case statements, has it
been included?
Input/output faults • Are all input variables used?
• Are all output variables assigned a value before they are
output?
• Can unexpected inputs cause corruption?
Software Quality Assurance
39
An inspection checklist (b)
Fault class Inspection check
Interface faults • Do all function and method calls have the correct
number of parameters?
• Do formal and actual parameter types match?
• Are the parameters in the right order?
• If components access shared memory, do they have the
same model of the shared memory structure?
Storage management • If a linked structure is modified, have all links been
faults correctly reassigned?
• If dynamic storage is used, has space been allocated
correctly?
• Is space explicitly deallocated after it is no longer
required?
Exception management • Have all possible error conditions been taken into
faults account?
Software Quality Assurance
40
Key points
• Software quality management is concerned with ensuring that software has
a low number of defects and that it reaches the required standards of
maintainability, reliability, portability etc. Software standards are important
for quality assurance as they represent an identification of ‘best practice’.
When developing software, standards provide a solid foundation for
building good quality software.
• Reviews of the software process deliverables involve a team of people who
check that quality standards are being followed. Reviews are the most
widely used technique for assessing quality.
Software Quality Assurance
41
Key points
• In a program inspection or peer review, a small team systematically checks
the code. They read the code in detail and look for possible errors and
omissions. The problems detected are discussed at a code review
meeting.
• Agile quality management relies on establishing a quality culture where the
development team works together to improve software quality.
• Software measurement can be used to gather quantitative data about
software and the software process.
Software Quality Assurance
42
Key points
• You may be able to use the values of the software metrics that are
collected to make inferences about product and process quality.
• Product quality metrics are particularly useful for highlighting anomalous
components that may have quality problems. These components should
then be analyzed in more detail.
• Software analytics is the automated analysis of large volumes of software
product and process data to discover relationships that may provide
insights for project managers and developers.
