Software Testing:
Verification and Validation
• Verification
“Are we building the product right?”
• Validation
“Are we building the right product?”
Barry Boehm, 1979
Ch.19 - Verification & Validation Tuesday, December 7, 2021 1
Verification and Validation
Techniques
• Static Techniques
– Software Inspections
(against source code)
• Dynamic Techniques
– Software Testing
(requires executable program)
Ch.19 - Verification & Validation Tuesday, December 7, 2021 2
Verification and Validation
Static techniques
• Software Inspections
– of requirements documents
– of design documents (design reviews)
– of source code (code reviews)
– automated static analysis
Ch.19 - Verification & Validation Tuesday, December 7, 2021 3
Verification and Validation
Dynamic techniques
• Software Testing
– specification vs. implementation
• Defect testing [Ch.20]
– verifying non-functional requirements (e.g.
performance; reliability)
• Statistical testing [Ch.21]
– automated dynamic analysis (if applicable)
Ch.19 - Verification & Validation Tuesday, December 7, 2021 4
Verification and Validation
Goals
• Establish that software is fit for purpose,
not “bug-free”
• “Good enough” depends on:
– Software function (critical nature?)
– User expectations
– Market
• competition, price
Ch.19 - Verification & Validation Tuesday, December 7, 2021 5
Testing vs. Debugging
• Verification and Validation
– looking and categorizing existence of system
defects [example] [bug list]
– “What?”
• Debugging
– locating and correcting these defects
– “Why?”
Ch.19 - Verification & Validation Tuesday, December 7, 2021 6
Regression Testing
• Canned test runs to verify that new defects
were not introduced during “debugging”
session.
• Not exhaustive
• Targeted to a particular interface
– components, sub-systems, integrated system
• Different levels (lengths) of regression tests
– Targeted regressions
Ch.19 - Verification & Validation Tuesday, December 7, 2021 7
The Test Plan
• Planning should begin right after
requirements specification
– Acceptance tests can be written then
• System, sub-system tests can be written
against designs
Ch.19 - Verification & Validation Tuesday, December 7, 2021 8
The Test Plan
Requir ements System System Detailed
specification specification design design
System Sub-system Module and
Acceptance
integration integration unit code
test plan
test plan test plan and tess
Acceptance System Sub-system
Service
test integration test integration test
Ch.19 - Verification & Validation Tuesday, December 7, 2021 9
Software Inspections (code reviews)
• >60% of program errors can be detected in
code review [Fagan86]
• >90% if more formal approach used (e.g.
“Cleanroom” process) [Mills87]
(We’ll talk about Cleanroom later)
Ch.19 - Verification & Validation Tuesday, December 7, 2021 10
Software Inspections (code reviews)
• Why are reviews more effective for finding defects
in systems/subsystems (i.e., before acceptance
testing)?
– Bugs are often masked by other bugs
– Leverage domain/programming knowledge
• inspectors are skilled programmers
• Common practice: code reviews and then
acceptance testing
– reviews can also help with development of tests
Ch.19 - Verification & Validation Tuesday, December 7, 2021 11
Software Inspections (code reviews)
• Sample procedure:
– Announce review meeting in advance (a week?)
– Provide design document, implementation
overview, and pointer to code
– Reviewers read code (and make notes) in
advance of meeting
– During meeting, directives recorded by Scribe
– Testers/documenters attend too
Ch.19 - Verification & Validation Tuesday, December 7, 2021 12
Automated Static Analysis
• CASE tools that catch program curiosities
that are usually indicative of bugs:
– unreachable code
– uninitialized variables
– unreferenced variables
• Programming language dependent
– e.g., LINT (for C)
Ch.19 - Verification & Validation Tuesday, December 7, 2021 13
Automated Dynamic Analysis
• Tools which do bookkeeping while program
is run/tested.
• Can find some dynamic problems that
compiler cannot catch (depends on
language…)
• C/C++ tools: Purify, BoundsChecker
Ch.19 - Verification & Validation Tuesday, December 7, 2021 14
