Software Testing
Software testing:
“The Process of executing a program with the intent
of finding errors”
(Myers)
"Testing is any activity aimed at evaluating an
attribute or capability of a program or system and
determining that it meets its required results.“
(Hetzel)
2
Software testing:
“The process of operating a system or component
under specified conditions, observing or recording
the results, and making an evaluation of some
aspect of the system or component”
(IEEE Standard 610.12-1990)
3
Software testing:
“The process of analyzing a software item to detect the
differences between existing and required conditions
(that is, bugs) and to evaluate the features of the
software item.”
(IEEE Standard 829-1998 )
4
Testing is not easy!
• Takes time, must be scheduled
• Takes resources, it costs and it costs much.
• May include heavy debugging, can cause the re-development of
the product.
• There is no net to catch all the bugs (Exhaustive vs Selective testing).
5
Testing – Debugging Cycle
Software Delivery
No Error
Software Coding Software Testing
Error/s
Software Debugging
Testing cost!
7
Why do we spend so much?
• Why is the testing being done?
• Find failures (not faults!)
• Certification
• Safety: criticality
• Usability
• Performance
• Acceptance
• Load
• Compatibility
• Security
8
Testing Activities
1. Test planning and design
2. Verification of test plans, designs and cases
3. Test case selection, creation and verification
4. Planning expected results
5. Data preparation
6. Execution and recording of results
7. Analysis of results
8. Wrap-up
9
Software Testing Life Cycle:
10
What to test?
• Any part of a system can be tested
• choice of part constrains choices in other dimensions
• Units are typically the smallest parts tested
• a unit can be a function, a class, or a work assignment
• Some parts of the software may not be tested at all
• e.g. exception handlers, debug code, commercial
components, library functions
• For some testing
• stubs and drivers may have to be written (and tested)
11
Stub and Driver:
12
Who should do the testing?
Developer Independent tester
Understands the system Must learn about the system,
but, will test "gently" but, will attempt to break it
and, is driven by "delivery" and, is driven by “quality”
Constructive Approach Destructive Approach
13
Skills of a Tester:
• Not all testers are developers
• e.g. testers at Corel who are photographers (Corel
Draw testing - end user testing).
• e.g. testing buddies at Microsoft.
• Testers may also be developers.
• Some testers may need specialized domain knowledge
related to the application area.
14
Skills of a Tester:
• Testers must have
• ability to select good test cases
• ability to withstand the most severe deadline and
resource pressures
• ability to organize and analyze large volumes of data
• ability to sort out complex version control issues
15
•Inexperienced testers often fail to report:
• timing problems
• transient bugs
• what they can’t quickly replicate
• what they think can be attributable to their
misunderstanding
• what they think the reader of the report might consider
minor
• lots of other problems
16
Test Case:
“A test case is a scenario to be executed with the intention of
finding a particular error/defect.”
“Inputs to test the system and the predicted outputs from these
inputs if the system operates according to its specification.”
17
A good test case:
• A good test case:
• has a high probability of finding an error
• is not redundant
• should be “best of breed”
• should be neither too simple nor too complex
18
Successful Test:
• A test which causes a program to behave in an anomalous
way, is said to be a successful test case.
• Tests show the presence not the absence of defects.
• Only exhaustive testing can show a program is free from
errors. However, exhaustive testing is impossible.
19
Testing Principles:
1. All tests should be traceable to customer requirements.
2. Tests should be planned long before testing begins.
3. The Pareto principle applies to software testing.
4. Testing should begin “in the small” and progress toward testing
“in the large”.
5. Exhaustive Testing is not possible.
6. To be most effective, testing should be conducted by an
independent third party.
20
Pareto Principle:
“80% of all the errors uncovered during testing will
likely to be traceable to 20% of all the program
components.”
21
Parts of a test case:
22
23
Test plan:
24
Test report:
25
When to stop testing?
• How is the decision to stop testing made?
• Coverage
• Limits set by management
• User acceptance e.g. Early adopters
• Contractual
CLUCREFAME
• Reliability threshold
• Fault detection rate threshold
• Meeting a standard
26
References:
• Sommerville, Ian “Software Engineering 6th edition.”
• Shepard, Terry “Dimensions of Testing”
• Pressman, Roger “Software Engineering, A
Practitioner’s Approach 6th edition. ”
27
Thank You!
