Brief Review
Program Verification
Lecture 18: Functional Testing
A program is correct if it meets its requirements specification
Requirements Specs
F(input) = output
Functional Contract, should be as specific as possible
Test Cases
Kenneth M. Anderson Input, Documentation, and Expected Output;
Software Methods and Tools Test Suite - a collection of test cases
Test Run
CSCI 3308 - Fall Semester, 2003 Run each test case and record pass/fail
repeat until all tests pass
October 27, 2003 © University of Colorado, 2003 2
Major Problem Functional Testing
How do you pick test cases? In functional testing, we test the
Two main approaches functionality of the system without
Functional Testing
a.k.a. Black Box Testing
regard to its implementation
Structural Testing The system is, in a sense, a black box
a.k.a. White Box Testing because we cannot look inside to see how it
Note: current testing research has moved beyond computes its output
these concepts…
folding and sampling techniques are current
We provide input and receive output
…but they are used in this class as an introduction
October 27, 2003 © University of Colorado, 2003 3 October 27, 2003 © University of Colorado, 2003 4
Functional Testing, continued Functional Testing, continued
Functional Testing is a strategy for helping a Functional testing helps create test suites by
software engineer pick test cases providing a criterion for selecting test cases:
The requirements specification of a program lists functions
This is useful, since selecting test cases is a that the program must perform
tricky problem A test suite is complete when it tests every function
A test suite should be “complete”… For each function, determine “categories” of input that a
with respect to the program’s specification function should treat equivalently
but how many test cases do you need to be complete? boundary conditions can be useful guides
A test suite should be precise test both “typical” input and error conditions
no duplicate test cases a test suite will need at least one test case for each category
if a test suite takes too long to run, then it will get run less associated with each function
often (which increases the chance that a fault goes
undetected)
October 27, 2003 © University of Colorado, 2003 5 October 27, 2003 © University of Colorado, 2003 6
Functional Testing: Step 1 Functional Testing: Step 2
Identify functional categories in the Identify specification items in the spec that correspond to
functions the program must perform
requirements specification that broadly Each item should be assignable to one of your functional
classifies functions the program must perform categories
Could be an iterative process, in which a specification item
Example: A database of cars (for a car identifies a new functional category
Car Database Example:
dealer) Generate a report listing all cars in inventory by their identification
Persistence of Information number from smallest to largest (report generation, sorting)
Generate a report listing all cars in inventory by the time a car has
Generation of Reports been in inventory from longest to shortest (report generation,
sorting)
Sorting
Information on car sales must be stored for at least two years
(persistence of information)
October 27, 2003 © University of Colorado, 2003 7 October 27, 2003 © University of Colorado, 2003 8
Functional Testing: Step 3 Step 3, continued
Identify functional equivalence classes for each The functional classes might be… (continued)
specification item (like the GCD example in lecture Cars have been entered, deleted, and then re-entered into
database
16)
Cars were entered in the order that they should be printed
Consider the first function of the car database Cars were entered in the opposite order that they should be printed
List cars in inventory by identification number Cars were entered in a random order
The functional classes might be Database has two cars with the same identification number
Database has zero cars Discussion
Database has one car This is way more functional equivalence classes than normal, in
fact, when you find a item like this it might be good to split the
Database has many cars specification item like
Cars have only been entered into the database List cars in inventory by id
Cars have been entered and then deleted Sort cars in inventory by id
October 27, 2003 © University of Colorado, 2003 9 October 27, 2003 © University of Colorado, 2003 10
Functional Testing: Step 4 Step 4, continued
Determine inputs for each functional class Identifying test cases
e.g. pick test cases! Cars have only been entered into the database: one test
Each class should have its boundaries tested along with case
some “middle” case Cars have been entered and then deleted: two test cases
Identifying test cases one extra car entered and then deleted and more than one
extra car entered and then deleted
Database has zero cars: one test case
Cars have been entered, deleted, and re-entered: three test
Database has one car: one test case cases
Database has many cars: two test cases one was deleted and re-entered
one with two cars (a boundary condition) and one with more more than one was deleted, and one was re-entered
than two cars
more than one was deleted and re-entered
If a maximum had been specified we would test that too
October 27, 2003 © University of Colorado, 2003 11 October 27, 2003 © University of Colorado, 2003 12
Functional Testing: Step 5 Method 1
Determine the number of test cases for each function Adding test cases
There are two ways to do this zero cars : 1
Add each of the test cases for each category together, or
Look for orthogonal sets of categories that can be combined
one car : 1
and multiply their test cases more than one car : 2
We will be using the “addition” method for the testing
notebook
entered only : 1
Orthogonal here means categories that test distinctly deleted : 2
different things
deleted and re-entered: 3
in our example, we have three categories that deal with the
number of cars, and three which deal with how they were Total : 10
entered into the database
October 27, 2003 © University of Colorado, 2003 13 October 27, 2003 © University of Colorado, 2003 14
Method 2 Functional Testing: Step 6
Multiplying Test Cases Eliminate redundant test cases
(1 + 1 +2 ) * ( 1 + 2 + 3) = 4 * 6 = 24 For example zero cars in the database will probably be a
Think about it like this (12 of the 24 shown): functional equivalence class for several different spec. items;
entered one multiple one deleted A single test will cover that functional class for all such items
only deleted deleted one re-entered Prioritize test cases
zero You may not have the time or budget to test them all
cars As such, give critical test cases higher priority…
one …while test cases that test obscure or uncommon errors can
car be given lower priority
two
cars
You now have your test suite!
October 27, 2003 © University of Colorado, 2003 15 October 27, 2003 © University of Colorado, 2003 16
