Software Testing Techniques

- Testing fundamentals

-Functional Testing

- Non Functional Testing

-Manual Testing

-Automation Testing
 Characteristics of Testable
Software
• Operable
– The better it works (i.e., better quality), the easier it is to test

• Observable
– Incorrect output is easily identified; internal errors are
automatically detected

• Controllable
– The states and variables of the software can be controlled
directly by the tester

• Decomposable
– The software is built from independent modules that can be
tested independently
(more on next slide) 2
 Characteristics of Testable
Software (continued)
• Simple
– The program should exhibit functional, structural, and code
simplicity

• Stable
– Changes to the software during testing are infrequent and do not
invalidate existing tests

• Understandable
– The architectural design is well understood; documentation is
available and organized

3
 Test Characteristics
• A good test has a high probability of finding an error
– The tester must understand the software and how it might fail

• A good test is not redundant

– Testing time is limited; one test should not serve the same purpose
as another test

• A good test should be neither too simple nor too complex

– Each test should be executed separately; combining a series of tests
could cause side effects and mask certain errors

4
Software Testing

5
Manual and Automation Testing
• Manual Testing
• Manual testing includes testing a software manually, i.e., without
using any automated tool or any script. In this type, the tester
takes over the role of an end-user and tests the software to identify
any unexpected behavior or bug.
• There are different stages for manual testing such as unit testing,
integration testing, system testing, and user acceptance
testing.
• Testers use test plans, test cases, or test scenarios to test a
software to ensure the completeness of testing. Automation
Testing
• Automation testing, which is also known as Test Automation,
is when the tester writes scripts and uses another software to
test the product. This process involves automation of a manual
process. Automation Testing is used to re-run the test scenarios
that were performed manually, quickly, and repeatedly. 6
 Manual and Automation Testing
• The following tools can be used for automation testing −

• HP Quick Test Professional

• Selenium
• IBM Rational Functional Tester
• SilkTest
• TestComplete
• Testing Anywhere
• WinRunner
• LoadRunner
• Visual Studio Test Professional
• WATIR

7
 Non Functional Testing
• This section is based upon testing an application from its non-
functional attributes. Non-functional testing involves testing a
software from the requirements which are non functional in nature but
important such as performance, security, user interface, etc.
• Some of the important and commonly used non-functional testing
types are discussed below.
• Performance Testing
• It is mostly used to identify any bottlenecks or performance issues
rather than finding bugs in a software. There are different causes that
contribute in lowering the performance of a software

 Network delay
 Client-side processing
 Database transaction processing
 Load balancing between servers
 Data rendering 8
 Non Functional Testing
• testing is considered as one of the important and mandatory testing
type in terms of the following aspects −
 Speed (i.e. Response Time, data rendering and accessing)
 Capacity
 Stability
 Scalability

• Performance testing can be either qualitative or quantitative and can be

divided into different sub-types such as Load testing and Stress
testing.

9
 Non Functional Testing
• Load Testing
• It is a process of testing the behavior of a software by applying
maximum load in terms of software accessing and manipulating
large input data. It can be done at both normal and peak load
conditions. This type of testing identifies the maximum capacity of
software and its behavior at peak time.

• Most of the time, load testing is performed with the help of

automated tools such as Load Runner, AppLoader, IBM Rational
Performance Tester, Apache JMeter, Silk Performer, Visual Studio
Load Test, etc.

• Virtual users (VUsers) are defined in the automated testing tool

and the script is executed to verify the load testing for the
software. The number of users can be increased or decreased
10
concurrently or incrementally based upon the requirements.
 Non Functional Testing
• Stress Testing

• Stress testing includes testing the behavior of a software under

abnormal conditions. For example, it may include taking away some
resources or applying a load beyond the actual load limit.
• The aim of stress testing is to test the software by applying the load to
the system and taking over the resources used by the software to
identify the breaking point. This testing can be performed by testing
different scenarios such as −
• Shutdown or restart of network ports randomly
• Turning the database on or off
• Running different processes that consume resources such as CPU,
memory, server, etc.

11
 Non Functional Testing
• Usability testing
• Usability testing is a black-box technique and is used to identify
any error(s) and improvements in the software by observing the
users through their usage and operation.
• According to Nielsen, usability can be defined in terms of five
factors, i.e. efficiency of use, learn-ability, memory-ability,
errors/safety, and satisfaction. According to him, the usability of a
product will be good and the system is usable if it possesses the
above factors.
• Nigel Bevan and Macleod considered that usability is the quality
requirement that can be measured as the outcome of interactions
with a computer system. This requirement can be fulfilled and the
end-user will be satisfied if the intended goals are achieved
effectively with the use of proper resources.
12
 User Acceptance Testing
• Alpha Testing
• Beta Testing

13
 Alpha Testing
• Alpha Testing
• This test is the first stage of testing and will be performed
amongst the teams (developer and QA teams). Unit testing,
integration testing and system testing when combined
together is known as alpha testing.
• During this phase, the following aspects will be tested in the
application −
 Spelling Mistakes
 Broken Links
 Cloudy Directions
 The Application will be tested on machines with the lowest
specification to test loading times and any latency problems.

14
 Beta Testing
• This test is performed after alpha testing has been successfully
performed.
• In beta testing, a sample of the intended audience tests the
application. Beta testing is also known as pre-release testing
• In this phase, the audience will be testing the following −
 Users will install, run the application and send their feedback to
the project team.
 Typographical errors, confusing application flow, and even
crashes.
 Getting the feedback, the project team can fix the problems before
releasing the software to the actual users.
 The more issues you fix that solve real user problems, the higher
the quality of your application will be.
 Having a higher-quality application when you release it to the
general public will increase customer satisfaction.
15
 Two Unit Testing Techniques
• Black-box testing
– Knowing the specified function that a product has been designed to perform,
test to see if that function is fully operational and error free
– Includes tests that are conducted at the software interface
– Not concerned with internal logical structure of the software

• White-box testing
– Knowing the internal workings of a product, test that all internal operations
are performed according to specifications and all internal components have
been exercised
– Involves tests that concentrate on close examination of procedural detail
– Logical paths through the software are tested
– Test cases exercise specific sets of conditions and loops

16
 White-box Testing
• Uses the control structure part of component-level design to
derive the test cases
• These test cases
– Guarantee that all independent paths within a module have been
exercised at least once
– Exercise all logical decisions on their true and false sides
– Execute all loops at their boundaries and within their operational
bounds
– Exercise internal data structures to ensure their validity
• Types of white box testing are
– Control Flow Testing
– Branch Testing
– Basis Path
– Data Flow Testing
– Loop Testing 17
 Basis Path Testing
• White-box testing technique proposed by Tom McCabe

• Enables the test case designer to derive a logical complexity

measure of a procedural design

• Uses this measure as a guide for defining a basis set of execution

paths

• Test cases derived to exercise the basis set are guaranteed to

execute every statement in the program at least one time during
testing

18
 Basis Path Testing
• An execution path is a set of nodes and directed edges in a flow
graph that connects (in a directed fashion) the start node to a
terminal node.

• Two execution paths are said to be independent if they do not

include the same set of nodes and edges.

• A basic set of execution paths for a flow graph is an independent

maximum set of paths in which all nodes and edges of the graph
are included at least once

19
 Flow Graph Notation
• A circle in a graph represents a node, which stands for a sequence of
one or more procedural statements

• A node containing a simple conditional expression is referred to as a

predicate node
– Each compound condition in a conditional expression containing one or
more Boolean operators (e.g., and, or) is represented by a separate
predicate node
– A predicate node has two edges leading out from it (True and False)

• An edge, or a link, is a an arrow representing flow of control in a

specific direction
– An edge must start and terminate at a node
– An edge does not intersect or cross over another edge

• Areas bounded by a set of edges and nodes are called regions

• When counting regions, include the area outside the graph as a
region, too
20
21
 Flow Graph Example
FLOW CHART FLOW GRAPH
0 0

R4
1 1

2 2

3 R3
3

6 4 6 4
R2
7 8 5
7 R1 8 5
9
9
11 10 22
11 10
 Cyclomatic Complexity
• Provides a quantitative measure of the logical complexity of a program
• Defines the number of independent paths in the basis set

• Provides an upper bound for the number of tests that must be

conducted to ensure all statements have been executed at least once

• Can be computed three ways

– The number of regions
– V(G) = E – N + 2, where E is the number of edges and N is the number of
nodes in graph G
– V(G) = P + 1, where P is the number of predicate nodes in the flow graph G

• Results in the following equations for the example flow graph

– Number of regions = 4
– V(G) = 14 edges – 12 nodes + 2 = 4
– V(G) = 3 predicate nodes + 1 = 4

23
 Independent Program Paths
• Defined as a path through the program from the start node
until the end node that introduces at least one new set of
processing statements or a new condition (i.e., new nodes)

• Must move along at least one edge that has not been traversed
before by a previous path

• Basis set for flow graph on previous slide

– Path 1: 0-1-11
– Path 2: 0-1-2-3-4-5-10-1-11
– Path 3: 0-1-2-3-6-8-9-10-1-11
– Path 4: 0-1-2-3-6-7-9-10-1-11

• The number of paths in the basis set is determined by the

cyclomatic complexity
24
 Deriving the Basis Set and Test
Cases
1) Using the design or code as a foundation, draw a corresponding
flow graph

2) Determine the cyclomatic complexity of the resultant flow graph

3) Determine a basis set of linearly independent paths

4) Prepare test cases that will force execution of each path in the
basis set

25
 A Second Flow Graph Example
1 int functionY(void) 3
2 {
3 int x = 0; 4
4 int y = 19;

5 A: x++; 5
6 if (x > 999)
7 goto D;
8 if (x % 11 == 0) 6
9 goto B;
10 else goto A; 8 7
11 B: if (x % y == 0) 10 9 16
12 goto C;
13 else goto A;
11 17
14 C: printf("%d\n", x);
15 goto A; 13 12
16 D: printf("End of list\
n"); 14
17 return 0;
18 } 26
15
Graph Matrics

27
Loop Testing

28
 Black-box Testing
• Complements white-box testing by uncovering different classes of
errors

• Focuses on the functional requirements and the information

domain of the software

• Used during the later stages of testing after white box testing has
been performed

• The tester identifies a set of input conditions that will fully

exercise all functional requirements for a program

• The test cases satisfy the following:

– Reduce, by a count greater than one, the number of additional test
cases that must be designed to achieve reasonable testing
– Tell us something about the presence or absence of classes of errors,
rather than an error associated only with the specific task at hand
29
 Black-box Testing Categories
• Incorrect or missing functions
• Interface errors
• Errors in data structures or external data base access
• Behavior or performance errors
• Initialization and termination errors

30
 Equivalence Partitioning
• A black-box testing method that divides the input domain of a
program into classes of data from which test cases are derived

• An ideal test case single-handedly uncovers a complete class of

errors, thereby reducing the total number of test cases that must
be developed.

• Test case design is based on an evaluation of equivalence classes

for an input condition.

• An equivalence class represents a set of valid or invalid states for

input conditions.

• From each equivalence class, test cases are selected so that the
largest number of attributes of an equivalence class are exercise at
once
31
 Guidelines for Defining
Equivalence Classes
• If an input condition specifies a range, one valid and two invalid
equivalence classes are defined
– Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}

• If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
– Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}

• If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined
– Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}

• If an input condition is a Boolean value, one valid and one invalid class are
define
– Input: {true condition} Eq classes: {true condition}, {false condition}

32
 Boundary Value Analysis
• A greater number of errors occur at the boundaries of the
input domain rather than in the "center“

• Boundary value analysis is a test case design method that

complements equivalence partitioning
– It selects test cases at the edges of a class
– It derives test cases from both the input domain and output domain

33
 Guidelines for
Boundary Value Analysis
• 1. If an input condition specifies a range bounded by values a and
b, test cases should be designed with values a and b as well as
values just above and just below a and b

• 2. If an input condition specifies a number of values, test case

should be developed that exercise the minimum and maximum
numbers. Values just above and just below the minimum and
maximum are also tested

• Apply guidelines 1 and 2 to output conditions; produce output that

reflects the minimum and the maximum values expected; also test
the values just below and just above

• If internal program data structures have prescribed boundaries

(e.g., an array), design a test case to exercise the data structure at
its minimum and maximum boundaries 34