Integration Testing
Testing Classification
l
Software systems are built with components
that must interoperate
l
l
A System is composed of Components
Components are themselves systems of smaller
components
Several types of testing
l
l
l
l
l
January 16
Unit Testing
Subsystem Testing
Integration Testing
Application Testing
Regression Testing
SISE
Integration Testing
l
Primary purpose: To reveal component interoperability
faults so that testing at system scope may proceed with
the fewest possible interruptions
Contrast:
l
l
Component scope testing intracomponent faults
System scope testing system scope responsibility
Component
(Focus of integration)
System
(Scope of integration)
Typical intercomponent interface
(Location of integration faults)
Method
Class
Instance variables
Intraclass messages
Class
Cluster
Interclass messages
Cluster
Subsystem
Interclass messages
Interpackage messages
Subsystem
System
Interprocces communication
Remote procedure call
OS services
January 16
SISE
Integration Testing Strategy
l
An integration strategy must answer three questions
l
l
l
January 16
Which components are the focus of integration test?
In what sequence will component interfaces be exercised?
Which test design technique should be used to exercise each
interface?
SISE
Integration in OO Systems
l
Integration in object-oriented development
takes place at all scopes
l
l
l
l
l
l
January 16
Within a class
Within a class hierarchy
Between a client and its servers
Within a cluster of related classes
Within a subsystem
Within an application system
SISE
Integration Faults
l
Integration testing can reveal component faults that
cause failures when components interact
Typical interface bugs include the following
l
l
l
l
l
l
l
January 16
Configuration/version control problems
Missing, overlapping, or conflicting functions
The client send a message that violates the server's precondition
or sequential constraints
Wrong object bound to message (polymorphic target)
Wrong parameters, or incorrect parameter values
Incorrect usage of virtual machine, ORB, or OS services
SISE
Integration Patterns - 1
l
l
l
l
l
Big Bang Integration: Attempt to demonstrate system
stability by testing all components at the same time
Bottom-up Integration: Interleave component and
integration testing by following usage dependencies
Top-down Integration: Interleave component and
integration testing by following application control
hierarchy
Collaboration Integration: Choose an order of integration
according to collaborations and their dependencies
Layer Integration: Incrementally exercises interfaces and
components in a layered architecture
January 16
SISE
Integration Patterns - 2
l
l
l
Backbone Integration: Combine Top-down
Integration, Bottom-up Integration e Big Bang
Integration to reach a stable system that will
support iterative development
Client/Server Integration: Exercise a loosely
coupled network of components, all of which use a
common server component
Distributed Services Integration: Exercise a loosely
coupled network of peer-to-peer components
High-frequency Integration: Develop and rerun an
integration test suite test hourly, daily or weekly
January 16
SISE
Scope-specific Considerations
l
Classes
l The system is the class under test
l The components to be integrated are CUT methods,
superclass methods, instance variables, and
parameters in messages received and sent by the
CUT
l Testing of class responsibilities is so closely coupled
to class integration that it does not make sense to
treat the two as separate test design patterns
January 16
SISE
10
Cluster-specific Considerations
l
The components to be integrated are server
objects used by the CUT
l
l
l
l
January 16
Bottom-up Integration is the most widely used technique
Top-down Integration is possible if one class is the head of the
cluster
If the class under test catches exceptions, Controlled Exception
Test may be useful
Big Bang integration is not recommended
There is no direct analogue to the other patterns
SISE
12
System-specific Considerations
l
Subsystem/system
l The scope of an implementation under test is often
larger than of a single class or small cluster
l At subsystem and system scopes, the goal of
integration testing is achieving a sufficient stable
system so that responsibility testing at system scope
may proceed smoothly
January 16
SISE
13
System-specific Considerations 2
l
l
l
l
l
l
l
Bottom-up Integration works well for small to medium
systems
Top-down, Collaboration, and High-frequency Integration
work for almost any scope or architecture
Layer Integration applies to layered architectures
Client/Server Integration is appropriate for client/server
architectures
Distributed Services Integration is appropriate for
decentralized networks containing peer-to-peer nodes
Backbone Integration is well suited to small to medium
systems and especially useful for embedded systems
Big Bang Integration is useful in a few limited circumstances
January 16
SISE
14
Drivers and Stubs
l
Driver: A program that calls the interface
procedures of the module being tested and reports
the results
l
A driver simulates a module that calls the module currently
being tested
Stub: A program that has the same interface
procedures as a module that is being called by the
module being tested but is simpler.
l
January 16
A stub simulates a module called by the module currently
being tested
SISE
15
Big Bang Integration
l
Intent
l
Demonstrate stability by attempting to exercise an entire system
with a few test runs
Context
l
January 16
The SUT is stabilized and only a few components have been
added or changed since the last time it passed a system scope
test suite
The SUT is small and testable, and each of its components has
passed adequate component scope tests
When components are so tightly coupled that they can not be
exercised separately
SISE
17
Fault Model and Strategy
l
Fault Model
l
l
The fault model is ambiguous and opportunistic
The hope is that the system will run and thereby demonstrate
that system testing can begin
Strategy
l
l
January 16
Dispenses incremental integration testing
The entire system is built and a test suite is applied to
demonstrate minimal operability at system scope
Test suit may be developed at system scope by using an
appropriate responsibility-based test design pattern
SISE
18
Big Bang Integration
January 16
SISE
19
Entry and Exit Criteria
l
Entry Criteria
l All components have passed component scope
testing
Exit Criteria
l The test suite passes
January 16
SISE
20
Consequences
l
Consequences
l Debugging can be difficult because you receive fewer
clues about fault locations
l Even if SUT passes, many interfaces faults can hide
and waylay subsequent system scope testing
l Under favourable circumstances it can result in quick
completion of integration testing
l Few (if any) integration drivers or stubs are developed
Known Uses
l It is used in Backbone Integration
January 16
SISE
21
Integration Testing
l
Incremental integration is the most effective technique
l
l
Add components a few at a time and then test their
interoperability
Advantages:
l Interfaces are systematically exercised
l Observed failures are most likely to come from the most
recently added component
Strategy
l
l
January 16
A sequence of components should be identified using careful
analysis of component dependencies
Testing must be planned and managed to follow these
dependencies
SISE
23
Dependency Analysis
l
Components typically depend on each other in many
ways
Class and cluster scope dependencies result from
explicit binding mechanisms
l
l
l
l
Composition and aggregation; Inheritance
Global variables; Server objects
Objects used as message parameters
...
Similar intercomponent dependencies occur at
subsystem and system scopes.
Dependencies often dictate the sequence of testing
January 16
SISE
24
Bottom-up Integration
l
Intent
l
Demonstrates stability by adding components to the
SUT in uses-dependency order, beginning with
components having the fewest dependencies
Context
l
l
January 16
Components with the least number of dependencies
are tested first
When these components pass, their drivers are
replaced with their clients; another round of testing
then begins
It is often used to support unit scope testing in the
iterative and incremental development of a
subsystems components
SISE
28
Strategy
l
Test Model
l
l
Dependency tree
Responsibility for each component
Test Procedure
l
For (n = leaf level; n <= root level; n++)
l
l
Code all components of level n
Develop responsibility test suite for each
component in level n
l
January 16
may need to add extra test cases. Why?
Test test suites of level n
SISE
29
Bottom-up Integration
January 16
SISE
31
Bottom-up Integration
January 16
SISE
32
Bottom-up Integration
January 16
SISE
33
Bottom-up Integration
l
Automation
l It is necessary one driver for each component or
component that is the root of a subtree in the
dependency relationship
l Avoid combining the test suites for components that
can be tested by individual drivers into a single driver
l The drivers may be revised as the class under test is
revised
January 16
SISE
34
Entry and Exit Criteria
l
Entry Criteria
l
The components to be integrated in each stage are
minimally operable
Exit Criteria
l
l
l
January 16
Each driver component meets the exit criteria for its test
pattern
The interface to each component has been exercised at
least once
Integration testing is complete when all root-level
components pass their test suites
SISE
35
Consequences
l
Disadvantages
l
l
l
Driver development is the most significant cost
The driver does not directly exercise intercomponent interfaces
Postpones checking critical control interfaces and collaborations until
the end of the development cycle
If a fix, revision, or enhancement is made to previously tested
component, the test configuration in which this component was first
tested should be revised accordingly and rerun
Advantages
l
l
l
l
January 16
May begin as soon as any leaf-level component is ready
Work may proceed in parallel
Although this pattern reduces stubbing, stubs may still be needed to
break a cycle or simulate exceptions
It is suitable to responsibility-based design
SISE
36
Top-down Integration
l
Intent
l Demonstrate stability by adding components to the
SUT in control hierarchy order, beginning with the toplevel control objects
Context
l Control objects typically implement essential and
nontrivial control strategies and therefore present
relatively high risk
l Top-down integration focuses on control components
first, making the demonstration of system scope endto-end operability a high priority
January 16
SISE
37
Strategy: Test Model
l
January 16
The apex of control may be represented in a
Collaboration Diagram, Sequence Diagram or
Statechart of the SUT
A responsibility test suite may be developed with
any appropriate test design pattern
l Collaboration Integration, Round-trip Scenario Test
or Covered in CRUD
SISE
38
Strategy: Test Procedure
l
l
Model the control hierarchy as a dependency tree
Develop a staged plan for implementation and
testing
Design a responsibility-based test suite at system
scope
1.
2.
3.
January 16
Develop and test the component(s) at the highest level
of control first.
Continue in breadth-first swath at each level, replacing
the server stubs with a full implementation
Continue in this manner until all servers in the SUT have
been implemented and exercised
SISE
39
Top-down Integration Step 1
and 2
SISE
January 16
40
Top-down Integration Step 3
and 4
January 16
SISE
41
Top-down Integration Step 5
January 16
SISE
43
Automation
l
l
January 16
Requires a single driver for the control apex
A stub is needed for each component in the layer
below the current focus of integration
SISE
44
Entry and Exit Criteria
l
Entry Criteria
l
The components to be integrated in each stage are
minimally operable
Exit Criteria
l
l
l
January 16
Each driven component meets the exit criteria of its test
pattern
The interface to each subcomponent has been exercised at
least one
Integration testing is complete when a build that includes all
leaf-level components passes the system scope test suite
SISE
45
Consequences
l
Disadvantages
l
Setting up a test requires that a large number of stubs be coded to
provide the desired response
An unforeseen requirement in a lower-level component may
necessitate last-minute changes to many top-components, breaking
part of the test suite
The stubs are necessarily implementation-specific and likely to be
brittle
It may be difficult to exercise lower-level components sufficiently
Advantages
l
l
l
January 16
Testing and integration may begin early
The cost of driver development is reduced
Initially, components may be developed in parallel
SISE
46
