UNIT-3

STATIC TESTING

❖ Static testing is a complimentary technique to dynamic testing technique to acquire higher quality
software.

❖ Static testing techniques do not execute the software and they do not require the bulk of test
cases.

❖ This type of testing is also known as non-computer based testing or human testing.

❖ All the bugs cannot be caught alone by the dynamic testing technique; static testing reveals the
errors which are not shown by dynamic testing.

❖ Static testing can be applied for most of the verification activities. Since verification activities are
required at every stage of SDLC till coding, static testing also can be applied at all these phases.
❖ Requirements, design specifications, test plans, source code, user’s manuals,
maintenance procedures are some of the items that can be statically tested.

❖ Static testing has proved to be a cost-effective technique of error detection.

❖ Fagan reported that more than 60% of the errors in a program can be
detected using static testing.

❖ It provides the exact location of a bug, whereas dynamic testing provides no

indication of the exact source code location of the bug. In other words, we
can say that static testing finds the in-process errors before they become
bugs.
BENEFITS

Static testing techniques help to produce a better product. Given below

are some of the benefits of adopting a static testing approach

❑ As defects are found and fixed, the quality of the product increases.

❑ A more technically correct base is available for each new phase of

development.
❑ The overall software life cycle cost is lower, since defects are found early
and are easier and less expensive to fix.

❑ The effectiveness of the dynamic test activity is increased and less time
needs to be devoted for testing the product.

❑ Immediate evaluation and feedback to the author from his/her peers

which will bring about improvements in the quality of future products.
OBJECTIVES

❑ To identify errors in any phase of SDLC as early as possible

❑ To verify that the components of software are in conformance with
its requirements
❑ To provide information for project monitoring
❑ To improve the software quality and increase productivity
Types of Static Testing

Static testing can be categorized into the following types:

❑ Software inspections
❑ Walkthroughs
❑ Technical reviews
SOFTWARE INSPECTIONS

Inspection process is an in-process manual examination of an item to detect bugs. It may be applied to
any product or partial product of the software development process, including requirements, design and
code, project management plan, SQA plan, software configuration plan (SCM plan), risk management
plan, test cases, user manual, etc. Inspections are embedded in the process of developing products and
are done in the early stages of each product’s development.
Software inspection does not execute the code, so it is machine-independent, requires no target system
resources or changes to the program’s operational behavior, and can be used much before the target
hardware is available for dynamic testing purposes.

The inspection process is carried out by a group of peers. The group of peers first inspect the product at the
individual level. After this, they discuss the potential defects of the product observed in a formal meeting.
The second important thing about the inspection process is that it is a formal process of verifying a software
product. The documents which can be inspected are SRS, SDD, code, and test plan.
An inspection process involves the interaction of the following elements:
❑ Inspection steps
❑ Role for participants
❑ Item being inspected

The entry and exit criteria are used to determine whether an item is ready to be inspected.

Entry criteria mean that the item to be inspected is mature enough to be used. For
example, for code inspection, the entry criterion is that the code has been compiled
successfully.

Exit criterion is that once the item has been given for inspection, it should not be updated,
otherwise it will not know how many bugs have been reported and corrected through the
inspection process and the whole purpose of the inspection is lost.
INSPECTION TEAM

❖ For the inspection process, a minimum of the following four team members are
required.
❖ Author/Owner/Producer A programmer or designer responsible for producing the
program or document. He is also responsible for fixing defects discovered during the
inspection process.
❖ Inspector A peer member of the team, i.e. he is not a manager or supervisor. He is not
directly related to the product under inspection and may be concerned with some
other product. He finds errors, omissions, and inconsistencies in programs and
documents.
❖ Moderator A team member who manages the whole inspection process. He schedules,
leads, and controls the inspection session. He is the key person with the responsibility of
planning and successful execution of the inspection.
❖ Recorder One who records all the results of the inspection meeting.
INSPECTION PROCESS

A general inspection process has the following stages:

Planning During this phase, the following is executed:

➢ The product to be inspected is identified.

➢ A moderator is assigned.

➢ The objective of the inspection is stated, i.e. whether the inspection is to be

conducted for defect detection or something else. If the objective is defect
detection, then the type of defect detection like design error, interface error,
code error must be specified. The aim is to defi ne an objective for the
meeting so that the effort spent in inspections is properly utilized.
During planning, the moderator performs the following activities:
✓ Assures that the product is ready for inspection
✓ Selects the inspection team and assigns their roles
✓ Schedules the meeting venue and time
✓ Distributes the inspection material like the item to be inspected, checklists, etc.
Overview In this stage, the inspection team is provided with the background information for
inspection. The author presents the rationale for the product, its relationship to the rest of the
products being developed, its function and intended use, and the approach used to develop it.
This information is necessary for the inspection team to perform a successful inspection.
The opening meeting may also be called by the moderator. In this meeting, the objective of
inspection is explained to the team members. The idea is that every member should be familiar
with the overall purpose of the inspection.
Individual preparation After the overview, the reviewers individually prepare themselves for the
inspection process by studying the documents provided to them in the overview session. They
point out potential errors or problems found and record them in a log. This log is then submitted
to the moderator. The moderator compiles the logs of different members and gives a copy of this
compiled list to the author of the inspected item.
The moderator reviews the logs submitted by each inspector to determine whether the team is adequately
prepared. The moderator also checks for trouble spots that may need extra attention during inspection,
common defects that can be categorized quickly, and the areas of major concern. If the logs indicate that the
team is not adequately prepared, the moderator should reschedule the inspection meeting. After this, the
compiled log fi le is submitted to the author.

Inspection meeting The inspection meeting starts with the author of the inspected item who has created it.
The author first discusses every issue raised by different members in the compiled log fi le. After the discussion,
all the members arrive at a consensus whether the issues pointed out are in fact errors and if they are errors,
should they be admitted by the author. It may be possible that during the discussion on any issue, another error
is found. Then, this new error is also discussed and recorded as an error by the author.

The basic goal of the inspection meeting is to uncover any bug in the item. However, no effort is made in the
meeting to fi x the bug. It means that bugs are only being notified to the author, which he will fi x later. If there is
any clarification regarding these bugs, then it should be asked or discussed with other members during the
meeting.
At the end, the moderator concludes the meeting and produces a summary of the
inspection meeting. This summary is basically a list of errors found in the item that need to be
resolved by the author

Rework The summary list of the bugs that arise during the inspection meeting needs to be
reworked by the author. The author fixes all these bugs and reports back to the moderator.

Follow-up It is the responsibility of the moderator to check that all the bugs found in the last
meeting have been addressed and fixed. He prepares a report and ascertains that all issues
have been resolved. The document is then approved for release. If this is not the case, then
the unresolved issues are mentioned in a report and another inspection meeting is called by
the moderator.
 BENEFITS OF INSPECTION PROCESS

Bug reduction The number of bugs is reduced through the inspection process. L.H. Fenton [86] reported that
through the inspection process in IBM, the number of bugs per thousand lines of code has been reduced by
two-thirds. Thus, inspection helps reduce bug injection and detection rates. According to Capers Jones,
‘Inspection is by far the most effective way to remove bugs.’
Bug prevention Inspections can also be used for bug prevention. Based on the experiences of previous
inspections, analysis can be made for future inspections or projects, thereby preventing the bugs which
have appeared earlier. Programmers must understand why bugs appear and what can be done to avoid
them in future. At the same time, they should provide inspection results to the quality assurance team.
Productivity Since all phases of SDLC may be inspected without waiting for code development and its
execution, the cost of finding bugs decreases, resulting in an increase in productivity. Moreover, the errors
are found at their exact places, therefore reducing the need of dynamic testing and debugging. In the
article by Fagan [43], an increase of 23% in coding productivity and a 25% reduction in schedules were
reported.
Real-time feedback to software engineers

The inspections also benefit software engineers/developers because they get feedback on their products on
a relatively real-time basis. Developers find out the type of mistakes they make and what is the error density.
Since they get this feedback in the early stages of development, they may improve their capability. Thus,
inspections benefit software engineers/ developers in the sense that they can recognize their weakness and
improve accordingly, which in turn benefits the cost of the project.

Reduction in development resource The cost of rework is surprisingly high if inspections are not used and errors
are found during development or testing. Therefore, techniques should be adopted such that errors are
found and fixed as close to their place of origin as possible. Inspections reduce the effort required for
dynamic testing and any rework during design and code, thereby causing an overall net reduction in the
development resource. Without inspections, more resources may be required during design and dynamic
testing. But, with inspection, the resource requirement is greatly reduced.

Quality improvement We know that the direct consequence of testing is improvement in the quality of
software. The direct consequence of static testing also results in the improvement of quality of the final
product. Inspections help to improve the quality by checking the standard compliance, modularity, clarity,
and simplicity
Project management A project needs monitoring and control. It depends on some data obtained
from the development team. However, this data cannot be relied on forever. Inspection is another
effective tool for monitoring the progress of the project.

Checking coupling and cohesion The modules’ coupling and cohesion can be checked easily through
inspection as compared to dynamic testing. This also reduces the maintenance work.

Learning through inspection Inspection also improves the capability of different team members, as they
learn from the discussions on various types of bugs and the reasons why they occur. It may be more
beneficial for new members. They can learn about the project in a very short time. This helps them in the
later stages of development and testing.

Process improvement There is always scope of learning from the results of one process. An analysis of
why the errors occurred or the frequent places where the errors occurred can be done by the
inspection team members. The analysis results can then be used to improve the inspection process so
that the current as well as future projects can benefit. Discussed below are the issues of process
improvement.
EFFECTIVENESS OF INSPECTION PROCESS

In an analysis done by Oliver Lautenberg, the inspection process was found to be effective as
compared to structural testing because the inspection process alone found 52% errors. Of the remaining
48% of defects, on an average, only 17% are detected by structural testing.

The effectiveness of the inspection process lies in its rate of inspection. The rate of inspection refers to
how much evaluation of an item has been done by the team. If the rate is very fast, it means the
coverage of item to be evaluated is high. Similarly if the rate is too slow, it means that the coverage is
not much. However, the rate should also be considered in the perspective of detection of errors. If the
rate is too fast, then it may be possible that it reveals only few errors. If the rate is too slow, then the cost
of project increases. Thus, a balanced rate of inspection should be followed so that it concentrates on a
reasonable number of defects. It may be calculated as the error detection efficiency of the inspection
process, as given below:
 The rate of inspection also depends on some factors like the experience of the
inspection team, the programming language, and the application domain.
COST OF INSPECTION PROCESS

With at least four members involved in an inspection team, the cost of inspecting 100
lines of code is roughly equivalent to one person-day of effort. This assumes that the
inspection itself takes about an hour and that each member spends 1-2 hours preparing
for the inspection. Testing costs are variable and depend on the number of faults in the
software. However, the effort required for the program inspection is less than half the
effort that would be required for equivalent dynamic testing. Further, it has been
estimated that the cost of inspection can be 5–10% of the total cost of the project.
VARIANTS OF INSPECTION PROCESS
Active Design Reviews

 Active Design Reviews [87] are for inspecting the design stage of SDLC. In this
process, several reviews are conducted targeting a particular type of bugs and
conducted by the reviewers who are experts in that area. Thus, it covers all the
sections of the design document based on several small reviews instead of only one
inspection. It is also based on the use of questionnaires to give the reviewers better-
defined responsibilities and to make them play a more active role.

 The technique has been used in the Naval Research Laboratory’s (Washington)
software cost reduction (SCR) project for several years with good results. The SCR
project involves the experimental redevelopment of the operational flight program
(OFP) for the Navy’s A-7E aircraft.
These are conducted in the following stages
 Overview A brief overview of the module being reviewed is presented. The overview
explains the modular structure in case it is unfamiliar to reviewers, and shows them where
the module belongs in the structure.

 Review Reviewers are assigned sections of the document to be reviewed and

questionnaires based on the bug type. Reviewers answer the questions in the
questionnaires. They are also assigned a timeframe during which they may raise any
questions they have, and a time to meet with designers after the designers have read the
completed questionnaires.

 Meeting The designers read the completed questionnaires and meet the reviewers to
resolve any queries that the designers may have about the reviewer’s answers to the
questionnaires. The reviewers may be asked to defend their answers. This interaction
continues until both designers and reviewers understand the issues, although an
agreement on these issues may not be reached. After the review, the designers produce a
new version of the documentation.
Formal Technical Asynchronous Review Method ( FTArm)

In this process, the meeting phase of inspection is

considered expensive and therefore, the idea is to
eliminate this phase. The inspection process is carried
out without having a meeting of the members. This is a
type of asynchronous inspection [88] in which the
inspectors never have to simultaneously meet. For this
process, an online version of the document is made
available to every member where they can add their
comments and point out the bugs.
 Setup It involves choosing the members and preparing the document for an
asynchronous inspection process. The document is prepared as a hypertext
document.

 Orientation It is same as the overview step of the inspection process discussed

earlier.

 Private review It is same as the preparation phase discussed in the inspection

process. Here, each reviewer or inspector individually gives his comments on the
document being inspected. However, each inspector provides comments
individually and is unable to see the other inspector’s comments.
 Public review In this step, all comments provided privately are made public and all
inspectors are able to see each other’s comments and put forward their suggestions.
Based on this feedback and with consultation of the author, it is decided that there is a
bug.

 Consolidation In this step, the moderator analyses the result of private and public reviews
and lists the findings and unresolved issues, if any.

 Group meeting If required, any unresolved issues are discussed in this step. But the
decision to conduct a group meeting is taken in the previous step only by the moderator.

 Conclusion The final report of the inspection process along with the analysis is produced
by the moderator
 Gilb Inspection

Gilb and Graham [89] defined this process. It differs from Fagan
inspection in that the defect detection is carried out by individual
inspectors at their own level rather than in a group. Therefore, a
checking phase has been introduced. Three different roles are
defined in this type of inspection:

 Leader is responsible for planning and running the inspection.

 Author of the document

 Checker is responsible for finding and reporting the defects in

the document.
 Entry The document must pass through an entry criteria so that the inspection
time is not wasted on a document which is fundamentally flawed.

 Planning The leader determines the inspection participants and schedules the
meeting.

 Kick-off The relevant documents are distributed, participants are assigned roles
and brief

 Checking Each checker works individually and finds defects.

 Logging Potential defects are collected and logged.

 Brainstorm In this stage, process improvement suggestions are recorded based on the
reported bugs.

 Edit After all the defects have been reported, the author takes the list and works
accordingly.

 Follow-up The leader ensures that the edit phase has been executed properly.

 Exit The inspection must pass the exit criteria as fi xed for the completion of the inspection
process.
Humphrey’s Inspection Process

 It was described by Watts Humphrey.

In this process, the preparation phase
emphasizes on finding and logging of
bugs, unlike Fagan inspections. It also
includes an analysis phase between
preparation and meeting. In the
analysis phase, individual logs are
analyzed and combined into a single
list.
N-Fold Inspection

It is based on the idea that the effectiveness of the inspection process can be increased by
replicating it [91]. If we increase the number of teams inspecting the item, the percentage
of defects found may increase. But sometimes the cost of organizing multiple teams is higher
as compared to the number of defects found. A proper evaluation of the situation is
required. Originally, this process was used for inspecting requirement specifications, but it
can also be used for any phase. As discussed, this process consists of many independent
inspection teams. This process needs a coordinator who coordinates various teams, collects
and collates the inspection data received by them. For this purpose, he also meets the
moderator of every inspection team.
 Planning and overview This is the formal planning and overview stage.
In addition, it also includes the planning of how many teams will participate in the
inspection process.

 Inspection stages There are many inspection processes adopted by many teams. It
is not necessary that every team will choose the same inspection process. The team
is free to adopt any process.

 Collation phase The results from each inspection process are gathered, collated,
and a master list of all detected defects is prepared.

 Rework and follow-up This step is same as the tradition Fagan inspection process.
Phased Inspection

Phased inspections [92] are designed to verify the product in a particular domain. If
we feel that in a particular area, we may encounter errors, then phased inspections
are carried out. But for this purpose, only experts who have experience in that
particular domain are called. Thus, this inspection process provides a chance to utilize
human resources. In this process, inspection is divided into more than one phase. There
is an ordered set of phases and each phase is designed such that it will check a
particular feature in the product. The next phase in the ordered set of phases assumes
that the particular feature has been verified in the previous phase. There are two types
of phases, as discussed below.
 Single inspector In this phase, a rigorous checklist is used by a single inspector to
verify whether the features specified are there in the item to be inspected.

 Multiple inspector Here, a checklist cannot be used. It means that the item cannot be
verified just with the questions mentioned in the checklist. There are many inspectors
who are distributed the required documents for verification of an item. Each inspector
examines this information and develops a list of questions of their own based on a
particular feature. These questions are not in binary form as in single inspection. The
item is then inspected individually by all the inspectors based on a self-developed
checklist which is either application or domain specific. After individual checking by
the inspectors, a reconciliation meeting is organized where inspectors compare their
findings about the item.
READING TECHNIQUES

A reading technique can be regarded as a mechanism or strategy for the individual

inspector to detect defects in the inspected product. Most of the techniques found in the
literature support individual inspection work. The various reading techniques are discussed
below.

Ad hoc method In this method, there is no direction or guidelines provided for inspection.
However, ad hoc does not mean that inspection participants do not scrutinize the
inspected product systematically. The word ad hoc only refers to the fact that no technical
support on how to detect defects in a software artifact is given to them. In this case, defect
detection fully depends on the skills, knowledge, and experience of an inspector.
Checklists A checklist is a list of items that focus the inspector’s attention on specific topics, such as
common defects or organizational rules, while reviewing a software document [43]. The purpose of
checklists is to gather expertise concerning the most common defects and thereby supporting
inspections. The checklists are in the form of questions which are very general and are used by the
inspection team members to verify the item. However, checklists have some drawbacks too:

 The questions, being general in nature, are not sufficiently tailored to take into account a
particular development environment.

 Instruction about using a checklist is often missing.

 There is a probability that some defects are not taken care of. It happens in the case of those
type of defects which have not been previously detected.
Scenario-based reading Checklists are general in nature and do not cover different types
of bugs. Scenario-based reading [93] is another reading technique which stresses on
finding different kind of defects. It is based on scenarios, wherein inspectors are provided
with more specific instructions than typical checklists. Additionally, they are provided with
different scenarios, focusing on different kind of defects.

The following methods have been developed based on the criteria given by scenario-
based reading.
Perspective-based reading The idea behind this method is that a software item should be inspected
from the perspective of different stakeholders [95,104,105]. Inspectors of an inspection team have to
check the software quality as well as the software quality factors of a software artifact from different
perspectives. The perspectives mainly depend upon the roles people have within the software
development or maintenance process. For each perspective, either one or multiple scenarios are
defined, consisting of repeatable activities an inspector has to perform, and the questions an
inspector has to answer. For example, a testing expert first creates a test plan based on the
requirements specification and then attempts to find defects from it. This is designed specifically for
requirements inspection, and inspectors who participate in inspection should have expertise in
inspecting requirements of their own area of expertise. Later, perspective-based reading has been
applied to code inspections and design inspections
Usage-based reading This method proposed by Thelin is applied in design inspections. Design
documentation is inspected based on use cases, which are documented in requirements
specification. Since use-cases are the basis of inspection, the focus is on finding functional
defects, which are relevant from the users’ point of view.

Abstraction driven reading This method given by Dunsmore is designed for code inspections. In
this method, an inspector reads a sequence of statements in the code and abstracts the
functions these statements compute. An inspector repeats this procedure until the final function of
the inspected code artifact has been abstracted and can be compared to the specification.
Thus, the inspector creates an abstraction level specification based on the code under inspection
to ensure that the inspector has really understood the code.
Task-driven reading This method is also valid for code inspections proposed by Kelly & Shepard
[102]. In this method, the inspector has to create a data dictionary, a complete description of
the logic and a cross-reference between the code and the specifications.

Function-point based scenarios This is based on scenarios for defect detection in requirements
documents [103]. This approach is based on function-point analysis ( FPA). FPA defines a
software system in terms of its inputs, fi les, inquiries, and outputs. The scenarios designed
around function-points are known as function-point scenarios. It consists of questions and
directs the focus of an inspector to a specific function-point item within the inspected
requirements document
CHECKLISTS FOR INSPECTION PROCESS

The inspection team must have a checklist against which they detect errors. The
checklist is according to the item to be inspected. For example, the design document
and the code of the module should have different checklists. Checklists can be
prepared with the points mentioned in the verification of each phase. Checklists should
be prepared in consultation with experienced staff and regularly updated as more
experience is gained by the inspection process.

The checklist may vary according to the environment and needs of the organization.
Each organization should prepare its own checklists for every item
STRUCTURED WALKTHROUGHS

The idea of structured walkthroughs was proposed by Yourdon. It is a less formal

and less rigorous technique as compared to inspection. The common term used
for static testing is inspection but it is a very formal process. If you want to go for a
less formal process having no bars of organized meeting, then walkthroughs are a
good option. A typical structured walkthrough team consists of the following
members:
 Coordinator Organizes, moderates, and follows up the walkthrough activities.
 Presenter/Developer Introduces the item to be inspected. This member is
optional.
 Scribe/Recorder Notes down the defects found and suggestion proposed by
the members.
 Reviewer/Tester Finds the defects in the item.
 Maintenance Oracle Focuses on long-term implications and future maintenance of the
project
 Standards Bearer Assesses adherence to standards.
 User Representative/Accreditation Agent Reflects the needs and concerns of the user.
The steps of a walkthrough process are shown
The inspection team uses the checklist approach for uncovering errors. A walkthrough is
less formal, has fewer steps and does not use a checklist to guide or a written report to
document the team’s work. Rather than simply reading the program or using error
checklists, the participants ‘play computer’. The person designated as a tester comes to
the meeting armed with a small set of paper test cases—representative sets of inputs and
expected outputs for the program or module. During the meeting, each test case is
mentally executed. That is, the test data are walked through the logic of the program.
The state of the program is monitored on a paper or any other presentation media. The
walkthrough should have a follow-up process similar to that described in the inspection
process.
 TECHNICAL REVIEWS

A technical review is intended to evaluate the software in the light of development standards, guidelines, and
specifications and to provide the management with evidence that the development process is being carried
out according to the stated objectives. A review is similar to an inspection or walkthrough, except that the
review team also includes management. Therefore, it is considered a higher-level technique as compared to
inspection or walkthrough.

A technical review team is generally comprised of management-level representatives and project

management. Review agendas should focus less on technical issues and more on oversight than an
inspection. The purpose is to evaluate the system relative to specifications and standards, recording defects
and deficiencies. The moderator should gather and distribute the documentation to all team members for
examination before the review.
He should also prepare a set of indicators to measure the following points:
❑ Appropriateness of the problem definition and requirements
❑ Adequacy of all underlying assumptions
❑ Adherence to standards
❑ Consistency
❑ Completeness
❑ Documentation

The moderator may also prepare a checklist to help the team focus on the key points. The result of the review
should be a document recording the events of the meeting, deficiencies identified, and review team
recommendations. Appropriate actions should then be taken to correct any deficiencies and address all
recommendations.
VALIDATION ACTIVITIES

By validation, we mean that the module that has been prepared till now is in conformance
with the requirements which were set initially in the SRS or user manual.
Every validation testing focuses on a particular SDLC phase and thereby focuses on a
particular class of errors. We can see a one-to-one correspondence between development
and testing processes.
For example, the purpose of unit validation testing is to find discrepancies between the
developed module’s functionality and its requirements and interfaces specified in the SRS.
Software validation is achieved through a series of black-box tests that demonstrate
conformity with requirements
UNIT VALIDATION TESTING

Since unit is the smallest building block of the software system, it is the first piece of
system to be validated. Before we validate the entire software, units or modules must be
validated.
Units must also be validated to ensure that every unit of software has been built in the
right manner in conformance with user requirements. Unit tests ensure that the software
meets at least a baseline level of functionality prior to integration and system testing.
A module is not independent and cannot be tested in isolation. While testing the
module, all its interfaces must be simulated if the interfaced modules are not ready at
the time of testing the module under consideration. Discussed below are the types of
interface modules which must be simulated, if required, to test a module.
Drivers

Suppose a module is to be tested, wherein some inputs are to be received from another
module. However, this module which passes inputs to the module to be tested is not ready
and under development. In such a situation, we need to simulate the inputs required in the
module to be tested.
A driver can be defined as a software module which is used to invoke a module under
test and provide test inputs, control and monitor execution, and report test results or most
simplistically a line of code that calls a method and passes a value to that method.

A test driver may take inputs in the following form and call the unit to be tested:

❑ It may hard-code the inputs as parameters of the calling unit.

❑ It may take the inputs from the user.

❑ It may read the inputs from a file.

A test driver provides the following facilities to a unit to be tested:

❑ Initializes the environment desired for testing.

❑ Provides simulated inputs in the required format to the units to be tested.

Stubs

The module under testing may also call some other module which is not ready at the time
of testing. Therefore, these modules need to be simulated for testing. In most cases,
dummy modules instead of actual modules, which are not ready, are prepared for these
subordinate modules. These dummy modules are called stubs.
Thus, a stub can be defined as a piece of software that works similar to a unit which is
referenced by the unit being tested, but it is much simpler than the actual unit. A stub
works as a ‘stand-in’ for the subordinate unit and provides the minimum required behavior
for that unit.
Stubs have the following characteristics:

➢ Stub is a place holder for the actual module to be called. Therefore, it is not designed with
the functionalities performed by the actual module. It is a reduced implementation of the
actual module.

➢ It does not perform any action of its own and returns to the calling unit

➢ We may include a display instruction as a trace message in the body of stub. The idea is that
the module to be called is working fi ne by accepting the input parameters.

➢ A constant or null must be returned from the body of stub to the calling module.

➢ Stub may simulate exceptions or abnormal conditions, if required.

Benefits of Designing Stubs and Drivers

➢ Stubs allow the programmer to call a method in the code being developed, even if
the method does not have the desired behavior yet.
➢ By using stubs and drivers effectively, we can cut down our total debugging and
testing time by testing small parts of a program individually, helping us to narrow down
problems before they expand.
➢ Stubs and drivers can also be an effective tool for demonstrating progress in a business
environment.
Example

Program
main() (a) Driver for main() module: Stub for caldiff() Module
{ main() caldiff(int x, int y, int z)
Int a,b,c,sum,diff,mul; { {
scanf(“%d %d %d”, &a, &b, &c); printf(“Difference calculating module”);
int a, b, c, sum;
sum = calsum(a,b,c); return 0;
scanf(“%d %d %d”, &a, &b, &c);
diff = caldiff(a,b,c); }
mul = calmul(a,b,c); sum = calsum(a,b,c);
Stub for calmul() Module
printf(“%d %d %d”, sum, diff, mul); printf(“The output from calsum
calmul(int x, int y, int z)
} module is %d”, sum);
{
calsum(int x, int y, int z) }
printf(“Multiplication calculation
{
module”);
int d;
d = x + y + z; return 0;

return(d); }
}
 Integration Testing

Integration is the activity of combining the modules together when all the modules have been
prepared. Integration of modules is according to the design of software specified earlier.
Integration aims at constructing a working software system. Integration testing focuses on
bugs caused by interfacing between the modules while integrating them.
Modules are not standalone entities. They are a part of a software system which comprises of
many interfaces. Even if a single interface is mismatched, many modules may be affected.
Thus, integration testing is necessary for the following reasons:
1. Integration testing exposes inconsistency between the modules such as improper call or
return sequences.
2. Data can be lost across an interface.
3. One module when combined with another module may not give the desired result.
4. Data types and their valid ranges may mismatch between the modules.
There are three approaches for integration testing
 Decomposition-Based Integration

The idea for this type of integration is based on the decomposition of design into functional
components or modules. In the tree designed for decomposition-based integration, the nodes
represent the modules present in the system and the links/edges between the two modules
represent the calling sequence. The nodes on the last level in the tree are leaf nodes.
Integration testing methods are classified into two categories:
(a) non-incremental and
(b) incremental.
Non-Incremental Integration Testing
In this type of testing, either all untested modules are combined together and then tested or unit tested
modules are combined together. It is also known as Big-Bang integration testing.
Big-Bang method cannot be adopted practically. This theory has been discarded due to the following
reasons:
1. Big-Bang requires more work. For example, consider the following hierarchy of a software system.
According to the Big-Bang theory, if all unit tested modules are integrated in this example, then for unit
testing of all the modules independently, we require four drivers and seven stubs. This count will grow
according to the size of the system.
2. Actual modules are not interfaced directly until the end of the software system.
3. It will be difficult to localize the errors since the exact location of bugs
cannot be found easily.
 Incremental Integration Testing

In this type, you start with one module and unit test it. Then combine the module which has to
be merged with it and perform test on both the modules. In this way, incrementally keep on
adding the modules and test the recent environment. Thus, an integrated tested software
system is achieved. Incremental integration testing is beneficial for the following reasons:
1. Incremental approach does not require many drivers and stubs.
2. Interfacing errors are uncovered earlier.
3. It is easy to localize the errors since modules are combined one by one. The first suspect is the
recently added module. Thus, debugging becomes easy.
4. Incremental testing is a more thorough testing. Consider the example in Fig. 7.8. If we are
testing module 6, then although module 1 and 2 have been tested previously, they will get
the chance to be tested again and again. This gives the modules more exposure to the
probable bugs by the time the last module in the system is tested.
Types of Incremental Integration Testing
Incremental integration can be done either from top to bottom or bottom to top. Based
on this strategy, incremental integration testing is divided into two categories.

Top-down Integration Testing The strategy in top-down integration is to look at the design hierarchy
from top to bottom. Start with the high-level modules and move downward through the design
hierarchy. Modules subordinate to the top module are integrated in the following two ways:

Depth first integration In this type, all modules on a major control path of the design hierarchy are
integrated first. In the example shown in Fig. 7.8, modules 1, 2, 6, 7/8 will be integrated first. Next,
modules 1, 3, 4/5 will be integrated.

Breadth first integration In this type, all modules directly subordinate at each level, moving across
the design hierarchy horizontally, are integrated first. In the example shown in Fig. 7.8, modules 2
and 3 will be integrated first. Next, modules 6, 4, and 5 will be integrated. Modules 7 and 8 will be
integrated last.
Top-Down Integration Procedure
The procedure for top-down integration process is discussed in the following steps:
1. Start with the top or initial module in the software. Substitute the stubs for all the subordinate
modules of top module. Test the top module.
2. After testing the top module, stubs are replaced one at a time with the actual modules for
integration.
3. Perform testing on this recent integrated environment. 4. Regression testing may be
conducted to ensure that new errors have not appeared.
5. Repeat steps 2–4 for the whole design hierarchy.

Listed below are the drawbacks of top-down integration testing.

1. Stubs must be prepared as required for testing one module.
2. Stubs are often more complicated than they first appear.
3. Before the I/O functions are added, the representation of test cases in stubs can be difficult.
Bottom-up Integration Testing

The bottom-up strategy begins with the terminal or modules at the lowest level in the software structure. After
testing these modules, they are integrated and tested moving from bottom to top level. Since the processing
required for modules subordinate to a given level is always available, stubs are not required in this strategy.
Bottom-up integration can be considered as the opposite of top-down approach. Unlike top-down strategy, this
strategy does not require the architectural design of the system to be complete. Thus, bottom-up integration
can be performed at an early stage in the developmental process. It may be used where the system reuses and
modifies components from other systems.
The steps in bottom-up integration are as follows:
1. Start with the lowest level modules in the design hierarchy. These are the modules from which no other
module is being called.
2. Look for the super-ordinate module which calls the module selected in step 1. Design the driver module for
this super-ordinate module.
3. Test the module selected in step 1 with the driver designed in step 2.
4. The next module to be tested is any module whose subordinate modules (the modules it calls) have all been
tested.
5. Repeat steps 2 to 5 and move up in the design hierarchy.
6. Whenever, the actual modules are available, replace stubs and drivers with the actual one and test again
This type of integration is useful for integrating object-oriented systems, real-time systems, and
systems with strict performance requirements. Bottom-up integration has the disadvantage
that the software as a whole does not exist until the last module is added.
Comparison between Top-Down and Bottom-Up Integration Testing
Sandwich Integration testing

There are situations depending on the project in hand which will force to integrate the
modules by combining top-down and bottom-up techniques. This combined approach
is sometimes known as sandwich integration testing.

Selection of an integration testing strategy depends on software characteristics and

sometimes project schedules. In general, sandwich testing strategy that uses top-down
tests for upper levels of the program structure with bottom-up tests for subordinate levels
is the best compromise.
CALL GRAPH-BASED INTEGRATION

It is assumed that integration testing detects bugs which are structural. However, it is
also important to detect some behavioral bugs. If we can refi ne the functional
decomposition tree into a form of module calling graph, then we are moving towards
behavioral testing at the integration level. This can be done with the help of a call
graph, as given by Jorgensen.

A call graph is a directed graph, wherein the nodes are either modules or units, and a
directed edge from one node to another means one module has called another
module. The call graph can be captured in a matrix form which is known as the
adjacency matrix.
The idea behind using a call graph for integration testing is to avoid the efforts made in
developing the stubs and drivers. If we know the calling sequence, and if we wait for the called or calling
function, if not ready, then call graph-based integration can be used.
There are two types of integration testing based on call graph which are discussed next.
Pair-wise Integration If we consider only one pair of calling and called modules, then we can make a set of
pairs for all such modules, as shown in Fig. 7.11, for pairs 1–10 and 1–11. The resulting set will be the total test
sessions which will be equal to the sum of all edges in the call graph. For example, in the call graph shown in
Fig. 7.11, the number of test sessions is 19 which is equal to the number of edges in the call graph.
Neighbourhood Integration

There is not much reduction in the total number of test sessions in pair-wise integration as compared to
decomposition-based integration. If we consider the neighborhoods of a node in the call graph, then the number
of test sessions may reduce. The neighborhood for a node is the immediate predecessor as well as the immediate
successor nodes. The neighborhood of a node, thus, can be defined as the set of nodes that are one edge away
from the given node.

The total test sessions in neighborhood integration can be calculated as:

Neighborhoods = nodes – sink nodes
= 20 – 10 = 10
where sink node is an instruction in a module at which the execution terminates.
 PATH-BASED INTEGRATION

In path execution, there may be a call to another unit. At that point, the control is transferred
from the calling unit to the called unit. This passing of control from one unit to another unit is
necessary for integration testing. Also, there should be information within the module
regarding instructions that call the module or return to the module. This must be tested at the
time of integration. It can be done with the help of path-based integration defined by Paul C.
Jorgenson.
We need to understand the following definitions for path-based integration.
Source node It is an instruction in the module at which the execution starts or resumes. The nodes
where the control is being transferred after calling the module are also source nodes.
Sink node It is an instruction in a module at which the execution terminates. The nodes from which the control
is transferred are also sink nodes.
Module execution path ( MEP) It is a path consisting of a set of executable statements within a module like in a
flow graph.
Message When the control from one unit is transferred to another unit, then the programming language
mechanism used to do this is known as a message. For example, when there is a function call, then it is a
message from one unit (where the call is mentioned; caller module) to another unit (the unit which is being
called).
MM-path It is a path consisting of MEPs and messages. The path shows the sequence of executable
statements; it also crosses the boundary of a unit when a message is followed to call another unit. In other
words, MM-path is a set of MEPs and transfer of control among different units in the form of messages.
MM-path graph It can be defined as an extended flow graph where nodes are MEPs and edges are
messages. It returns from the last called unit to the first unit where the call was made. In this graph, messages
are highlighted with thick lines.
Now let us see the concept of path-based integration with the help of one example.
Fig shows the MM-path as a darken line.
FUNCTION TESTING

It is the process of attempting to detect discrepancies between the functional

specifications of a software and its actual behavior.
Every functionality of the system specified in the functions is tested according to its
external specifications. An external specification is a precise description of the software
behavior from the viewpoint of the outside world (e.g. user) Tests verify that the system
behaves correctly from the user/business perspective and functions according to the
requirements, models, or any other design paradigm used to specify the application.
The function test must determine if each component or business event:
1. performs in accordance to the specifications,
2. responds correctly to all conditions that may present themselves by incoming
events/data,
3. moves data correctly from one business event to the next (including data stores),
and
4. is initiated in the order required to meet the business objectives of the system.
Function testing can be performed after unit and integration testing, or
whenever the development team thinks that the system has sufficient functionality to
execute some tests. To keep a record of function testing, a function coverage metric is
used. Function coverage can be measured with a function coverage matrix. It keeps
track of those functions that exhibited the greatest number of errors. This information is
valuable because it tells us that these functions probably contain the preponderance of
errors that have not been detected yet.
The primary processes/deliverables for requirements based function test are discussed below.
Test planning During planning, the test leader with assistance from the test team defines
the scope, schedule, and deliverables for the function test cycle. He delivers a test plan
(document) and a test schedule (work plan)—these often undergo several revisions during the
testing cycle.
Partitioning/functional decomposition Functional decomposition of a system (or partitioning) is
the breakdown of a system into its functional components or functional areas. Another group in
the organization may take responsibility for the functional decomposition (or model) of the
system, but the testing organization should still review this deliverable for completeness before
accepting it into the test organization. If the functional decompositions or partitions have not
been defined or are deemed insufficient, then the testing organization will have to take
responsibility for creating and maintaining the partitions.
Requirement definition The testing organization needs specified requirements in the form of
proper documents to proceed with the function test. These requirements need to be itemized
under an appropriate functional partition.
Test case design A tester designs and implements a test case to validate that the product
performs in accordance with the requirements. These test cases need to be itemized
under an appropriate functional partition and mapped/ traced to the requirements being tested.
Traceability matrix formation Test cases need to be traced/mapped back to the appropriate
requirement. A function coverage matrix is prepared. This matrix is a table, listing specific functions to
be tested, the priority for testing each function, and test cases that contain tests for each function.
Once all the aspects of a function have been tested by one or more test cases, then the test design
activity for that function can be considered complete. This approach gives a more accurate picture
of the application when coverage analysis is done.
For example, in Table, function F2 test cases must be executed, as its priority is highest; and through
the function coverage matrix, we can track which functions are being tested through which test
cases.

Test case execution As in all phases of testing, an appropriate set of test cases need to be executed
and the results of those test cases recorded. Which test cases are to be executed should be defined
within the context of the test plan and the current state of the application being tested.
 SYSTEM TESTING

When all the modules have been integrated with the software system, then the software is
incorporated with other system elements (e.g. hardware, people, information).

System testing is the process of attempting to demonstrate that a program or system does not
meet its original requirements and objectives, as stated in the requirement specification. The
central purpose of system testing is to compare the system with its stated objectives.

System testing is actually a series of different tests to test the whole system on various grounds
where bugs have the probability to occur. The ground can be performance, security, maximum
load, etc. The integrated system is passed through various tests based on these grounds and
depending on the environment and type of project. After passing through these tests, the
resulting system is a system which is ready for acceptance testing which involves the user.
But it is difficult to test the system on various grounds, since there is no methodology for system
testing. One solution to this problem is to think from the perspective of the user and the problem
the user is trying to solve.
This dilemma is solved by using the program’s user documentation. Design the system test by
analysing the objectives; formulate test cases by analysing the user documentation. This has a
useful side-effect of comparing the program with its objectives and the user documentation, as
well as comparing the user documentation with the objectives

Fig: System testing

 CATEGORIES OF SYSTEM TESTS

Recovery Testing Recovery is just like the exception handling feature of a programming language. It is
the ability of a system to restart operations after the integrity of the application has been lost. It reverts to
a point where the system was functioning correctly and then, reprocesses the transactions to the point of
failure.

Some software systems (e.g. operating system, database management systems, etc.) must recover from
programming errors, hardware failures, data errors, or any disaster in the system. So the purpose of this
type of system testing is to show that these recovery functions do not work correctly.

The main purpose of this test is to determine how good the developed software is when it faces a
disaster. Disaster can be anything from unplugging the system which is running the software from power,
network etc., also stopping the database, or crashing the developed software itself.
Some examples of recovery testing are given below:
➢ While the application is running, suddenly restart the computer and thereafter, check the validity
of application’s data integrity.
➢ While the application receives data from the network, unplug the cable and plug-in after awhile, and analyse
the application’s ability to continue receiving data from that point, when the network connection
disappeared.
➢ Restart the system while the browser has a definite number of sessions and after rebooting, check that it is able
to recover all of them.
Beizer proposes that testers should work on the following areas during recovery testing:
Restart If there is a failure and we want to recover and start again, then fi rst the current system state and
transaction states are discarded. Following the criteria of checkpoints as discussed above, the most recent
checkpoint record is retrieved and the system is initialized to the states in the checkpoint record. Thus, by using
checkpoints, a system can be recovered and started again from a new state. Testers must ensure that all
transactions have been reconstructed correctly and that all devices are in proper states. The system now is in a
position to begin to process new transactions.
Switchover Recovery can also be done if there are standby components and in case of failure of one
component, the standby takes over the control. The ability of the system to switch to a new component must be
tested.
 Security Testing

Security is a protection system that is needed to assure the customers that their data will be protected. For example, if Internet users feel
that their personal data/information is not secure, the system loses its accountability. Security may include controlling access to data,
encrypting data in communication, ensuring secrecy of stored data, auditing security events, etc. The effects of security breaches could
be extensive and can cause loss of information, corruption of information, misinformation, privacy violations, denial of service, etc.

Types of Security Requirements:

❑ Security requirements should be associated with each functional requirement. Each functional requirement, most likely, has a specifi c
set of related security issues to be addressed in the software implementation. For example, the log-on requirement in a client-server
system must specify the number of retries allowed, the action to be taken if the log-on fails, and so on

❑ A software project has security issues that are global in nature, and are therefore, related to the application’s architecture and
overall implementation. For example, a Web application may have a global requirement that all private customer data of any kind is
stored in encrypted form in the database.
Security vulnerabilities Vulnerability is an error that an attacker can exploit.
Security vulnerabilities are of the following types:
❖ Bugs at the implementation level, such as local implementation errors or inter-procedural interface errors
❖ Design-level mistakes. Design-level vulnerabilities are the hardest defect category to handle, but they are
also the most prevalent and critical. Examples of design-level security flaws include problem in error-
handling, unprotected data channels, incorrect or missing access control mechanisms, and timing errors
especially in multithreaded systems.
How to perform security testing Testers must use a risk-based approach, grounded in both the system’s
architectural reality and the attacker’s mindset, to gauge software security adequately. By identifying risks and
potential loss associated with those risks in the system and creating tests driven by those risks, the tester can
properly focus on areas of code in which an attack is likely to succeed.
Risk management and security testing Software security practitioners perform many different tasks to manage
software security risks, including:
1. Creating security abuse/misuse cases
2. Listing normative security requirements
3. Performing architectural risk analysis
4. Building risk-based security test plans
5. Wielding static analysis tools
6. Performing security tests
Thus, security testing must necessarily involve two diverse approaches:
▪ Testing security mechanisms to ensure that their functionality is properly implemented
▪ Performing risk-based security testing motivated by understanding and simulating the
attacker’s approach
Elements of security testing The basic security concepts that need to be covered by security
testing are discussed below:
Confidentiality A security measure which protects against the disclosure of information to parties
other than the intended recipient. Integrity A measure intended to allow the receiver to
determine that the information which it receives has not been altered in transit or by anyone
other than the originator of the information.
Integrity schemes often use some of the same underlying technologies as confidentiality
schemes, but they usually involve adding additional information to a communication to form
the basis of an algorithmic check rather than encoding all the communication.
Authentication A measure designed to establish the validity of a transmission, message, or
originator. It allows the receiver to have confidence that the information it receives originates
from a specific known source.
Authorization It is the process of determining that a requester is allowed to receive a service or
perform an operation. Access control is an example of authorization. Availability It assures that
the information and communication services will be ready for use when expected. Information
must be kept available for authorized persons when they need it.
Non-repudiation A measure intended to prevent the later denial that an action happened, or
a communication took place, etc. In communication terms, this often involves the interchange
of authentication information combined with some form of provable timestamp.
 Performance Testing

Performance testing is a type of non-functional testing aimed at determining how a system

performs under a particular workload. It measures the responsiveness, stability, scalability, and
speed of the application.
Objectives:
1. Identify performance bottlenecks.
2. Ensure the system meets performance requirements.
3. Validate system capacity under different load conditions.
4. Assess the stability of the system under sustained use.
Types of Performance Testing:
• Load Testing: Evaluates system behavior under anticipated load.
• Stress Testing: Determines the system's capacity to handle extreme loads.
• Endurance Testing: Checks system stability over an extended period.
• Scalability Testing: Measures the system's ability to scale up or down as demand changes.
• Volume Testing: Assesses the system's performance with large data volumes.
• Spike Testing: Examines system response to sudden spikes in load.

Key Performance Metrics:

1. Response Time: The time taken for the system to respond to a user request.
2. Throughput: The number of transactions processed per unit of time.
3. Resource Utilization: Monitoring CPU, memory, disk I/O, and network usage.
4. Concurrency: Assessing the system's ability to handle multiple users simultaneously.
5. Error Rate: Monitoring the frequency of errors or timeouts.
Load Testing

Load testing is a type of non-functional testing. A load test is type of software testing which
is conducted to understand the behavior of the application under a specific expected
load. When a system is tested with a load that causes it to allocate its resources in
maximum amounts, it is called load testing. The idea is to create an environment more
demanding than the application would experience under normal workloads. Load testing
checks how a system performs when many users are using it at the same time. To make
sure the system can handle busy times without slowing down or crashing.
Load Testing Process:

1.Test Environment Setup: Firstly create a dedicated test

environment setup for performing the load testing. It ensures
that testing would be done in a proper way.
2.Load Test Scenario: In second step load test scenarios are
created. Then load testing transactions are determined for
an application and data is prepared for each transaction.
3.Test Scenario Execution: Load test scenarios that were
created in previous step are now executed. Different
measurements and metrices are gathered to collect the
information.
4.Test Result Analysis: Results of the testing performed is
analyzed and various recommendations are made.
5.Re-test: If the test is failed then the test is performed again
in order to get the result in correct way
Examples:

1. 1. E-commerce Website: Simulate hundreds of users browsing, adding items to carts, and
checking out simultaneously.
2. Social Media Platform: Test thousands of users posting updates, sharing content, and
interacting with each other concurrently.
3. Online Banking System: Simulate numerous users logging in, checking balances,
transferring funds, and paying bills concurrently.
 Stress Testing

Stress testing is also a type of load testing, but the difference is that the system is put under loads
beyond the limits so that the system breaks. Thus, stress testing tries to break the system under test
by overwhelming its resources in order to find the circumstances under which it will crash. The areas
that may be stressed in a system are:

➢ Input transactions

➢ Disk space

➢ Output

➢ Communications

➢ Interaction with users

Stress testing is important for real-time systems where unpredictable events may occur, resulting
in input loads that exceed the values described in the specification, and the system cannot
afford to fail due to maximum load on resources. Therefore, in real-time systems, the entire
threshold values and system limits must be noted carefully. Then, the system must be stress-
tested on each individual value.
Stress testing demands the amount of time it takes to prepare for the test and the amount of
resources consumed during the actual execution of the test. Therefore, stress testing cost must
be weighed against the risk of not identifying volume-related failures. For example, many of the
defense systems which are real-time in nature, demand the systems to perform continuously in
warfare conditions. Thus, for a real-time defense system, we must stress-test the system;
otherwise, there may be loss of equipment as well as life.
Usability Testing

Usability testing identifies discrepancies between the user interfaces of a product and the
human engineering requirements of its potential users. This type of system testing is related
to a system’s presentation rather than its functionality. System testing can be performed
to find human-factors or usability problems on the system. The idea is to adapt the
software to users’ actual work styles rather than forcing the users to adapt according to
the software. Thus, the goal of usability testing is to verify that intended users of the system
are able to interact properly with the system while having a positive and convenient
experience.
What the user wants or expects from the system can be determined using several
ways like:
Area experts The usability problems or expectations can be best understood by the subject or
area experts who have worked for years in the same area. They analyse the system’s specific
requirements from the user’s perspective and provide valuable suggestions.
Group meetings Group meeting is an interesting idea to elicit usability requirements from the
user. These meetings result in potential customers’ comments on what they would like to see in
an interface.
Survey Surveys are another medium to interact with the user. It can also yield valuable
information about how potential customers would use a software product to accomplish their
tasks.
Analyse similar products We can also analyse the experiences in similar kinds of projects done
previously and use it in the current project. This study will also give us clues regarding usability
requirements.
Usability characteristics against which testing is conducted are discussed below:
Ease of use The users enter, navigate, and exit with relative ease. Each user interface
must be tailored to the intelligence, educational background, and environmental pressures of the
end-user.
Interface steps User interface steps should not be misleading. The steps should also not be very
complex to understand either.
Response time The time taken in responding to the user should not be so high that the user is
frustrated or will move to some other option in the interface.
Help system A good user interface provides help to the user at every step. The help
documentation should not be redundant; it should be very precise and easily understood by
every user of the system.
Error messages For every exception in the system, there must be an error message in text form so
that users can understand what has happened in the system. Error messages should be clear and
meaningful.
 Compatibility/Conversion/Configuration Testing:

Configuration testing allows developers/testers to evaluate system performance and

availability when hardware exchanges and reconfigurations occur. The number of possible
combinations for checking the compatibilities of available hardware and software can be
too high, making this type of testing a complex job. Compatibility testing is to check the
compatibility of a system being developed with different operating system, hardware and
software configuration available, etc.
Guidelines for compatibility testing:
Operating systems The specifications must state all the targeted end-user operating
systems on which the system being developed will be run.
Software/Hardware The product may need to operate with certain versions of Web browsers, with hardware
devices such as printers, or with other software such as virus scanners or word processors.
Conversion testing Compatibility may also extend to upgrades from previous versions of the software. So, It
should be specified whether the new system will be backward compatible with the previous version. Also, if
other user’s preferences or settings are to be preserved or not.
Ranking of possible configurations The testers must rank the possible configurations in order, from the most to
the least common, for the target system.
Identification of test cases Testers must identify appropriate test cases and data for compatibility testing. It is
necessary to select the most representative set of test cases that confirms the application’s proper functioning
on a particular platform.
Updating the compatibility test cases
(i) Tech-support calls provide a good source of information for updating the compatibility test suite.
(ii) A beta-test program can also provide a large set of data regarding real end-user configurations and
compatibility issues prior to the final release of a system.
 ACCEPTANCE TESTING

Developers/testers must keep in mind that the software is being built to satisfy the user requirements
and no matter how elegant its design is, it will not be accepted by the users unless it helps them
achieve their goals as specified in the requirements. The purpose of acceptance testing is to give the
end-user a chance to provide the development team with feedback as to whether or not the software
meets their needs. Ultimately, it’s the user that needs to be satisfied with the application, not the
testers, managers, or contract writers.

Thus, acceptance testing is the formal testing conducted to determine whether a software
system satisfies its acceptance criteria and to enable buyers to determine whether to accept the
system or not. It determines whether the developed system meets the predetermined functionality,
performance, quality, and interface criteria acceptable to the user. User acceptance testing, on the
other hand, should be carried out by end-users.
Thus, acceptance testing is designed to:
1. Determine whether the software is fit for the user.
2. Making users confident about the product.
3. Determine whether a software system satisfies its acceptance criteria.
4. Enable the buyer to determine whether to accept the system or not.
Entry Criteria
❑ System testing is complete and defects identified are either fixed or documented.
❑ Acceptance plan is prepared and resources have been identified.
❑ Test environment for the acceptance testing is available.
Exit Criteria
❑ Acceptance decision is made for the software.
❑ In case of any warning, the development team is notified.
Types of Acceptance Testing:
Acceptance testing is classified into the following two categories:
1. Alpha Testing
2. Beta Testing
 ALPHA TESTING

Alpha is the test period during which the product is complete and usable in a test environment,
but not necessarily bug-free. It is the final chance to get verification from the customers that the
tradeoffs made in the final development stage are coherent.

Therefore, alpha testing is typically done for two reasons:

(i) to give confidence that the software is in a suitable state to be seen by the customers.

(ii) to find bugs that may only be found under operational conditions. Any other major defects or
performance issues should be discovered in this stage. Since alpha testing is performed at the
development site, testers and users together perform this testing. Therefore, the testing is in a
controlled manner such that if any problem comes up, it can be managed by the testing
team.
Entry Criteria to Alpha:
▪ All features are complete/testable (no urgent bugs).
▪ High bugs on primary platforms are fixed/verified.
▪ 50% of medium bugs on primary platforms are fixed/verified.
▪ All features have been tested on primary platforms.
▪ Performance has been measured/compared with previous releases (user
functions).
▪ Usability testing and feedback (ongoing).
▪ Alpha sites are ready for installation.
Exit Criteria from Alpha
▪ Get responses/feedbacks from customers.
▪ Prepare a report of any serious bugs being noticed
▪ Notify bug-fixing issues to developers.
 BETA TESTING

Beta is the test period during which the product should be complete and usable in a production
environment.
The purpose of the beta ship and test period is to test the company’s ability to deliver and support
the product (and not to test the product itself). Beta also serves as a chance to get a final ‘vote of
confidence’ from a few customers to help validate our own belief that the product is now ready
for volume shipment to all customers.
Versions of the software, known as beta-versions, are released to a limited audience outside the
company. The software is released to groups of people so that further testing can ensure the
product has few or no bugs. Sometimes, beta-versions are made available to the open public.
Testing during the beta phase, informally called beta testing, is generally constrained to black-box
techniques The term beta test can refer to a stage of the software—closer to release than being ‘in
alpha’—or it can refer to the particular group and process being done at that stage.
Entry Criteria to Beta:
➢ Positive responses from alpha sites.
➢ Customer bugs in alpha testing have been addressed.
➢ There are no fatal errors which can affect the functionality of the software.
➢ Secondary platform compatibility testing is complete.
➢ Regression testing corresponding to bug fixes has been done.
➢ Beta sites are ready for installation.
Exit Criteria from Beta
➢ Get responses/feedbacks from the beta testers.
➢ Prepare a report of all serious bugs.
➢ Notify bug-fi xing issues to developers.
Guidelines for Beta Testing
➢ Don’t expect to release new builds to beta testers more than once every two
weeks.
➢ Don’t plan a beta with fewer than four releases.
 PROGRESSIVE VS. REGRESSIVE TESTING

From verification to validation, the testing process progresses towards the release of the product.

However, to maintain the software, bug fixing may be required during any stage of development
and therefore, there is a need to check the software again to validate that there has been no
adverse effect on the already working software.

A system under test ( SUT) is said to regress if

I. a modified component fails, or

II. a new component, when used with unchanged components, causes failures in the
unchanged components by generating side-effects or feature interactions.
Therefore, now the following versions will be there in the system:
Baseline version The version of a component (system) that has passed a test suite.
Delta version A changed version that has not passed a regression test.
Delta build An executable configuration of the SUT that contains all the delta and baseline
components.
Thus, it can be said that most test cases begin as progressive test cases and eventually
become regression test cases. Regression testing is not another testing activity. Rather, it is
the re-execution of some or all of the already developed test cases.
Regression testing

Regression testing is the selective retesting of a system or component to verify that

modifications have not caused unintended effects and that the system or component still
complies with its specified requirements.

The purpose of regression testing is to ensure that bug-fixes and new functionalities
introduced in a new version of the software do not adversely affect the correct
functionality inherited from the previous version. It may be possible that small changes in
one part of a program may have subtle undesired effects on other unrelated modules of
the software. It is not necessary that if you are getting the desired outputs on the screen,
then there is no problem.
Importance of regression testing
i. It validates the parts of the software where changes occur.
ii. It validates the parts of the software which may be affected by some changes, but
otherwise unrelated.
iii. It ensures proper functioning of the software, as it was before changes occurred.
iv. It enhances the quality of software, as it reduces the risk of high-risk bugs.

Fig: Regression testing produces Quality Software

 Regression Testability

Regression testability refers to the property of a program, modification, or test suite that lets it be
effectively and efficiently regression-tested.

Leung and White classify a program as regression testable if most single statement modifications
to the program entail rerunning a small proportion of the current test suite. Under this definition,
regression testability is a function of both the design of the program and the test suite.

To consider regression testability, a regression number is computed. It is the average number of

affected test cases in the test suite that are affected by any modification to a single instruction.
This number is computed using information about the test suite coverage of the program.

If regression testability is considered at an early stage of development, it can provide significant

savings in the cost of development and maintenance of the software.
 Objectives of Regression testing

 It tests to check that the bug has been addressed The first objective in bug fix testing is to
check whether the bug-fixing has worked or not. Therefore, you run exactly the same test
that was executed when the problem was first found. If the program fails on this testing, it
means the bug has not been fixed correctly and there is no need to do any regression
testing further.
 It finds other related bugs It may be possible that the developer has fixed only the symptoms
of the reported bugs without fixing the underlying bug. Moreover, there may be various ways
to produce that bug. Therefore, regression tests are necessary to validate that the system
does not have any related bugs.
 It tests to check the effect on other parts of the program It may be possible that bug-fixing has
unwanted consequences on other parts of a program. Therefore, regression tests are
necessary to check the influence of changes in one part on other parts of the program.
 When Regression testing is done?

 Software Maintenance
▪ Corrective maintenance Changes made to correct a system after a failure has been observed
▪ Adaptive maintenance Changes made to achieve continuing compatibility with the target environment or
other systems.
▪ Perfective maintenance Changes designed to improve or add capabilities.
▪ Preventive maintenance Changes made to increase robustness, maintainability, portability, and other features.
► Rapid Iterative Development
The extreme programming approach requires that a test be developed for each class and that this test be re-run
every time the class changes.
► Compatibility Assessment and Benchmarking
Some test suites are designed to be run on a wide range of platforms and applications to establish conformance
with a standard or to evaluate time and space performance. These test suites are meant for regression testing, but
not intended to reveal regression bugs.
Regression testing types

 Bug-Fix regression This testing is performed after a bug has been reported and fixed. Its

goal is to repeat the test cases that expose the problem in the first place.

 Side-Effect regression/Stability regression It involves retesting a substantial part of the

product. The goal is to prove that the change has no detrimental effect on something

that was earlier in order. It tests the overall integrity of the program, not the success of

software fixes.
 Regression testing techniques

There are three different techniques for regression testing. They are discussed below.

❖ Regression test selection technique This technique attempt to reduce the time required to
retest a modified program by selecting some subset of the existing test suite.

❖ Test case prioritization technique Regression test prioritization attempts to reorder a

regression test suite so that those tests with the highest priority, according to some
established criteria, are executed earlier in the regression testing process rather than those
with lower priority. There are two types of prioritization:
❖ General Test Case Prioritization For a given program P and test suite T, we prioritize the
test cases in T that will be useful over a succession of subsequent modified versions of P,
without any knowledge of the modified version.

❖ Version-Specific Test Case Prioritization We prioritize the test cases in T, when P is

modified to P’, with the knowledge of the changes made in P

❖ Test suite reduction technique It reduces testing costs by permanently eliminating

redundant test cases from test suites in terms of codes or functionalities exercised.
Notations used in Regression testing

✓ P denotes a program or procedure

✓ P’ denotes a modified version of P
✓ S denotes the specification for program P
✓ S’ denotes the specification for program P’
✓ P(i) refer to the output of P on input i
✓ P’(i) refer to the output of P’ on input i
✓ and T = {t1,….. tn} denotes a test suite or test set for P.
 SELECTIVE RETEST TECHNIQUE

Selective retest techniques attempt to reduce the cost of testing by identifying the portions of P’
that must be exercised by the regression test suite. Selective retesting is distinctly different from a
retest-all approach that always executes every test in an existing regression test suite ( RTS). Thus,
the objective of selective retest technique is cost reduction. It is the process of selecting a subset
of the regression test suite that tests the changes.

characteristic features of the selective retest technique:

➢ It minimizes the resources required to regression test a new version.
➢ It is achieved by minimizing the number of test cases applied to the new version.
➢ It is needed because a regression test suite grows with each version, resulting in broken,
obsolete, uncontrollable, redundant test cases.
➢ It analyses the relationship between the test cases and the software elements they cover.
➢ It uses the information about changes to select test cases.
Steps taken by this technique in the following manner

Fig: Selective retest technique

1. Select T ‘ subset of T, a set of test cases to execute on P ‘.

2. Test P ‘ with T ‘, establishing correctness of P ‘ with respect to T ‘.
3. If necessary, create T “, a set of new functional or structural test cases for P ‘.
4. Test P ‘ with T “, establishing correctness of P ‘ with respect to T “.
5. Create T ‘’’, a new test suite and test execution profile for P ‘, from T,and T “.
Each of these steps involves the following important problems:
Regression test selection problem Step 1 involves this problem. The problem
is to select a subset T ‘ of T with which P ‘ will be tested.
Coverage identification problem Step 3 addresses this problem. The problem is
to identify portions of P ‘ or its specifications that requires additional testing.
Test suite execution problem Steps 2 and 4 address the test suite execution
problem. The problem is to execute test suites efficiently and checking test
results for correctness.
Test suite maintenance problem Step 5 addresses this problem. The problem
is to update and store test information.
Strategy for Test Case Selection
Effective testing requires strategies to trade-off between the two opposing needs of amplifying
testing thoroughness on one side (for which a high number of test cases would be desirable) and reducing
time and costs on the other (for which the fewer the test cases, the better). Given that test resources are
limited, how the test cases are selected is of crucial importance. Indeed, the problem of test cases selection
has been the largely dominating topic in software testing research to the extent that in the literature ‘software
testing’ is often taken as synonymous for ‘test case selection’. A decision procedure for selecting the test cases
is provided by a test criterion.
Selection Criteria Based on Code
The motivation for code-based testing is that potential failures can only be detected if the parts of code that
can cause faults are executed. All the code based regression test selection techniques attempt to select a
subset T ‘ of T that will be helpful in establishing confidence that P ‘s functionality has been preserved where
required. In this sense, all code-based test selection techniques are concerned with locating tests in T that
expose faults in P ‘. The following tests are based on these criteria.
Fault-revealing test cases A test case t detects a fault in P ‘ if it causes P ‘ to fail. Hence t is fault-revealing for P ‘. There is
no effective procedure to find the tests in T that are fault-revealing for P ‘. Under certain conditions, however, a
regression test selection technique can select a superset of the tests in T that are fault-revealing for P ‘ Under these
conditions, such a technique omits no tests in T that can reveal faults in P ‘.
Modification-revealing test cases A test case t is modification-revealing for P and P ‘ if and only if it causes the outputs of
P and P ‘ to differ.
Modification-traversing test cases A test case t is modification-traversing if and only if it executes new or modified code in
P ‘. These tests are useful to consider because a non-obsolete test t in T can only be modification-revealing for P and P ‘
if it is modification-traversing for P and P ‘.
Regression Test Selection Techniques
A variety of regression test selection techniques have been described in the research literature. We consider the following selective retest
techniques:
Minimization techniques Minimization-based regression test selection techniques attempt to select minimal sets of test cases from T that yield
coverage of modified or affected portions of P.
Dataflow techniques Dataflow coverage-based regression test selection techniques select test cases that exercise data interactions that have
been affected by modifications.
Safe techniques Most regression test selection techniques, minimization and dataflow techniques among
them, are not designed to be safe. Techniques that are not safe can fail to select a test case that would have revealed a fault in the modified
program. In contrast, when an explicit set of safety conditions can be satisfied, safe regression test selection techniques guarantee that the
selected subset T ‘ contains all test cases in the original test suite T that can reveal faultsin P ‘.
Ad hoc/Random techniques When time constraints prohibit the use of a retestall approach, but no test selection tool is available, developers
often select test cases based on ‘intuitions’ or loose associations of test cases with functionality. Another simple approach is to randomly select
a predetermined number of test cases from T.
Retest-all technique The retest-all technique simply reuses all existing test cases. To test P ‘, the technique effectively selects all test cases in T.
Evaluating Regression Test Selection Techniques
To choose a technique for practical application, Rothermel et al. [51] have recognized the following categories for
evaluating the regression test selection techniques:
Inclusiveness It measures the extent to which M chooses modification revealing test from T for inclusion in T ‘,
where M is a regression test selection technique.
Suppose, T contains n tests that are modification-revealing for P and P ‘,
and suppose M selects m of these tests. The inclusiveness of M relative to P and P ‘ and T is:
1. INCL(M) = (100 \ (m/n)%, if n ≠ 0
2. INCL(M)% = 100%, if n = 0
For example, if T contains 100 tests of which 20 are modification-revealing for P and P ‘, and M selects 4 of these
20 tests, then M is 20% inclusive relative to P, P ‘, and T. If T contains no modification-revealing tests, then every
test selection technique is 100% inclusive relative to P, P ‘, and T. If for all P, P ‘, and T, M is 100% inclusive
relative to P, P ‘, and T, then M is safe.
Precision It measures the extent to which M omits tests that are non-modification revealing.
Suppose T contains n tests that are non-modification revealing for P and P ‘, and suppose M omits m of these tests. The
precision of M relative to P, P ‘, and T is given by
Precision = 100 \ (m/n) % if n ≠ 0
Precision = 100 % if n = 0
For example, if T contains 50 tests of which 22 are non-modification-revealing for P and P ‘, and M omits 11 of these 22
tests, then M is 50% precise relative to P, P ‘, and T. If T contains no non-modification-revealing tests, then every test
selection technique is 100% precise relative to P, P ‘, and T.
Efficiency The efficiency of regression test selection techniques is measured in terms of their space and time requirements.
Where time is concerned, a test selection technique is more economical than the retest-all technique, if the cost of selecting
T ‘ is less than the cost of running the tests in T–T ‘. Space efficiency primarily depends on the test history and program
analysis information a technique must store. Thus, both space and time efficiency depends on the size of the test suite that a
technique selects, and on the computational cost of that technique.
REGRESSION TEST PRIORITIZATION
The regression test prioritization approach is different as compared to selective retest techniques. The prioritization
methods order the execution of RTS with the intention that a fault is detected earlier. In other words, regression test
prioritization attempts to reorder a regression test suite so that those tests with the highest priority, according to
some established criterion, are executed earlier in the regression testing process than those with a lower priority. By
prioritizing the execution of a regression test suite, these methods reveal important defects in a software system
earlier in the regression testing process.
This approach can be used along with the previous selective retest technique.
The steps for this approach are given below:
1. Select T ‘ from T, a set of test to execute on P ‘.
2. Produce T ‘p , a permutation of T ‘, such that T ‘p will have a better rate of fault detection than T ‘.
3. Test P ‘ with T ‘p in order to establish the correctness of P ‘ with respect to T ‘p .
4. If necessary, create T “, a set of new functional or structural tests for P “.
5. Test P ‘ with T “ in order to establish the correctness of P’ with respect to T “.
6. Create T ‘’’, a new test suite for P ‘, from T, T ‘p , and T “.
Step 1 allows the tester to select T ‘such that it is a proper subset of T or that it actually contains every test that T contains.
Furthermore, step 2 could produce T ‘p so that it contains an execution ordering for the regression test that
is the same or different than the order provided by T ‘. The intention behind prioritizing the execution of a regression test
suite is to ensure that the defects normally introduced by competent programmers are discovered earlier in the
testing process