International Software Testing Qualification Broad

Chapter 1: Fundamentals of Testing

Main Contents
Why is Testing

ecessary!
What is Testing! Se"en Testing #rinciples Fundamental Test

#rocess

Why is Testing necessary!

Cause of software

defects
$ole of Testing in

Software %e"elopment& Maintenance and 'perations

Testing and Quality (ow much Testing is

)nough!

%ifferentiate *)rror+ %efect+Failure,

 - person ma.es an error

*mista.e,
#roduces a defect *fault&

/ug,
Causes a failure in an

operation

Where /ugs come from!

#eople create defects

*/ugs, into the system

Code */usiness logic and

user interface,

 %ocumentation

*$e0uirements and design specifications in all electronic and hard document, When the

implementation of these defects */ugs, are e1ecuted& failures occur

Which main cause of /ugs are!

Bugs occur due to2
#rogrammer& analyst&

and other indi"idual contri/utor *including tester, falli/ility

 Time pressure

*%eadlines,
Comple1ity of

$e0uirement3Technology
4ac. of e1perience3s.ill 4ac. of information Fre0uent changes

Failures occur due to

/ugs and en"ironmental conditions *radiation& magnetism& pollution2,

The conse0uence of defects

4eads to In5ury3%eath 4eads to loss of Time

 4eads to loss of money 4eads to Bad reputation 4eads to #ollution

$ole of testing in software de"elopment life cycle

Identify defects $educe failures or ris.s

in operation
Impro"e the 0uality of

software system

 Software testing may

also /e re0uired to compliance with:

Contractual or legal

re0uirements
Industry+specific

standards such as:

BS

6789+8:177:; Software Component Testing :87:177:; Standard for Software Test %ocumentation

I)))

Testing and Quality

Testing measures the

software 0uality /y:

The num/er of defects

found

 The num/er of test cases

run
The percentage of

software code e1ercised

#assing tests reduce the

le"el of 0uality ris.

Failing tests pro"ide a

chance to impro"e 0uality <= Impro"e and gi"e confidence software 0uality

 Testing can /e done for

Functional and on+ functional software re0uirements and characteristics *relia/ility& usa/ility& efficiency& maintaina/ility and porta/ility,

Testing and Quality *cont;;,

Quality is measured /y

loo.ing at the attri/utes of the product; *e;g; software relia/ility,

 Quality is fitness for use Quality is /ased on

good manufacturing processes and meet defined re0uirements e1pectation of time& money and effort aspects used efficiently and effecti"ely the feelings of an indi"idual;

Quality includes

Quality depended on

(ow much Testing is enough!

 %epends on le"el of

$IS>
$is. of missing important

faults
$is. of incurring failure

costs
$is. of releasing

untested or under+tested software

$is. of losing credi/ility

and mar.et share

$is. of missing a mar.et

window
$is. of o"er+testing&

ineffecti"e testing

<= #rioriti?e testing

including comple1ity& criticality& "isi/ility and relia/ility; %epends on Time and

Budget *#ro5ect Constraints,

What is Testing
Common o/5ecti"es of

Testing

The purpose of testing

in software de"elopment& maintenance and operations

 %ifferentiate Testing

from %e/ugging

Common '/5ecti"es of Testing

Finding defects @aining confidence

a/out the le"el of 0uality

#ro"iding information

for decision+ma.ing
#re"enting defects

The purpose of Testing and Test #hases

 Anit3Component Test
Find /ugs in the

indi"idual pieces of the system under test /efore the pieces are fully integrated into the system Integration Test
Find /ugs in the

relationships and interfaces /etween pairs and groups of components in the system under test as the pieces come together System Test

 Find /ugs in the o"erall

and particular /eha"iors& functions& and responses of the system under test as a whole -cceptance3#ilot Test
%emonstrate that the

product is ready for deployment3release or to assess 0uality and gi"e information on the ris. of deployment3release Maintenance Test
Chec. for errors

introduced during de"elopment of the changes

 'perational Test
-ssess non+functional

system characteristics such as relia/ility or a"aila/ility

%ifferentiate *Testing+ %e/ugging,

Testing:
Find defect $etest /ugs fi1ed and

-pply regression test for o"erall system

Tester test

%e/ugging:

 Identify the cause of

/ugs
$epair the code Chec. that the code is

e1ecuted as e1pected
#rogrammer de/ugs

Testing #rinciples
Testing shows presence

of defects

)1hausti"e testing is

impossi/le

)arly testing %efect clustering #esticide parado1

 Testing is conte1t

dependent
-/sence of errors

fallacy

Fundamental Test #rocess

Test #lanning and

Control
Test -nalysis and

%esign
Test Implementation

and )1ecution

 )"aluating )1it Criteria

and $eporting
Test Closure -cti"ities

Test #lanning and Control

Planning
%etermine test scope&

ris.s& o/5ecti"es& strategies

%etermine Test

-pproach

%etermine re0uired test

resources

 Implement the test

policies& test strategies

Schedule test analysis

and design
Schedule

implementation& e1ecution and e"aluation of tests

%etermine the test e1it

criteria Control
Measure and analy?er

results of re"iew and testing

 Monitor and document

progress& co"erage and test e1it criteria

#ro"ide information on

Testing

Initiate correcti"e

actions

Ma.e decisions

Test -nalysis and %esign

Analysis

 $e"iew the test /asis

*e;g; re0uirements or design specifications& networ.3system architecture& 0uality ris.s,

Identify test conditions&

test data /ased on analysis of test items& its specification& /eha"ior and structure

Design

 Select specific

com/inations of test data& actions& and e1pected results to co"er the test /asis <= test case& test procedure
)"aluate testa/ility of

the re0uirements and system

%esign the test

en"ironment

Identify any re0uired

infrastructure and tools

Test Implementation and )1ecution

Implementation
%e"elop and prioriti?e

test cases& create test data& write and prioriti?e test procedures
#repare test harnesses

and write automated test scripts

'rgani?e test suites and

se0uences of test cases for efficient test e1ecution

 Berify that the test

en"ironment has /een set up correctly

Berifying and updating

/i+directional tracea/ility /etween the test /asis and test cases

Execution
)1ecute test suites and

test case *manual or automated,

 4og test results& and the

"ersion of the software under test& test tools and the test ware
Compare actual and

e1pected results
$eport and analy?e

incidents
$epeat corrected and3or

updated tests
)1ecute confirmation

and3or regression tests

)"aluating e1it criteria and reporting

 Chec. test logs against

the test e1it criteria specified in test planning

-ssess if more tests are

needed or if the e1it criteria specified should /e changed

Write a test summary

report for sta.eholders

Test Closure -cti"ities

Chec.ing

which planned deli"era/les ha"e /een deli"ered

 Closing

incident reports or raising change records for any that remain open the acceptance of the system and archi"ing test ware& the test en"ironment and the test infrastructure for later reuse

%ocumenting

Finali?ing

 (anding

o"er the test ware to the maintenance organi?ation lessons learned to determine changes needed for future releases and pro5ects the information gathered to impro"e test maturity

-naly?ing

Asing

%ifferentiate *Confirmation Testing C $egression Testing,

Confirmation Testing

*$e+Testing,
)1ecuting test cases that

failed the last time they were run

Must /e e1actly

repeata/le
Asing the same inputs&

pre+condition and en"ironment

 If test now passes& fault

has /een fi1ed correctly or has it! $egression Testing

)1ecuting test cases that

ha"e /een e1ecuted /efore

Test all test cases that

passed and failed

It is performed when the

software or its en"ironment is changed

)nsure that defects has

not presented in new "ersion

Chapter 8: Testing Throughout the Software 4ife

Main Contents:
Software %e"elopment

Models
Test 4e"els Test Types Maintenance Testing

Software %e"elopment Models

Waterfall model

 B+Model Incremental

%e"elopment Model

#rototyping model $apid -pplication

%e"elopment *$-%,
$ational Anified #rocess

*$A#,
-gile de"elopment

Testing within a life cycle model

Se"eral characteristics

of good testing:

 For e"ery de"elopment

acti"ity there is a corresponding testing acti"ity

)ach test le"el has test

o/5ecti"es specific to that le"el

The analysis and design

of tests for a gi"en test le"el should /egin during the corresponding de"elopment acti"ity

 Testers should /e

in"ol"ed in re"iewing documents as soon as drafts are a"aila/le in the de"elopment cycle;

Test 4e"els
Key Contents
Component Testing Integration Testing System Testing -cceptance Testing

Component3Anit Testing

 Test Objective: Find

/ugs in the indi"idual pieces of the system under test prior to system integration
Test

asis: Component re0uirements& detailed design& code Components& #rograms& %ata con"ersion3Migration programs& %ata/ase module

Test Objects:

 Test Types: Functional&

performance& structural& ro/ustness testing

!esponsibility:

#rogrammer

Integration Testing
Test Objective: Find

/ugs in the relationships and interfaces /etween pairs and group of components in the system under test as the pieces come together

 Test

asis: Software and System %esign& -rchitecture& Wor.flow and Ase+cases Su/system data/ase implementation& Infrastructure& Interfaces

Test Objects:

Test Types: Functional&

performance
!esponsibility: Ideally

/oth Tester and %e"eloper

System Testing

 Test Objective: Find

/ugs in the o"erall and particular /eha"iors& functions& and responses of the system under test as a whole
Test

asis: System and software re0uirement specification& Ase cases& Functional specification& $is. analysis reports; user and operation manuals& System configuration

Test Objects: System&

 Test Types:

Functionality& security& performance& relia/ility& usa/ility& porta/ility2

!esponsibility: -n

independent test team

-cceptance Testing
Test Objective:

)sta/lish confidence in the system& parts of the system or specific non+ functional characteristics of the system

<= %emonstrate that the product is ready for deployment3release

Test

asis: Aser re0uirements& System re0uirements& Ase cases& Business processes& $is. analysis reports processes on fully integrated system& 'perational and maintenance processes& Aser procedures& Forms& $eports

Test Objects: Business

 Test Types: Aser

acceptance testing& 'perational acceptance testing& Contract and regulation acceptance testing& -lpha and Beta testing;
!esponsibility: the

customers or users of a system& other sta.eholders may /e in"ol"ed as well;

Test Types
Key Contents:
Functional Testing

 on+Functional Testing Structural Testing Confirmation and

$egression Testing

Functional Testing
Testing /ased on an

analysis of the specification of the functionality of a component or system; $eferred to /lac. /o1 testing and /y Tester

#erform at all Test 4e"el

 %ocuments: Functional

re0uirements& Functional specification& Ase cases

Functional Testing can

/e done from 8 perspecti"e:

testing

$e0uirement /ased Business+process+/ased

testing

on+Functional Testing

 Testing 0uality

characteristics: Dhow wellE or Dhow fastE the system wor.s

#erform at all Test

4e"els
on+Functional Testing

types: $elia/ility& Asa/ility& )fficiency& Maintaina/ility& #orta/ility

%ifferentiate /etween

#erformance Testing& 4oad Testing& Stress Testing

Structural Testing
Testing /ased on an

analysis of the internal structure of the component or system *while /o1 testing, thoroughness of testing through the co"erage of a set of structural elements or co"erage items

Measure the

Co"erage techni0ues:

code co"erage& statement co"erage& decision co"erage2

 #erform at all Test

le"els& especially in component testing or component integration testing

Maintenance Testing
Key contents:
$easons for

maintenance testing
Maintenance testing

"ersus new application testing

 $ole of regression

testing and impact analysis

$easons for Maintenance Testing

Test to identify pro/lem

after deploying system <= preser"e system 0uality

Maintenance testing

addresses the change itself and what wasnFt changed and shouldnFt change

 The maintenance of

e1isting software can account for o"er GH percent of all de"elopment effort
6HI of the cost of

software is de"oted to maintenance for maintenance testing: Modification& Migration& $etirement

Three typical triggers

Compare /etween Maintenance testing and new application testing $ole of regression testing and impact analysis
Software change impact

analysis is the acti"ity of the software maintenance process that determines possi/le effects of proposed software changes

 This acti"ity is

necessary to /e aware of unintentional+effects caused /y the change and record them so that nothing is o"erloo.ed
- change has not only

impact on the source code& /ut also on the other related software artifacts& such as re0uirements& design& and test

 Impact analysis can /e

efficiently supported through tracea/ility information

$egression test to show

that the rest of the system has not /een affected /y the maintenance wor.;