Testing Practice (TP)

Introduction to Software Testing

10/07/2013
 10/07/2013
Topics to be covered Testing Practice
Public

▶ What is Testing?
▶ Objectives of Testing?
▶ Why Testing is Necessary ?
▶ Product Risk and Project Risk
▶ The Psychology of Testing
▶ Role of a Tester.
▶ What makes a good test engineer?

2
 10/07/2013
What is Testing Testing Practice
Public

The Definition of Testing

▶ The process of exercising software to verify
that it satisfies specified requirements and
to detect errors.
– It is a risk reduction process - the more
risks, more testing
– It is aimed at discovering faults - the
removal of faults increases the software
quality
– It is an iterative process
– Testing is the measurement of software quality
– Testing can not ensure defect free system

Note :There is not one set of standard testing terms within the worldwide
testing community, which has led to different terms being used by different
organizations.
BS 7925-1 is now becoming widely accepted.

3
 10/07/2013
Objectives of Testing Testing Practice
Public

▶ What are the objectives of testing?

– Systems are fit for purpose
– Systems do what is expected of them
– Systems do not do what is not expected of them.

▶ Typically testing is done to find faults in the behavior of systems, to assess

whether the system is ready for release, to raise the confidence that the system
works and to provide quantifiable results on which management can assess the
risks of implementing a system.

▶ It is performed to demonstrate that a system meets its specification and does

not display incorrect behavior when used as intended. But equally we must not
forget the reverse of that and remember to test for invalid scenarios/situations -
this is also known as negative testing. This is effectively guessing at what might
go wrong.

4
 10/07/2013
Why Testing is Necessary Testing Practice
Public

▶ We will cover:
– Why is testing necessary?
– Errors and how they occur
– The cost of errors
– Why exhaustive testing is impossible
– Quality attributes of a system
– Testing and Risk
– Testing and Quality
• Quality Control
• Quality Assurance
– Testing and Requirements
– How much Testing is enough ?
– Everything needs to be tested

5
 10/07/2013
Why Testing is Necessary Testing Practice
Public

▶ To err is human, to test is essential.

▶ Perfection is very difficult!!

▶ People make errors , Errors can cause problems
▶ Deliverables can be defective
▶ Defects can cause failures , Failure can be a big problem.

What is Error, Fault, Failure.

▶ Error - mistake made, we are not perfect.

▶ Fault - the result of an error. Also commonly known as bugs or defects.
▶ Failure - the result of wrong behavior, deviation from the expected.

6
 10/07/2013
Why Testing is Necessary Testing Practice
Public

Errors and how they occur

▶ An error can be introduced at any stage within the system development lifecycle
(SDLC)
– User Requirements
– System Specification
– System Design
– System Construction

▶ An error can also be due to environmental reasons.

▶ Errors may occur during testing and subsequent de-bugging.

7
 10/07/2013
Why Testing is Necessary Testing Practice
Public

8
 10/07/2013
Why Testing is Necessary Testing Practice
Public

▶ Errors and How they occur

9
 10/07/2013
Why Testing is Necessary Testing Practice
Public

▶ The Cost of Errors

10
 10/07/2013
Why Testing is Necessary Testing Practice
Public

▶ The Cost of Errors

11
 10/07/2013
Why Testing is Necessary Testing Practice
Public

Why exhaustive testing is impossible

▶ It is impractical to try and test everything - time and resources
▶ It is not cost-effective.

Think of this …
▶ How many tests are required to ensure only valid values are entered into a 1
digit numeric field?

12
 10/07/2013
Exercise - Answer Testing Practice
Public

– Numeric (0-9)
– Upper case alpha characters (A-Z)
– Lower case alpha characters (a-z)
– Special characters (@ £ $)
– Blank.

13
 10/07/2013
Why is Testing Necessary Testing Practice
Public

▶ Quality attributes of a system

[Link]. Quality Attribute Meaning

1 Correctness The ability of the system to do the work for which it was
intended. Accountability for satisfying all requirements of the
system
2 Reliability The probability that software will not cause the failure of a
system for a specified time under specified conditions
3 Usability The ease of use and of training the end users of the system.
Sub qualities: learn ability, efficiency, helpfulness
4 Maintainability The ease with which a software system can accommodate
changes to its software

5 Re-usability The degree to which existing applications can be reused in new

applications
6 Testability The ease with which software can be made to demonstrate its
faults
7 Availability The measure of time that the system is up and running
correctly; the length of time between failures and the length of
time needed to resume operation after a failure.

14
 10/07/2013
Why Testing is Necessary Contd.. Testing Practice
Public

Testing and Risk

▶ Risk-based testing (RBT) is a type of software testing that prioritizes the tests
of features and functions based on the risk of their failure - a function of their
importance and likelihood or impact of failure

Types of Risks
▶ Business or Operational
• High use of a subsystem, function or feature
• Criticality of a subsystem, function or feature, including the cost of failure
▶ Technical
• Geographic distribution of development team
• Complexity of a subsystem or function
▶ External
• Sponsor or executive preference
• Regulatory requirements

15
 10/07/2013
Product Risk and Project Risk Testing Practice
Public

What is Product Risk?

▶ Product risk is the risk associated with the software or system, the possibility
that software or system may fail to satisfy end user/customers expectations is
known as product risk.

▶ There may be the possibility that the software or system does not have the
functionality specified by the customer or the stakeholders which leads to
unsatisfactory software.

▶ Unsatisfactory software has risk and has the possibility of failure which can
cause major functional damage.

▶ Low quality software can have many problems like functionality, reliability,
usability or performance.

16
 10/07/2013
Product Risk and Project Risk Testing Practice
Public

What is Project Risk?

▶ The risks associated with the testing activity which can endanger the test
project cycle is known as project risk.

▶ Some of the risks associated with project are:

Delay in the test build to test team.
Unavailability of test environment.
Delay in fixing test environment due to lack of system admin.
Delay in fixing defects by development team.
Organizational problems which can be like shortage of staff. Required skills etc
Major changes in the SRS which invalidates the test cases and requires changes
in the test case.

17
 10/07/2013
Why Testing is Necessary Contd.. Testing Practice
Public

▶ Testing and Quality

– Quality Control
• Is the activity performed to ensure that a product is 'fit for purpose'
– Quality control begins at the start of a project
– Quality control should be built into the whole SDLC.

– Quality Assurance
– Is checking that the process is being followed
– Is fixing the process to deliver a product that is ‘fit for purpose'.

18
 10/07/2013
Why is Testing Necessary Contd… Testing Practice
Public

Testing and Contractual Requirements

▶ Contracts can exist between:
– Different departments within the same company
– Software suppliers and 'the company'
– Software testers and 'the company'.

Testing and Legal and Regulatory Requirements

▶ Do your systems have to conform to legal requirements? Data Protection Act?
Disability Discrimination Act?
– If they should and they don't, your company could be prosecuted
▶ Is there a regulatory authority that oversees your industry? Financial Services
Authority? OFTEL?
– If there is, and your systems are non-compliant, then that authority can
stop your company trading.

19
 10/07/2013
Why is Testing Necessary Testing Practice
Public

▶ How much testing is enough?

– Determine what to test

• Requirements documentation provides the scope of your testing - exit
criteria

– Determine when to stop testing

• When the exit criteria has been met.

20
 10/07/2013
Why is Testing Necessary Testing Practice
Public

▶ Everything needs to be tested

21
 10/07/2013
The Psychology of Testing Testing Practice
Public

We will cover -
▶ Why do we test
▶ Characteristics of the developer
▶ Characteristics of the tester
▶ The developer/tester relationship
▶ Independence

22
 10/07/2013
The Psychology of Testing Testing Practice
Public

▶ Why do we test?

Primarily to find faults in the software, rather than demonstrating correctness.

So,

▶ This can be perceived as being a destructive process, not constructive

▶ How does this fit with the mindsets of the developer and tester?

23
 10/07/2013
The Psychology of Testing Testing Practice
Public

Developer Characteristics Tester Characteristics

▶ Specialized ▶ Trained/Experienced
▶ Trained ▶ Methodical
▶ Creative ▶ Persistent
▶ Sensitive to criticism. ▶ Happy when they are finding
▶ Why are developers sensitive? faults.
▶ Do testers contribute to the ▶ Good communicators
problem? ▶ Do developers contribute to their
problems?

24
 10/07/2013
The Psychology of Testing Testing Practice
Public

The developer/tester relationship

▶ Communication is, the key!!
▶ Communication must be constructive
not destructive
▶ Developers must inform testers of
any changes made to the system
▶ Testers must report problems to
developers clearly and concisely
▶ It is a two-way process!! It’s a Team-
work.

25
 10/07/2013
The Psychology of Testing Testing Practice
Public

▶ Independence
Independent testing is more effective. The author should not test their own
work
Assumptions made are carried into testing
People see what they want to see
There can be emotional attachment
There can be a vested interest not to find faults.

▶ Levels of Independence
Where test cases are....
– designed by the person who writes the software under test
– designed by another person
– designed by people from another department
– designed by people from another organization
– not chosen by a person.

26
 10/07/2013
Role of a Test leader Testing Practice
Public

Typical test leader task may include:

▶ Coordinating the development of the test strategy and plan with project
managers and others.

▶ Writing or reviewing test strategies produced for the project, and test policies
produced for the organization.

▶ Contributing the testing perspective to other project activities, such as

development delivery schedules.

▶ Planning the development of the required tests - which will include ensuring that
the development uses the correct understanding of the risks, selecting the
required test approaches (test levels, cycles, approach, objectives and incident
management planning), estimating the time and effort and converting to the
cost of testing and acquiring the right resources.

27
 10/07/2013
Role of a Test leader Testing Practice
Public

▶ Managing the specification, preparation, implementation and execution of tests,

including the monitoring and control of all the specification and execution.

▶ Taking the required action, including adapting the planning, based on test
results and progress (sometimes documented in status reports), and any action
necessary to compensate for problems or delays.

▶ Putting in place suitable metrics for measuring test progress and evaluating the
quality of the testing delivered and the product.

▶ Agreeing what should be automated, to what degree, and how, ensuring it is

implemented as planned.

▶ Scheduling all testing activity.

▶ At the end of the project, writing a test summary report based on the
information gathered during testing.

28
 10/07/2013
Role of a Tester Testing Practice
Public

The tasks typically undertaken by a tester may include:

▶ Reviewing and contributing to the development of test plans.

▶ Analyzing, reviewing and assessing user requirements, specifications and

models for testability.

▶ Creating test specifications from the test basis.

▶ Setting up the test environment (often coordinating with system administration

and network management).

▶ Preparing and acquiring/copying/creating test data.

29
 10/07/2013
Role of a Tester Testing Practice
Public

▶ Implementing tests on all test levels, executing and logging the tests,
evaluating the results and documenting the deviations from expected results as
defects.

▶ Using test administration or management and test monitoring tools as required.

▶ Automating tests that may be supported by a developer or a test automation

expert.

▶ Where required, running the tests and measuring the performance of

components and systems.

30
 10/07/2013
What makes a good test engineer? Testing Practice
Public

A good test engineer should have

▶ A ‘test to break’ attitude.

▶ An ability to take the point of view of the
customer.
▶ A strong desire for quality, and an attention
to detail.
▶ Tact and diplomacy are useful in maintaining
a cooperative relationship with developers.
▶ An ability to communicate with both technical
(developers) and non-technical (customers,
management) people is useful.
▶ Judgment skills are needed to assess high-risk areas of an application on which
to focus testing efforts when time is limited.

31
 10/07/2013
Questions Testing Practice
Public

32
Thank you

Atos, the Atos logo, Atos Consulting, Atos Worldline, Atos Sphere,
Atos Cloud and Atos WorldGrid
are registered trademarks of Atos SA. June 2011

© 2013 Atos. Confidential information owned by Atos, to be used by

the recipient only. This document, or any part of it, may not be
reproduced, copied, circulated and/or distributed nor quoted without
prior written approval from Atos.

10/07/2013