Testing Concepts and Test Automation
Software Quality Revolution
Software Testing is a process that consists of all test life cycle activities like static and dynamic testing concerned with planning, preparation and evaluation of software products to determine that the software products satisfy customers requirements and are fit for customers use. Software Testing is done to find software defects or failures in advance.
Lean principle
A systematic approach to identifying and eliminating waste through continuous improvement, flowing the product at the pull of the customer in pursuit of perfection.
The Quality Revolution
It illustrate the activity sequence:
Setting goals Assigning them to measurable milestones Assessing the progress against the milestones Take action to improve the process in the next cycle
The Objectives of Testing
It does work
It does not work
Reduce the risk of failures
Reduce the cost of testing
The Concept of Complete Testing
Complete or exhaustive testing means There are no undisclosed faults at the end of test phase
Complete testing is near impossible for most of the system The domain of possible inputs of a program is too large Valid inputs Invalid inputs The design issues may be too complex to completely test It may not be possible to create all possible execution environments of the system
Failure, Error, Fault and Defect
Failure A failure is said to occur whenever the external behavior of a system does not conform to that prescribed in the system specification Error An error is a state of the system. An error state could lead to a failure in the absence of any corrective action by the system Fault A fault is the adjudged cause of an error Defect It is synonymous of fault It a.k.a. bug
Testing Level
Unit testing Individual program units, such as procedure, methods in isolation Integration testing Modules are assembled to construct larger subsystem and tested System testing Includes wide spectrum of testing such as functionality, and load Acceptance testing Customers expectations from the system Two types of acceptance testing UAT BAT UAT: System satisfies the contractual acceptance criteria BAT: System will eventually pass the user acceptance test
V model
Test Automation
But, Is This Really Automation?
Analyze
product Design test Run test 1st time Evaluate results Report 1st bug Save code Save result Document test
Re-run
-----------
human human human human human human human human MACHINE
MACHINE
the test
result
(plus
Evaluate
human is needed if theres any mismatch) human
Maintain
result --
Automated Regression Pros and Cons
Advantages
Disadvantages
Breaks Tests Pays
Dominant automation paradigm Conceptually simple
easily (GUI based)
are expensive off late to failure because:
Straightforward
Same approach for all tests Fast implementation Variations are easy Repeatable tests
Prone
difficult financing,
architectural, and maintenance issues
Low
power even when successful (finds few defects)
GUI Automation is Expensive
Estimates run from 3-5 times the time to create and manually execute a test case.
You usually have to increase the testing staff in order to generate automated tests. Otherwise, how will you achieve the same breadth of testing? Your most technically skilled staff are tied up in automation Automation can delay testing, adding even more cost (albeit hidden cost.)
Maintaining GUI Automation
GUI test tools must be tuned to the product and the environment GUI changes break the tests
May need to wait for GUI stabilization Most early test failures are due to cosmetic changes False alarms are expensive We must investigate every reported anomaly We have to fix or throw away the test when we find a test or tool problem Maintainability is a key issue because our main payback is usually in the next release, not this one.
Common Mistakes about Test Automation
Dont expect to be more productive over the short term.
The reality is that most of the benefits from automation dont happen until the second release. It takes 3 to 10+ times the effort to create an automated test than to just manually do the test. Apparent productivity drops at least 66% and possibly over 90%. Additional effort is required to create and administer automated test tools.
Think About
Automation is software development. Regression automation is expensive and can be inefficient. Automation need not be regression--you can run new tests instead of old ones. Maintainability is essential. Design to your requirements. Set management expectations with care.
Issues Faced in a Automated Test
What is being tested? How is the test set up? Where are the inputs coming from?
What is being checked?
Where are the expected results? How do you know pass or fail?
TestComplete
TestComplete Properties
TestComplete Explorer
TestComplete Run
TestComplete Report - Details
TestComplete Report - Details
Thank You
Ganesh B.
