Portfolio report of CSE 565
Naga MAnoj Makkena
SCAI
Arizona State University
Tempe, AZ
[email protected] Abstract—In CSE 565- Software verification validation and impacts the vocabulary, that is changes from test based to
Testing course I have learnt about various tools used for testing behavior based which is an attribute of the design philosophy.
the software quality and verifying the results. There are in total 5 Common comparibilities among them are: In JBehave and
assignments related to the various stages of the software quality
testing. I have learnt about unit test frame working, pairwise PyUnit, the common capability is that we can define group
testing, coverage report, mobile regression testing, AI and Ma- fixtures. The difference between the JBehave and PyUnit is
chine learning algorithms and Software reliability estimation and that in JBehave unittest does not contain generator methods in
prediction tool(SREPT) module ‘unittest.TestCase’ where as in PyUnit it has generator
Index Terms—unit test,pairwisr testing,regression testing, cov- methods in module ‘unittest.TestCase’
erage report, SREPT
C. Knowledge acquired
I. U NIT TEST FRAMEWORK Learnt about unit test frame working. Identified and un-
A. Introduction derstood the working of the two unit test frame working
Unit testing frameworks are software tools used for writing models JBehave and PyUnit. Common comparable functions
and running unit tests. They are also responsible for stating the of JBehave and PyUnit. Unit test frame working on Binary
foundation for building the unit tests. The unit tests execution Search algorithm.
and report of the results is the functionality of the unit testing II. PAIRWISE TESTING
framework. The unit test is specific to testing the particular A. Introduction
function in the production code and in general a good unit
Pairwise testing, sometimes called all-pairs testing, is a
test will be independent of the other functionalities in the
combinatorial approach to software testing. It is a strategy
production code in [1].
for trying every conceivable permutation of the parameters in
B. Summary question in [2]. Say we have a piece of software that has to be
Unit testing frameworks help developers to understand the tested, and that programme has 10 input fields, each with 10
API and other related functionalities of the unit test. Unit potential configurations. In that case, there are 10,000 possible
test framework helps the developer with the class libraries inputs. Even if we wanted to try every conceivable permuta-
that provide base cases or interfaces to inherit, attributes to tion, it would be difficult to do a thorough test here. Designed
identify the different parts of the code to notify which methods a software application for ordering customized phone from a
are tests and assertion classes with assertion methods that we manufacture.
can utilize to verify our code. Unit testing has many benefits B. Summary
that contribute to the quality of the software application. As shown in the below figures, fig:1, fig:2 the input and
Before deploying the code the unit testing ensures that all number of different kinds of items an input takes the pair-
the vulnerabilities are mitigated to meet the quality standards. wise testing tool generates the possible test cases taking into
This ensures that the software has paramount quality. In the account all the input values and its kind.
software application development process the unit testing saves
a lot of time and energy which helps the developers to write the
code quickly and effectively. The two types of unit test frame
working are: PyUnit : It is a standard unit test framework for
Python. In TestComplete, you can run Python unit tests and
Selenium tests created by using the PyUnit testing framework
as part of your TestComplete project. In the PyUnit Test editor,
you configure the tests to be run. JBehave: For behavior-driven
development (BDD), JBehave is used. The evolution of test-
driven development (TDD) is BBD which is used to make Fig. 1. Pairwise testing tool
these practices accessible to beginners and experts alike. It
Fig. 2. Testcases
Fig. 3. Testcases
C. Knowledge acquired
Understood the working of the pairwise testing tool and its unused import, the improper naming convention as per the
application. above screenshot reported by pylint are Invalid-name, Unused-
import, Missing-function-docstring.
III. C OVERAGE R EPORT
This tool basically helps in finding the code smells such
A. Introduction as checking the developer coding standards with industry
There are various types of code coverage tools available in level standards, refactoring the variables, simplifying complex
the market, one among them is the coverage.py tool which code into simpler code, and identifying the warnings and
I used for part-1 in identifying the code coverage for the unnecessary multiple lines of code.
python source code used in the Homework1. This tool helps in
C. Knowledge acquired
monitoring and determining which part of the code had been
executed by test cases created by testers and which part of Understood the importance of coverage report and its ap-
the code has not been tested. This tool also helps in providing plications, Learnt the characteristics of pylint tool and its
information such as the line numbers of the uncovered code by applications. Demonstrated the coverage report for binary
the test cases. Below are the types of coverage it gives us: Line search algorim and able to identify the test cases that covers
(Statement) coverage, Function coverage, Branch coverage, all the possible test cases.
Conditions coverage etc in [3].
IV. M OBILE R EGRESSION TESTING
B. Summary A. Introduction
Pylint tool is used for monitoring the percentage of the code Mobile regression testing ensures that app changes don’t
covered by the test cases created for the application. This tool break its functionality. It’s done after every app update or
provides additional information such as the lines which are version. Changes include bug fixes and new features. In either
uncovered by the test cases so that it can be easy for the situation, mobile regression tests are needed to ensure system
testers to create a new test case for covering the missing code functionality. Mobile regression testing seeks new defects in
coverage and boost the percentage of the code coverage. The working code, whereas new feature testing verifies new code
tool also provides code coverage in different formats such as in [4], [5].
console-based output, HTML and XML, etc.
The coverage report is shown in the below figure, Fig.3 B. Summary
Pylint is one of the available source code analysis tools for The challenges of mobile regression testing are
python in the market. It helps in identifying the industry-level Test cases need to be created for testing the application and
coding standard, helps in refactoring such as variables, method they should be effective to identify the failed conditions in the
names, class names, detecting duplicate or unused code, application if any. It can consume more time if the features in
Warnings, Errors, and Bug detection. It is fully customizable the application increases since every feature should be tested.
through a file called .pylintrc which helps in choosing the Need additional testing because even for a small change every
relevant issues in our code. It can also be used for generating feature should be tested again. Because of continuous testing
the UML diagrams using pyreverse. Each problem reported a dedicated tester or automated testing should be considered
by this tool will be in the format of ID and Message object for effectiveness. Training of testers should be crucial because
which are described below C - Convention, R – Refactor, W they should be well equipped with the software application.
– Warning, E – Error, F – Fatal. Perfecto can be used as a mobile application regressive
The data flow anomalies typically caught by a static anal- testing tool. It provides advanced automated abilities for unit,
ysis tool are of type (Convention ) C and Warning (W) functional and performance test cases. Parallel testing can
be used to accerlate the testing procedure. Advanced test approach works, a software failure dataset is used to test the
reporting, easily finds the bugs and reports the issue. CI proposed neuro-fuzzy method in [7].
integrations enables that the escaped defects can be avoided.
B. Summary
It performs tests on real and virtual platforms. It integrates
with entire toolchain, including appium,selenium and codeless The SREPT follows two approaches for software reliability
solutions. prediction, they are black-box based and architecture based.
AI and Machine learning techniques streamlines continuous The Inputs and outputs of black-box based approach: Software
testing by eliminating human/manual interaction, which might Product/Process metrics is taken as inputs and Estimate of
induce judgement errors. AI helps QA teams launch unat- number of faults is the output. Test Coverage is taken as input
tended test cycles, which helps uncover problems and triggers and predicted failure intensity,number of faults remaining and
remedial steps based on historical datasets and occurrences. reliability after use are the outputs. Sequence of Interfailure
For example: Testim is an AI-based software testing solution times is taken as input and predicted failure intensity,number
that allows start-to-finish testing. Testim’s cyclic model im- of faults remaining, coverage and reliability after use are the
proves testing results.AI technologies reduce or eliminate bar- outputs. Interfailure times and Release criteria is taken is
riers that slow down testing. Using Testim’s record functions, input and predicted release time and predicted coverage are
non-coders can quickly create start-to-finish tests. Architects the outputs. Failure intensity debugging policies and predicted
can help code improve and mix. Testim prevents tests from failure intensity,number of faults remaining and reliability after
failing. And it’s all easy, requiring no sophisticated or short- use are the outputs. The Inputs of Architecture based approach:
sighted questions. Testim builds and runs low-maintenance Architecture is taken as the input and Predicted Reliability
tests. Testim helps control group and test modifications, and Performance are taken as the output. Failure behavior
demonstrate application quality, and steer change. Time-saving of components and Predicted Reliability and Performance are
test-writer. Fast and efficient test execution. Saved time speeds taken as the output.
up development. AI has enhanced code coverage. AI’s tests The following model to predict the reliability of software
increase project quality and application quality. AI creates is based on self-organized maps and fuzzy time series. In the
enhanced test cases, improving application quality. proposed method, predicting the reliability of software in a
software system is modelled as a fuzzy time series forecasting
C. Knowledge Acquired problem. This is because software reliability is hard to predict.
Understood the importance of mobile regression tesing. AI There is only one dependent variable in the failure dataset,
and machine learning testing is alo studied in the assignment. and there are no independent variables. As part of fuzzy time
Understood the working of perfecto tool. series forecasting, a Software failure dataset that shows the
history of how long it took for a software system to break is
V. S OFTWARE R ELIABILITY E STIMATION AND used to predict how long it will take for a software system
P REDICTION T OOL (SREPT) to break. A Software failure dataset is split up into a number
A. Introduction of intervals. A self-organized-map (SOM) with two layers of
The regression tree model is a goal-oriented statistical an unsupervised neural network was used to divide the dataset
method that uses the static product/process metrics to try to into intervals. Also, fuzzy sets are used to show these intervals
predict the number of bugs in a software module. Data sets by defining linguistic terms.
from similar software development projects from the past are C. Knowledge acquired
used to build the tree, which is then used to predict how the
current project will go. So, software product/process metrics Understood the importance of the Software reliability esti-
can be used to predict early on how many bugs are in a mation and prediction tool and also machine learning tool for
software product’s modules. This can help point testing and reliability prediction. Comparing between the above two kinds
validation efforts in the right direction in [6]. of tools.
Machine learning tool is used for reliability prediction. R EFERENCES
Software engineers need to be able to predict reliability
[1] https://fortegrp.com/the-importance-of-unit-testing.
because there is more demand than ever for high-quality [2] https://www.pairwise.org/tools.html
software. Even though different models for predicting software [3] https://www.geeksforgeeks.org/pylint-module-in-python/
reliability have been made so far, there is always a need [4] https://katalon.com/resources-center/blog/regression-testing
[5] https://www.testim.io/
for a more accurate model in today’s competitive software [6] B.W. Boehm, A spiral model of software development and enhancement,
industry. This Machine Learning algorithm proposes a neuro- IEEE Comput. 21 (5) (1988) 61–72
fuzzy hybridized method for predicting the reliability of a [7] [Amin et al. 2013] Amin, A., Grunske, L., Colman, A.: “An approach to
software reliability prediction based on time series modeling”; Journal
software system by combining self-organized map (SOM) of Systems and Software, 86, 7 (2013), 1923- 1932
and fuzzy time series (FTS) forecasting. In this algorithm,
the well-known supervised clustering algorithm SOM and
FTS forecasting are combined to make a hybrid model for
predicting software reliability. To prove that the proposed
