[AnDevCon 2016] Mutation Testing for Android

Android Mutation
Testing
Hazem Saleh
Mobile Architect @Nickelodeon

About
 More than twelve years of experience in Java and mobile solutions.
 Apache Open Source Committer.
 Author of four technical books.
 DeveloperWorks Contributing Author and DZone MVB.
 Technical Speaker (JavaOne, ApacheCon, Geecon, JavaLand …etc).
 JSR Experts Group Member.
 An X-IBMer and Currently a Mobile Software Architect in Nickelodeon.

Agenda
 Developers Life without Unit Testing.
 Unit Testing 101.
 Unit Testing in Android.
 Code Coverage and Jacoco.
 Code Coverage Demo.
 Current challenges of traditional code
coverage.
 Mutation Testing 101.
 What and Why PIT?
 PIT Android Integration.
 Mutation Testing Demo.
 Quiz.
 Q & A.

Developers Life without Unit Testing
Complex integration between the system components.

Unmanaged number of new/regression defects especially
when the system complexity increases.

Low application quality.
Longer testing cycle.
test
fix
test
fix
test
fix
testfix
test
fix
test
fix
test
fix

Unit Testing 101
A unit testing is a piece of code (usually a method) that
invokes another piece of code and later checks the
correctness of some assumptions.
Unit testing helps in detecting BUGGY components in the
early stages of the project.
A test suite is a set of test cases, and a test case is a set of
tests which verifies the system components.

AutomatedRepeatable
Easy to run.
Fast
Easy to
understand
Incremental
Unit Testing 101

Unit Testing in ANDROID
 Generally, Unit tests in Android are implemented using:
– JUnit.
– Mockito (for mocking testable class dependencies).
 However, there are some challenges:
– Mocking Static methods.
– Mocking final methods or classes.
– Mocking private methods.
 Thanks to PowerMock, the previous challenges are met.

Unit Testing in ANDROID
Other challenge:
–Unit testing classes with huge dependency on
Android SDK is hard (such as activities /
fragments).
Thanks to Robolectric:
–This challenge is also met:
http://robolectric.org/
–Robolectric mocks all Android classes by Shadow classes.
–Robolectric adds a great power to Android apps unit testing.

Code Coverage and Jacoco
 Code Coverage represents the amount of source code
which will be executed when test cases run.
 In order to measure the amount of tested source code,
there are popular coverage criteria:
–Statement coverage
–Function coverage
–Branch coverage, which represents the amount of branches of
each control structure (such as if) covered.

 JaCoCo is a free code coverage library for Java.
 It includes the popular code coverage metrics: statement
coverage, branch coverage, and method coverage.

 Fortunately, JaCoCo is fully Integrated with Gradle.
 JaCoCo works fine with Android projects.
 In order to configure JaCoCo with Gradle:
–Enable code coverage for the build type(s) that you will be testing
with.
–Import JaCoCo in your gradle build file.
–Configure JaCoCo with your Java sources.

Demo
Sample URL:
https://github.com/hazems/Dagger-Sample

Challenges
 Traditional code
coverage only measure
the amount of executed
code.
coverage does not detect
code faults.
coverage does not show
how strong the code is.

Mutation testing 101
 Mutation testing is about seeding app source code with faults
(mutations).
 After seeding, unit tests then run.
 If a unit test fails, then a mutation is killed (and means that your
unit test is strong enough to face this mutation).
 If a unit test succeeds, then a mutation is lived (and means that
your unit test needs modification to be stronger).
 In mutation testing, the quality of the test can be measured by
the percentage of the killed mutations.

Mutation testing 101
 Mutations can be for example:
–Negate Conditionals Mutator (<= TO > and < TO >=)
–Increment Mutator (i++ TO i--)
–Invert Negatives Mutator (-I to I)
–Math Mutator (+ to – and – to + and * to / and * to /)
–Return Values Mutator (boolean  true to false)
–Void Method Call Mutator (removing call).

What and Why PIT?
 One of the mutation testing tools for Java.
 It has advantages
–Fast to execute (unlike most of the Java mutation testing tools).
–Compatible with:
•Ant
•Maven
•Gradle
–Active.
–Provides easy to read test reports.

PIT Android Integration
 Gradle Plugin for PIT is available:
http://gradle-pitest-plugin.solidsoft.info/
https://github.com/szpak/gradle-pitest-plugin
 Gradle Plugin works perfect with Java projects.
 However, for Android Gradle projects, you need to use
this fork:
https://github.com/koral--/gradle-pitest-plugin

 For using PIT in your Android apps, add PIT plugin to
your top-level build.gradle file as follows
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'pl.droidsonroids.gradle:gradle-
pitest-plugin:0.0.3'
}
}
 Apply plugin: 'com.android.application’
apply plugin: 'pl.droidsonroids.pitest'

 Configure PIT plugin basically as follows
pitest {
targetClasses = ['com.test.xyz.daggersample.presenter.*']
/* Specify target classes to be mutated */
excludedClasses = ['**Factory*']
/* Exclude the unnecessary classes */
pitestVersion = "1.1.0"
threads = 4
/* Specify number of threads to execute mutation testing */
outputFormats = ['XML', 'HTML']
/* Specify output format */
}

 Check the mutation results by executing the following
command:
>./gradlew clean pitest

Q & A
Twitter: @hazems
Email: hazems@apache.org

[AnDevCon 2016] Mutation Testing for Android

More Related Content

What's hot

Similar to [AnDevCon 2016] Mutation Testing for Android

More from Hazem Saleh

Recently uploaded

[AnDevCon 2016] Mutation Testing for Android

Editor's Notes