Integrating PHP Projects
with Jenkins
Sebastian Bergmann
OSCON July 16th 2012
Sebastian Bergmann
Has instrumentally
contributed to tranforming
PHP into a reliable platform for
large-scale, critical projects.
Enterprises and PHP
developers around the world
benefit from the tools that he
has developed and the
experience he shares.
sharing experience
sharing experience
Continuous Integration
Software development practice where
members of a team integrate their work
frequently, usually each person integrates at
least daily leading to multiple integrations
per day. Each integration is verified by an
automated build (including test) to detect
integration errors as quickly as possible.
Martin Fowler
sharing experience
e
d
o
C
Test
Ship
[Link]
sharing experience
Release Early, Release Often
Our highest priority is to satisfy the customer
through early and continuous delivery of
valuable software.
Agile Manifesto
sharing experience
Release Early, Release Often
Software delivers no revenue until it is in the
hands of its users. [] For large companies
every week of delay between having an idea
and releasing the code that implements it can
represent millions of dollars in opportunity
costs []
Jez Humble and David Farley
sharing experience
a
e
d
I
C od e
Value
Release Early, Release Often
We need to get rid off the term release cycle.
Simon Stewart
sharing experience
Release Early, Release Often
[Link]
sharing experience
Quantum of Deployment
The smallest number of steps, with the smallest
number of people and the smallest amount of
ceremony required to get new code running on
your servers.
[Link]
sharing experience
Continuous Delivery
[Link]
sharing experience
(Elements of a) Path to Continuous Delivery
Code
Automated Tests
Latent Code Patterns (Feature Flags, ...)
Software Configuration Management
Feature Branches
Automation
Build
Deployment
Continuous Integration
sharing experience
Continuous Integration
Reduce risks
Reduce repetitive processes
Prevent low internal software quality
Generate deployable software
Enable better project visibility
Establish greater project confidence
sharing experience
This slide contains material by Paul M. Duvall
Jenkins
Continuous Integration Server
Open Source
Extendable
* [Link]
sharing experience
Installing Jenkins
mkdir /usr/local/jenkins
cd /usr/local/jenkins
wget [Link]
export JENKINS_HOME=/usr/local/jenkins
java -jar [Link]
sharing experience
Installing Plugins for Jenkins (using Jenkins CLI)
wget [Link]
java -jar [Link] -s [Link] install-plugin \
checkstyle cloverphp dry htmlpublisher jdepend plot pmd violations \
xunit
java -jar [Link] -s [Link] safe-restart
sharing experience
Build
A build acts as the process for putting source
code together and verifying that the software
works as a cohesive unit.
Paul M. Duvall
sharing experience
Build Automation
Build automation is the act of scripting or
automating a wide variety of tasks that software
developers do in their day-to-day activities
including compiling, packaging, running tests,
deployment to production
[Link]
sharing experience
What's in a build?
Code Generation / Code Transformation
(Compilation)
Automated Tests
Code Analysis
Documentation Generation
Packaging / Deployment
...
sharing experience
Code Generation / Code Transformation
Autoloader
Object-Relational Mapper
PHPAB*
Model PHP and SQL code
Framework
Scaffolding
* [Link]
sharing experience
Apache Ant [Link] Script
<project name="my-project" default="phpab">
<target name="phpab" description="Generate autoloader script">
<exec executable="phpab">
<arg value="--output" />
<arg path="${basedir}/src/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
</project>
sharing experience
Apache Ant [Link] Script
<project name="my-project" default="build">
<target name=build" depends="phpab"/>
<target name="phpab" description="Generate autoloader script">
<exec executable="phpab">
<arg value="--output" />
<arg path="${basedir}/src/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
</project>
sharing experience
Apache Ant [Link] Script
<project name="my-project" default="build">
<target name=build" depends="prepare" />
<target name=clean">
<!-- ... -->
</target>
<target name=prepare" depends="clean,phpab">
<!-- ... -->
</target>
<target name="phpab" description="Generate autoloader script">
<exec executable="phpab">
<arg value="--output" />
<arg path="${basedir}/src/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
</project>
sharing experience
Compilation
Compilation of PHP code to a native binary
HipHop
Syntax Check
php -l*
* [Link]
sharing experience
Syntax Check
<project name="my-project" default="build">
<target name=build" depends="prepare,lint" />
<!-- ... -->
<target name="lint">
<apply executable="php" failonerror="true">
<arg value="-l" />
<fileset dir="${basedir}/src">
<include name="**/*.php" />
</fileset>
<fileset dir="${basedir}/tests">
<include name="**/*.php" />
</fileset>
</apply>
</target>
</project>
sharing experience
Syntax Check
<project name="my-project" default="build">
<target name=build" depends="prepare,lint" />
<!-- ... -->
<target name="lint">
<apply executable="php" failonerror="true">
<arg value="-l" />
<fileset dir="${basedir}/src">
<include name="**/*.php" />
<modified />
</fileset>
<fileset dir="${basedir}/tests">
<include name="**/*.php" />
<modified />
</fileset>
</apply>
</target>
</project>
sharing experience
PHPUnit*
De-Facto standard for unit testing in PHP
Logfiles
Test Results in JUnit XML
Code Coverage in Clover XML
* [Link]
sharing experience
PHPUnit
<project name="my-project" default="build">
<target name=build" depends="prepare,lint,phpunit" />
<target name=clean">
<delete dir="${basedir}/build/coverage"/>
<delete dir="${basedir}/build/logs"/>
</target>
<target name=prepare" depends="clean,phpab">
<mkdir dir="${basedir}/build/coverage"/>
<mkdir dir="${basedir}/build/logs"/>
</target>
<target name="phpunit" description="Run unit tests with PHPUnit">
<exec executable="phpunit" failonerror="true"/>
</target>
</project>
sharing experience
PHPUnit
<phpunit bootstrap="src/[Link]">
<testsuite name="my-project">
<directory suffix="[Link]">tests</directory>
</testsuite>
<logging>
<log type="coverage-html" target="build/coverage" />
<log type="coverage-clover" target="build/logs/[Link]" />
<log type="junit" target="build/logs/[Link]" />
</logging>
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">src</directory>
<exclude>
<file>src/[Link]</file>
</exclude>
</whitelist>
</filter>
</phpunit>
sharing experience
Jenkins Plugin: xUnit*
This plugin makes it possible to publish the
test results of an execution of a testing tool in
Jenkins
JUnit XML is not really standardized
PHPUnit uses nested <testsuite> elements
* [Link]
sharing experience
Jenkins Plugin: Clover PHP*
This plugin allows you to capture code
coverage reports from PHPUnit
Only exists because the Clover plugin has
some quirks as it expects the real Clover
tool (for Java) to be used
* [Link]
sharing experience
PHPLOC*
A tool for quickly measuring the size of a PHP
project
Logfile: CSV
* [Link]
sharing experience
PHPLOC
Directories:
Files:
Lines of Code (LOC):
Cyclomatic Complexity / Lines of Code:
Comment Lines of Code (CLOC):
Non-Comment Lines of Code (NCLOC):
Namespaces:
Interfaces:
Traits:
Classes:
Abstract:
Concrete:
Average Class Length (NCLOC):
Methods:
Scope:
Non-Static:
Static:
Visibility:
Public:
Non-Public:
Average Method Length (NCLOC):
Cyclomatic Complexity / Number of Methods:
11
22
601
0.04
7
594
11
1
0
20
1 (5.00%)
19 (95.00%)
22
38
38 (100.00%)
0 (0.00%)
26 (68.42%)
12 (31.58%)
11
1.58
Anonymous Functions:
Functions:
1
0
Constants:
Global constants:
Class constants:
0
0
0
sharing experience
PHPLOC
<target name="phploc">
<exec executable="phploc">
<arg value="--log-csv" />
<arg value="${basedir}/build/logs/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
sharing experience
Jenkins Plugin: Plot*
This plugin provides generic plotting (or
graphing) capabilities in Jenkins
Used to plot the metrics collected by PHPLOC
over time
* [Link]
sharing experience
PHP_CodeSniffer*
Tokenises PHP, JavaScript and CSS files and
detects violations of a defined set of coding
standards
Logfile: Checkstyle XML
* [Link]
sharing experience
PHP_CodeSniffer
PHP CODE SNIFFER VIOLATION SOURCE SUMMARY
-------------------------------------------------------------------------------STANDARD CATEGORY
SNIFF
COUNT
-------------------------------------------------------------------------------CodeRevi Functions
Global function found
2297
CodeRevi PHP
Global keyword not allowed
938
Generic
PHP
No silenced errors discouraged
523
CodeRevi PHP
Forbidden functions found
77
Generic
Code analysis
For loop with test function call not allowe 53
Generic
Code analysis
Empty statement not allowed warning
34
Generic
PHP
Deprecated functions found
28
Generic
Code analysis
Useless overriding method found
4
Generic
Classes
Duplicate class name found
2
Generic
Code analysis
Unconditional if statement found
1
-------------------------------------------------------------------------------A TOTAL OF 3957 SNIFF VIOLATION(S) WERE FOUND IN 10 SOURCE(S)
-------------------------------------------------------------------------------Time: 08:02, Memory: 1750.25Mb
sharing experience
PHP_CodeSniffer
FILE: /tmp/wordpress/wp-includes/[Link]
-------------------------------------------------------------------------------FOUND 7 ERROR(S) AND 16 WARNING(S) AFFECTING 23 LINE(S)
-------------------------------------------------------------------------------18 | WARNING | Consider refactoring "_wp_admin_bar_init" to avoid global
|
| functions.
19 | ERROR
| Use of the "global" keyword is forbidden
52 | WARNING | Consider refactoring "wp_admin_bar_render" to avoid global
|
| functions.
53 | ERROR
| Use of the "global" keyword is forbidden
78 | WARNING | Consider refactoring "wp_admin_bar_my_account_menu" to avoid
|
| global functions.
79 | ERROR
| Use of the "global" keyword is forbidden
101 | WARNING | Consider refactoring "wp_admin_bar_dashboard_view_site_menu"
|
| to avoid global functions.
119 | WARNING | Consider refactoring "wp_admin_bar_my_sites_menu" to avoid
|
| global functions.
120 | ERROR
| Use of the "global" keyword is forbidden
154 | WARNING | Consider refactoring "wp_admin_bar_shortlink_menu" to avoid
|
| global functions.
176 | WARNING | Consider refactoring "wp_admin_bar_edit_menu" to avoid global
|
| functions.
.
.
.
sharing experience
PHP_CodeSniffer
<target name="phpcs">
<exec executable="phpcs" output="/dev/null">
<arg value="--report=checkstyle" />
<arg value="--report-file=${basedir}/build/logs/[Link]" />
<arg value="--standard=${basedir}/build/[Link]" />
<arg value="--ignore=[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
sharing experience
PHP_CodeSniffer
<ruleset name="name-of-your-coding-standard">
<description>Description of your coding standard</description>
<rule ref="[Link]"/>
<!-- ... -->
</ruleset>
sharing experience
Jenkins Plugin: Checkstyle*
This plugin generates the trend report for
Checkstyle, an open source static code analysis
program
Used to report PHP_CodeSniffer results
* [Link]
sharing experience
PHP Copy/Paste Detector (PHPCPD)*
Copy/Paste Detector (CPD) for PHP code
Logfile: PMD-CPD XML
* [Link]
sharing experience
PHP Copy/Paste Detector (PHPCPD)
phpcpd 1.3.2 by Sebastian Bergmann.
Found 26 exact clones with 459 duplicated lines in 4 files:
- wp-content/plugins/akismet/[Link]-500
wp-content/plugins/akismet/[Link]-549
- wp-content/plugins/akismet/[Link]-248
wp-content/plugins/akismet/[Link]-315
.
.
- wp-includes/[Link]-172
wp-includes/[Link]-232
- wp-includes/[Link]-187
wp-includes/[Link]-345
- wp-includes/[Link]-331
wp-includes/[Link]-398
0.27% duplicated lines out of 171170 total lines of code.
Time: 5 seconds, Memory: 73.25Mb
sharing experience
PHP Copy/Paste Detector (PHPCPD)
<target name="phpcpd">
<exec executable="phpcpd">
<arg value="--log-pmd" />
<arg value="${basedir}/build/logs/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
sharing experience
Jenkins Plugin: DRY*
This plugin generates the trend report for
duplicate code checkers like CPD or Simian
Used to report PHPCPD results
* [Link]
sharing experience
PHP_Depend*
PHP port of JDepend
Logfile: JDepend XML
Also: Software visualizations
* [Link]
sharing experience
PHP_Depend
pdepend --overview-pyramid=overview_pyramid.svg src
PHP_Depend 1.0.7 by Manuel Pichler
Parsing source files:
......................
22
Executing Coupling-Analyzer:
.....
108
Executing CyclomaticComplexity-Analyzer:
.....
105
Executing Inheritance-Analyzer:
.
36
Executing NodeCount-Analyzer:
...
70
Executing NodeLoc-Analyzer:
....
91
Generating pdepend log files, this may take a moment.
Time: 00:01; Memory: 18.00Mb
sharing experience
PHP_Depend
ANDC
AHH
NOP
NOC
NOM
LOC
CYCLO
CALLS
FANOUT
Average Number of Derived Classes
Average Hierarchy Height
Number of Packages
Number of Classes
Number of Methods
Lines of Code (non-comment, non-whitespace)
Cyclomatic Complexity
Number of Operation Calls
Number of Called Classes
sharing experience
PHP_Depend
pdepend --jdepend-chart=abstraction_instability.svg src
PHP_Depend 1.0.7 by Manuel Pichler
Parsing source files:
......................
22
Executing Dependency-Analyzer:
...
70
Generating pdepend log files, this may take a moment.
Time: 00:00; Memory: 9.75Mb
sharing experience
PHP_Depend
sharing experience
PHP_Depend
pdepend --jdepend-xml=[Link] src
PHP_Depend 1.0.7 by Manuel Pichler
Parsing source files:
......................
22
Executing Dependency-Analyzer:
...
70
Generating pdepend log files, this may take a moment.
Time: 00:00; Memory: 9.75Mb
[Link] [Link] -o [Link]
sharing experience
PHP_Depend
sharing experience
PHP_Depend
pdepend --summary-xml=[Link] src
PHP_Depend 1.0.7 by Manuel Pichler
Parsing source files:
......................
Executing CyclomaticComplexity-Analyzer:
.....
22
105
Executing ClassLevel-Analyzer:
....
85
Executing CodeRank-Analyzer:
.
32
Executing Cohesion-Analyzer:
.......
154
Executing Coupling-Analyzer:
.....
108
Executing Hierarchy-Analyzer:
....
87
Executing Inheritance-Analyzer:
.
36
Executing NPathComplexity-Analyzer:
.....
105
Executing NodeCount-Analyzer:
...
70
Executing NodeLoc-Analyzer:
....
91
Generating pdepend log files, this may take a moment.
Time: 00:00; Memory: 19.00Mb
sharing experience
Cyclomatic Complexity
Number of possible decision paths in a program or
program unit
Thomas J. McCabe, A Complexity Measure,
IEEE Transactions on Software Engineering 2, No. 4
(IEEE Computer Society Press, Los Alamitos, CA, USA,
1976).
sharing experience
Cyclomatic Complexity
[Link] [Link] --metric0 ccn
=================================================================
Name
| Value
=================================================================
BankAccountMapper::findById()
| 4.0000
Router::route()
| 4.0000
BankAccountController::execute()
| 3.0000
Request::__call()
| 3.0000
ControllerFactory::getController()
| 3.0000
BankAccount::setBalance()
| 2.0000
MapperFactory::getMapper()
| 2.0000
BankAccountMapper::getAllIds()
| 2.0000
BankAccountMapper::insert()
| 2.0000
BankAccountMapper::delete()
| 2.0000
BankAccountMapper::update()
| 2.0000
BankAccountListView::render()
| 2.0000
HashMap::get()
| 2.0000
BankAccount::depositMoney()
| 1.0000
.
.
.
sharing experience
NPath Complexity
Number of acyclic execution paths in a program or
program unit
Brian A. Nejmeh, NPATH: A Measure of Execution
Path Complexity and its Applications,
Communications of the ACM 31, Issue 2
(February 1988): 188200. ISSN 0001-0782.
sharing experience
NPath Complexity
[Link] [Link] --metric0 npath
=================================================================
Name
| Value
=================================================================
Router::route()
| 8.0000
Request::__call()
| 6.0000
BankAccountMapper::findById()
| 6.0000
BankAccountController::execute()
| 4.0000
ControllerFactory::getController()
| 3.0000
BankAccountMapper::getAllIds()
| 2.0000
BankAccountListView::render()
| 2.0000
BankAccount::setBalance()
| 2.0000
MapperFactory::getMapper()
| 2.0000
BankAccountMapper::update()
| 2.0000
BankAccountMapper::delete()
| 2.0000
BankAccountMapper::insert()
| 2.0000
HashMap::get()
| 2.0000
BankAccount::withdrawMoney()
| 1.0000
.
.
.
sharing experience
PHP_Depend
<target name="pdepend">
<exec executable="pdepend">
<arg value="--jdepend-xml=${basedir}/build/logs/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
sharing experience
Jenkins Plugin: JDepend*
The JDepend Plugin is a plugin to generate
JDepend reports for builds
Used to report PHP_Depend results
* [Link]
sharing experience
PHP Mess Detector (PHPMD)*
[PHPMD] is a spin-off project of PHP_Depend
and aims to be a PHP equivalent of the well
known Java tool PMD.
PHPMD can be seen as an user friendly and easy
way to configure frontend for the raw metrics
measured by PHP_Depend.
Logfile: PMD XML
* [Link]
sharing experience
PHP Mess Detector (PHPMD)
<target name="phpmd">
<exec executable="phpmd">
<arg path="${basedir}/src" />
<arg value="xml" />
<arg value="${basedir}/build/[Link]" />
<arg value="--reportfile" />
<arg value="${basedir}/build/logs/[Link]" />
</exec>
</target>
sharing experience
PHP Mess Detector (PHPMD)
<ruleset name="name-of-your-coding-standard">
<description>Description of your coding standard</description>
<rule ref="rulesets/[Link]/CyclomaticComplexity" />
<!-- ... -->
</ruleset>
sharing experience
Jenkins Plugin: PMD*
This plugin generates the trend report for
PMD, an open source static code analysis
program
Used to report PHPMD results
* [Link]
sharing experience
Jenkins Plugin: Violations*
This plug-in generates reports static code
violation detectors such as checkstyle, pmd,
cpd, findbugs, codenarc, fxcop, stylecop and
simian
Used to report the results of
PHP_CodeSniffer
PHP Copy/Paste Detector (PHPCPD)
PHP Mess Detector (PHPMD)
* [Link]
sharing experience
PHP_CodeBrowser*
Generates a browsable representation of PHP
code where sections with violations found by
quality assurance tools such as PHP_CodeSniffer
or PHPMD are highlighted
* [Link]
sharing experience
PHP_CodeBrowser
<target name="phpcb">
<exec executable="phpcb">
<arg value="--log" />
<arg path="${basedir}/build/logs" />
<arg value="--source" />
<arg path="${basedir}/src" />
<arg value="--output" />
<arg path="${basedir}/build/code-browser" />
</exec>
</target>
sharing experience
Jenkins Plugin: HTML Publisher*
This plugin publishes HTML reports
API Documentation (from phpdox, for instance)
Output from PHP_CodeBrowser
* [Link]
sharing experience
Jenkins PHP*
Template for Ant build scripts for PHP projects
Template for Jenkins jobs for PHP projects
* [Link]
sharing experience
Deployment
Are the required features implemented?
Did the required tests pass?
Are the required dependencies satisfied?
Version of PHP
PHP Extensions
Framework and Libraries
Database
...
sharing experience
Package Management
PEAR Installer
PHP Archive (PHAR)
OS Package Manager (RPM, DEB, ...)
sharing experience
PHP Archive (PHAR)
<target name="phar">
<exec executable="phpab">
<arg value="--phar" />
<arg value="--output" />
<arg path="${basedir}/build/[Link]" />
<arg path="${basedir}/src" />
</exec>
</target>
sharing experience
RPM/DEB/... Packages
Version requirements (min/max)
Dependencies on other packages
Configuration
Pre/Post Installation Scripts
sharing experience
RPM/DEB/... Packages
Reproducible deployment
Redeployable
Reversible
Automatable
Single Machine
Multiple Machines
sharing experience
Automated Deployment
Build package
Continuous Integration Server
Manually when needed
Deploy to test / stage system(s)
Add to production repository
sharing experience
Automated Deployment: Push vs. Pull
Push updates to server(s)
On-demand action with full control over update
Can lead to inconsistent server infrastructure
Pull updates on server(s)
Update only pushed to one place (repository)
Fully automated process where the server(s)
automatically pull(s) updates
sharing experience
Pull Deployment with RHEL / CentOS / Fedora
YUM Update Daemon
Enable automatic download
Enable automatic update installation
Red Hat Network Satellite
sharing experience
Pull Deployment in General
Chef
Puppet
...
sharing experience
Web
[Link]
[Link]
Mail
sebastian@[Link]
Twitter @S_Bergmann
Slides [Link]
sharing experience
