Showing posts with label Phing. Show all posts
Showing posts with label Phing. Show all posts

Friday, 22 April 2011

Enforcing target descriptions within build files with a Git hook

When automating mundane tasks of a project or development environment with a build tool like Phing or Ant, the driving build file will naturally accumulate several targets and tasks over time. To ease the build file acceptance within a team and at a later stage also the contribution rate by team members, it's crucial that all build targets have a description attribute to provide at least a rough outline of the build features at hand. When these attributes are in place the (potential) build file user will get such an outline by executing the build tool's list command (phing -l or ant -p). To get a better picture of the problem at hand imagine a project poorly covered with tests and your personal attitude towards extending it or just take a peek at the screenshot below showing a very poorly documented build file.

A poorly documented build file in Phing's list view

To overcome this accumulation of some sort of technical debt (i.e. poorly documented targets) there are various options at hand. The first one, not covered in this blog post, would be to add a pursuant test which verifies the existence of a description for every target/task of the build file under test. As it's very uncommon, at least from what I've heard, to have your build files covered by tests; the next thinkable approach would be to use a Git pre-commit hook to guard your repository/ies against the creeping in of such poorly documented build files.

The next listing shows such a Git hook (also available via GitHub) scribbled away in PHP, which detects any build file(s) following a common build file naming schema (i.e. build.xml|build.xml.dist|personal-build.xml|…) , prior to the actual commit. For every target element in the detected build file(s) it's then verified that it has a description attribute and that it's actual content is long enough to carry some meaning. If one of those two requirements aren't met, the commit is rejected while revealing the build file smells to the committer, so she can fix it, as shown in the outro screenshot. Happy build file sniffing.

#!/usr/bin/php
<?php
define('DEPENDENT_EXTENSION', 'SimpleXML');

if (!extension_loaded(DEPENDENT_EXTENSION)) {
    $consoleMessage = sprintf(
        "Skipping build file checks as the '%s' extension isn't available.", 
        DEPENDENT_EXTENSION
    );
    echo $consoleMessage . PHP_EOL;
    exit(0);
}

define('MIN_TARGET_DESCRIPTION_LENGTH', 10);
define('TARGET_DESCRIPTION_ATTRIBUTE', 'description');
define('TARGET_NAME_ATTRIBUTE', 'name');
define('CHECK_DESCRIPTION_LENGTH', true);

$possibleBuildFileNames = array(
    'build.xml.dist',
    'build.xml-dist',
    'build-dist.xml',
    'build.xml',
    'personal-build.xml'
);

$violations = getAllBuildFileViolationsOfCommit($possibleBuildFileNames);
fireBackPossibleViolationsAndExitAccordingly($violations);

function getAllBuildFileViolationsOfCommit(array $possibleBuildFileNames)
{
    $filesOfCommit = array();
    $gitCommand = 'git diff --cached --name-only';
    
    exec($gitCommand, $filesOfCommit, $commandReturnCode);
    
    $allViolations = array();
    foreach ($filesOfCommit as $file) {
      if (in_array(basename($file), $possibleBuildFileNames)) {
          $violations = checkBuildFileForViolations($file);
          if (count($violations) > 0) {
            $allViolations[$file] = $violations;
          }
      }
    }

    return $allViolations;
}

/**
 *  @param  array $allViolations
 *  @return void
 */
function fireBackPossibleViolationsAndExitAccordingly(array $allViolations)
{
    if (count($allViolations) > 0) {
        foreach ($allViolations as $buildFile => $violations) {

            $buildFileConsoleMessageHeader = sprintf("Build file '%s':", $buildFile);
            echo $buildFileConsoleMessageHeader . PHP_EOL;

            foreach ($violations as $violationMessage) {
                $buildFileConsoleMessageLine = sprintf(" + %s", $violationMessage);
                echo $buildFileConsoleMessageLine . PHP_EOL;
            }
        }
        if (count($allViolations) > 1) {
            $rejectCommitConsoleMessage = sprintf(
                "Therefore rejecting the commit of build files [ %s ].", 
                implode(', ', array_keys($allViolations))
            );
        } else {
            $rejectCommitConsoleMessage = sprintf(
                "Therefore rejecting the commit of build file [ %s ].", 
                implode(', ', array_keys($allViolations))
            );
        }

        echo $rejectCommitConsoleMessage . PHP_EOL;
        exit(1);
    }
    exit(0);
}
/**
 *  @param  string $buildfile
 *  @return array
 */
function checkBuildFileForViolations($buildFile) {
    if (!file_exists($buildFile)) {
        return array();
    }

    $buildfileXml = file_get_contents($buildFile);
    $buildXml = new SimpleXMLElement($buildfileXml);
    $allBuildTargets = $buildXml->xpath("//target");
    $violations = array();

    if (count($allBuildTargets) > 0) {

        $targetsWithNoDescription = $targetsWithTooShortDescription = array();

        foreach ($allBuildTargets as $buildTarget) {

            $actualTragetAttributes = $buildTarget->attributes();
            $allUsedTragetAttributes = array();
            $actualTargetName = null;

            foreach ($actualTragetAttributes as $attribute => $value) {
                $allUsedTragetAttributes[] = $attribute;

                if ($attribute === TARGET_NAME_ATTRIBUTE) {
                    $actualTargetName = $value;
                }

                if (CHECK_DESCRIPTION_LENGTH === true && 
                    $attribute === TARGET_DESCRIPTION_ATTRIBUTE && 
                    strlen($value) < MIN_TARGET_DESCRIPTION_LENGTH) 
                {
                    $targetsWithTooShortDescription[] = $actualTargetName;
                }
            }   

            if (!in_array(TARGET_DESCRIPTION_ATTRIBUTE, $allUsedTragetAttributes)) {
                $targetsWithNoDescription[] = $actualTargetName;
            }
        }
        if (count($targetsWithNoDescription) > 0) {
            if (count($targetsWithNoDescription) > 1) {
                $violations[] = sprintf(
                    "Build targets [ %s ] don't have mandatory descriptions.", 
                    implode(', ', $targetsWithNoDescription)
                );
            } else {
                $violations[] = sprintf(
                    "Build target [ %s ] doesn't have a mandatory description.", 
                    implode(', ', $targetsWithNoDescription)
                );
            }
        }

        if (count($targetsWithTooShortDescription) > 0) {
            if (count($targetsWithTooShortDescription) > 1) {
                $violations[] = sprintf(
                    "Build targets [ %s ] don't have an adequate target description length.", 
                    implode(', ', $targetsWithTooShortDescription),
                    MIN_TARGET_DESCRIPTION_LENGTH
                );
            } else {
                $violations[] = sprintf(
                    "Build target [ %s ] doesn't have an adequate target description length.", 
                    implode(', ', $targetsWithTooShortDescription),
                    MIN_TARGET_DESCRIPTION_LENGTH
                );
            }
        }
    }
    return $violations;
}
Non-Descriptive Phing build files rejected by a Git hook

Saturday, 20 November 2010

Measuring & displaying Phing build times with buildhawk

Recently I installed a Ruby gem called buildhawk which allows to measure and display the build times of Rake driven builds. As I like the idea behind this tool a lot but mostly use Phing for build orchestration, it was time to explore the possibility to interconnect them both. In this blog post I'll show an implementation of an apposite Phing Logger gathering the buildhawk compatible build times via git note(s) and how to put the interplay between those two tools to work.

Logging on

As mentioned above the build time of each build is stored as a git note and associated to the repository's HEAD, reflecting the current state of the system under build (SUB), which assumes that the SUB is versioned via Git. The next shown Phing Logger (i.e. BuildhawkLogger) grabs the overall build time by hooking into the buildFinished method of the extended DefaultLogger class, transforms it into a buildhawk specific format and finally adds it as a git note.
<?php

require_once 'phing/listener/DefaultLogger.php';

/**
 *  Writes a build event to the console and store the build time as a git notes in the   
 *  project's repository HEAD.
 *
 *  @author    Raphael Stolt <[email protected]>
 *  @see       BuildEvent
 *  @link      https://github.com/xaviershay/buildhawk Buildhawk on GitHub
 *  @package   phing.listener
 */
class BuildhawkLogger extends DefaultLogger {
    
    /**
     *  @var string
     */
    private $_gitNotesCommandResponse = null;

    /**
     *  Behaves like the original DefaultLogger, plus adds the total build time 
     *  as a git note to current repository HEAD.
     *
     *  @param  BuildEvent $event
     *  @see    BuildEvent::getException()
     *  @see    DefaultLogger::buildFinished
     *  @link   http://www.kernel.org/pub/software/scm/git/docs/git-notes.html
     */
    public function buildFinished(BuildEvent $event) {
        parent::buildFinished($event);
        if ($this->_isProjectGitDriven($event)) {
            $error = $event->getException();
            if ($error === null) {
                $buildtimeForBuildhawk = $this->_formatBuildhawkTime(
                    Phing::currentTimeMillis() - $this->startTime
                );
                if (!$this->_addBuildTimeAsGitNote($buildtimeForBuildhawk)) {
                    $message = sprintf(
                        "Failed to add git note due to '%s'",
                        $this->_gitNotesCommandResponse
                    );
                    $this->printMessage($message, $this->err, Project::MSG_ERR);
                }
            }
        }
    }
    
    /**
     * Checks (rudimentary) if the project is Git driven
     *
     *  @param  BuildEvent $event
     *  @return boolean
     */
    private function _isProjectGitDriven(BuildEvent $event)
    {
        $project = $event->getProject();
        $projectRelativeGitDir = sprintf(
            '%s/.git', $project->getBasedir()->getPath()
        );
        return file_exists($projectRelativeGitDir) && is_dir($projectRelativeGitDir);
    }
    
    /**
     *  Formats a time micro integer to buildhawk readable format.
     *
     *  @param  integer The time stamp
     */
    private function _formatBuildhawkTime($micros) {
        return sprintf("%0.3f", $micros);
    }
    
    /**
     *  Adds the build time as a git note to the current repository HEAD
     *
     *  @param  string  $buildTime The build time of the build
     *  @return mixed   True on sucess otherwise the command failure response
     */
    private function _addBuildTimeAsGitNote($buildTime) {
        $gitNotesCommand = sprintf(
            "git notes --ref=buildtime add -f -m '%s' HEAD 2>&1",
            $buildTime
        );
        $gitNotesCommandResponse = exec($gitNotesCommand, $output, $return);
        if ($return !== 0) {
            $this->_gitNotesCommandResponse = $gitNotesCommandResponse;
            return false;
        }
        return true;
    }
}

Putting the Logger to work

As the buildhawk logger is available via GitHub you can easily grab it by issuing sudo curl -s http://gist.github.com/raw/707868/BuildhawkLogger.php -o $PHING_HOME/listener/BuildhawkLogger.php. The next step, making the build times loggable, is achieved by using the -logger command line argument of the Phing Cli and specifying the buildhawk logger name or the path to it. In case you want the buildhawk logger to be used per default (it behaves like the default logger if the SUB isn't Git driven/managed) you can also add it to the Phing shell script.

The next console command issued in the directory of the SUB shows a Phing call utilizing the BuildhawkLogger, assumed it has been installed at $PHING_HOME/listener/BuildhawkLogger.php and not been made the default logger.
phing -logger phing.listener.BuildhawkLogger

Looking at them Phing build times

Now it's time to switch to buildhawk and let it finally perform it's designated task, rendering an with the commit SHAs, commit messages, and build times fed Erb template into an informative, viewable HTML page. To install it you simply have to run sudo gem install buildhawk and you're good to go.

The next console command shows the buildhawk call issued in the SUB's directory to produce it's build time report page.
buildhawk --title 'Examplr' > examplr-build-times.html
The outro screenshot below gives you a peek at a rendered build time report.Buildhawk report for a Phing driven build

Saturday, 22 August 2009

Kicking off custom Phing task development with TextMate

As a reader of this blog you migth have noticed that from time to time I like to utilize Phing's ability to write custom tasks. Though that's not an everyday routine for me and therefor I might, depending on my form of the day, end up with some real smelly code where for example the task's properties validation is handled in the task's main worker method. This is actually a bad habit/practice I'm aware of and to improve my future endeavours in custom Phing task development, I bended TextMate's snippet feature to my needs.

Snippets in TextMate are a very powerful feature that can be used to insert code that you do not want to type again and again, or like in my case might have forgotten over a certain time.

The next code listing shows the snippet providing a basic custom Phing task class skeleton which can be utilized over and over at the beginning of the implementation activities.

<?php
require_once 'phing/Task.php';

class ${1:CustomName}Task extends Task
{
private \$_${2:property} = null;

/**
* @param string \$${2:property} ${3:description}
*/
public function set${2/./\u$0/}(\$${2:property})
{
\$this->_${2:property} = trim(\$${2:property});
}
/**
* Initializes the task environment if necessary
*/
public function init()
{
}
/**
* Does the task main work or delegates it
* @throws BuildException
*/
public function main()
{
\$this->_validateProperties();
}
/**
* Validates the task properties
* @throws BuildException
*/
private function _validateProperties()
{
if (is_null(\$this->_${2:property})) {
throw new BuildException('${4:message}.');
}$0
}
}
To apply the snippet, after installing it, on a PHP source file it can either be selected from the Bundles menue or more comfortable via the assigned tab trigger i.e. ctask. After triggering the snippet it's possible to properly name the task under development and dynamically set it's first property, which is also treated as a mandatory property in the extracted _validateProperties method.

The outro image shows the above stated snippet in the TextMate Bundle Editor and it's configuration.

Phing snippet in the TextMate Bundle Editor

Sunday, 10 May 2009

Testing Phing buildfiles with PHPUnit

While transforming some of the Ant buildfile refactorings described in Julian Simpson's seminal essay into a Phing context, it felt plainly wrong that I didn't have any tests for the buildfile to back me up on obtaining the pristine behaviour throughout the process. While Ant users can rely on an Apache project called AntUnit there are currently no tailor-made tools available for testing or verifying Phing buildfiles. Therefor I took a weekend off, locked myself in the stuffy lab, and explored the abilities to test Phing buildfiles respectively their included properties, targets and tasks with the PHPUnit testing framework. In case you'd like to take a peek at the emerged lab jottings, keep on scanning.

Introducing the buildfile under test

The buildfile that will be used as an example is kept simple, and contains several targets ranging from common ones like initializing the build environment by creating the necessary directories to more specific ones like pulling an external artifact from GitHub. To get an overview of the buildfile under test have a look at the following listing.
<?xml version="1.0" encoding="UTF-8"?>
<project name="test-example" default="build" basedir=".">

  <property name="project.basedir" value="." override="true" />
  <property name="github.repos.dir" value="${project.basedir}/build/github-repos" override="true" />

  <target name="clean" depends="clean-github-repos" description="Removes runtime build artifacts">
    <delete dir="${project.basedir}/build" includeemptydirs="true" verbose="false" failonerror="true" />
    <delete dir="${project.basedir}/build/reports" includeemptydirs="true" verbose="false" failonerror="true" />
  </target>

  <target name="clean-github-repos" description="Removes runtime build artifacts">
    <delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
  </target>

  <target name="-log-build" description="A private target which should only be invoked internally">
    <!-- omitted -->
  </target>

  <target name="build" depends="clean" description="Builds the distributable product">
    <!-- omitted -->
  </target>

  <target name="database-setup" description="Sets up the database structure">
    <!-- omitted -->
  </target>

  <target name="init" description="Initalizes the build by creating directories etc">
    <mkdir dir="${project.basedir}/build/logs/performance/" />
    <mkdir dir="${project.basedir}/build/doc" />
    <mkdir dir="${project.basedir}/build/reports/phploc" />
  </target>

  <target name="init-ad-hoc-tasks" 
          description="Initalizes the ad hoc tasks for reusability in multiple targets">      
    <adhoc-task name="github-clone"><![CDATA[
  class Github_Clone extends Task {
    private $repository = null;
    private $destDirectory = null;

    function setRepos($repository) {
      $this->repository = $repository;
    }
    function setDest($destDirectory) {
      $this->destDirectory = $destDirectory;
    }
    function main() {
      // Get project name from repos Uri
      $projectName = str_replace('.git', '',
        substr(strrchr($this->repository, '/'), 1));

      $gitCommand = 'git clone ' . $this->repository . ' ' .
      $this->destDirectory . '/' . $projectName;

      exec(escapeshellcmd($gitCommand), $output, $return);

      if ($return !== 0) {
        throw new BuildException('Git clone failed');
      }
      $logMessage = 'Cloned Git repository ' . $this->repository .
        ' into ' . $this->destDirectory . '/' . $projectName;
      $this->log($logMessage);
    }
  }
]]></adhoc-task>

    <echo message="Intialized github-clone ad hoc task." />
  </target>

  <target name="github" depends="init-ad-hoc-tasks, clean-github-repos" 
          description="Clones given repositories from GitHub">
    <github-clone repos="git://github.com/raphaelstolt/phploc-phing.git" dest="${github.repos.dir}" />
  </target>
</project>

Testing the buildfile

All tests for the buildfile under test will be bundled, like 'normal' tests, in a class i.e. BuildfileTest extending the PHPUnit_Framework_TestCase class. When testing buildfiles it's possible to build some tests around the actual buildfile XML structure, by utilizing the xpath method of PHP's SimpleXMLElement class and asserting against the XPath query results, or around the dispatching of specific targets and asserting against the expected build artifacts. Furthermore these two identified groups, structure and artifact, can be used to organize the accumulating tests via PHPUnit's @group annotation.

To be able to dispatch specific build targets and feed them with properties if necessary I additionally developed a very basic build runner shown in the next code listing.
<?php
class Phing_Buildfile_Runner {

    private $_buildfilePath = null;

    public function __construct($buildfilePath) {
        if (!file_exists($buildfilePath)) {
            throw new Exception("Buildfile '{$buildfilePath}' doesn't exist");
        }
        $this->buildfilePath = realpath($buildfilePath); 
    }
    public function runTarget($targets = array(), $properties = array()) {
        $runTargetCommand = "phing " . "-f {$this->buildfilePath} ";
        if (count($targets) > 0) {
            foreach ($targets as $target) {
                $runTargetCommand.= $target . " ";
            }
        }
        if (count($properties) > 0) {
            foreach ($properties as $property => $value) {
                $runTargetCommand.= "-D{$property}={$value} ";
            }       
        }
        exec(escapeshellcmd($runTargetCommand), $output, $return);
        return array('output' => $output, 'return' => $return);
    }
}
Out of the box PHPUnit's assertion pool provides all the utilities to test buildfiles; although it would be cleaner to create domain specfic assertions for this testing domain this technique will be ignored for the sake of brevity.

After an initial 1000ft view on how to test buildfiles let's jump into the actual testing of a structural aspect of the buildfile under test. The test to come shows how to verify that a clean target is defined for playing along in the build orchestra by querying a XPath expression against the buildfile XML and asserting that a result is available.
/**
 * @test
 * @group structure
 */
public function buildfileShouldContainACleanTarget() {
    $xml = new SimpleXMLElement($this->_buildfileXml);
    $cleanElement = $xml->xpath("//target[@name='clean']");
    $this->assertTrue(count($cleanElement) > 0, "Buildfile doesn't contain a clean target");
}
The next artifactual test raises the bar an inch, by verifying that the defined init target of the build does initialize the build environment correctly, or to pick up the orchestra metaphor again that the specific instrument plays along and holds the directed tone. Therefor the build runner executes the target and afterwards asserts a list of expected artifacts against the current state of the build process.
/**
 * @test
 * @group artifact
 */
public function initTargetShouldCreateInitialBuildArtifacts() {
    $this->_isTearDownNecessary = true;
    $this->_buildfileRunner->runTarget(array('init'));
    $expectedInitArtifacts = array(
        "{$this->_buildfileBasedir}/build",
        "{$this->_buildfileBasedir}/build/logs/performance/",
        "{$this->_buildfileBasedir}/build/doc",
        "{$this->_buildfileBasedir}/build/reports"
    );

    foreach ($expectedInitArtifacts as $artifact) {
        $this->assertFileExists($artifact, "Expected file '{$artifact}' doesn't exist");
    }
}
The next code listing shows the whole picture of the BuildfileTest class containing additional test methods verifying different aspects of the buildfile under test and also the innards of the setup and teardown method.

The main assignment of the setup method is to load the XML of the buildfile under test and to intialize the build runner so an instance is available for an use in artifactual tests. The teardown method its sole responsibility is to reset the build state by running the clean target of the buildfile.
<?php
require_once 'PHPUnit/Framework.php';
require_once 'Phing/Buildfile/Runner.php';

class ExampleBuildfileTest extends PHPUnit_Framework_TestCase {

    protected $_buildfileXml = null;
    protected $_buildfileName = null;
    protected $_buildfileBasedir = null;
    protected $_buildfileRunner = null;
    protected $_isTearDownNecessary = false;

    protected function setUp() {
        $this->_buildfileName = realpath('../../build.xml');        
        $this->_buildfileBasedir = dirname($this->_buildfileName);
        $this->_buildfileXml = file_get_contents($this->_buildfileName);
        $this->_buildfileRunner = new Phing_Buildfile_Runner(
        $this->_buildfileName);
    }

    protected function tearDown() {
        if ($this->_isTearDownNecessary) {
            $this->_buildfileRunner->runTarget(array('clean'));
        }
    }

   /**
    * @test
    * @group structure
    */
    public function targetBuildShouldBeTheDefaultTarget() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//@default";
        $defaultElement = $xml->xpath($xpath);
        $this->assertSame('build', trim($defaultElement[0]->default), 
            "Buildfile doesn't have a default target named 'build'"
        );
    }
   /**
    * @test
    * @group structure
    */
    public function propertyGithubReposDirShouldBeSet() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//property[@name='github.repos.dir']/@value";
        $valueElement = $xml->xpath($xpath);
        $this->assertTrue($valueElement[0] instanceof SimpleXMLElement, 
            "Buildfile doesn't contain a 'github.repos.dir' property"
        );
        $this->assertGreaterThan(1, strlen($valueElement[0]->value));
    }
   /**
    * @test
    * @group structure
    */
    public function buildfileShouldContainACleanTarget() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $cleanElement = $xml->xpath("//target[@name='clean']");
        $this->assertTrue(count($cleanElement) > 0, 
            "Buildfile doesn't contain a clean target"
        );
    }
   /**
    * @test
    * @group structure
    */
    public function targetLogBuildShouldBeAPrivateOne() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $nameElement = $xml->xpath("//target[@name='-log-build']");
        $this->assertTrue(count($nameElement) > 0, 
            'Log build target is not a private target'
        );
    }
    /**
     * @test
     * @group structure
     */
    public function targetBuildShouldDependOnCleanTarget() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//target[@name='build']/@depends";
        $dependElement = $xml->xpath($xpath);
        $this->assertTrue(count($dependElement) > 0, 
            'Target build contains no depends attribute'
        );
        $dependantTasks = array_filter(explode(' ',
            trim($dependElement[0]->depends))
        );
        $this->assertContains('clean', $dependantTasks, "Target build doesn't 
            depend on the clean target"
        );
    }
    /**
     * @test
     * @group structure
     */
    public function allDefinedTargetsShouldHaveADescriptionAttribute() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//target";
        $targetElements = $xml->xpath($xpath);
        $describedTargetElements = array();
        foreach ($targetElements as $index => $targetElement) {
            $targetDescription = trim($targetElement->attributes()->description); 
            if ($targetDescription !== '') {
                $describedTargetElements[] = $targetDescription;
            }
        }
        $this->assertEquals(count($targetElements),
            count($describedTargetElements), 
            'Description not for all targets set'
        );
    }
    /**
     * @test
     * @group structure
     */
    public function githubCloneAdhocTaskShouldBeDefined() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//target[@name='init-ad-hoc-tasks']/adhoc-task";
        $adhocElement = $xml->xpath($xpath);
        $this->assertSame('github-clone',
            trim($adhocElement[0]->attributes()->name), 
            "Ad hoc task 'github-clone' isn't defined"
        );
    }
    /**
    * @test 
    * @group artifact
    */
    public function initTargetShouldCreateInitialBuildArtifacts() {
        $this->_isTearDownNecessary = true;
        $this->_buildfileRunner->runTarget(array('init'));

        $expectedInitArtifacts = array(
            "{$this->_buildfileBasedir}/build", 
            "{$this->_buildfileBasedir}/build/logs/performance/", 
            "{$this->_buildfileBasedir}/build/doc",
            "{$this->_buildfileBasedir}/build/reports"
        );

        foreach ($expectedInitArtifacts as $artifact) {
            $this->assertFileExists($artifact, 
                "Expected file '{$artifact}' doesn't exist"
            );
        }
    }
    /**
     * @test
     * @group artifact
     */
    public function sqlFilesForDatabaseSetupTargetShouldBeAvailable() {
        $expectedSqlFiles = array(
            "{$this->_buildfileBasedir}/sqlfiles", 
            "{$this->_buildfileBasedir}/sqlfiles/session-storage.sql", 
            "{$this->_buildfileBasedir}/sqlfiles/acl.sql", 
            "{$this->_buildfileBasedir}/sqlfiles/log.sql"
        );

        foreach ($expectedSqlFiles as $sqlFile) {
            $this->assertFileExists($sqlFile, 
                "SQL file '{$sqlFile}' doesn't exist"
            );
        }
    }
    /**
     * @test
     * @group artifact
     */
    public function githubTargetShouldFetchExpectedRepository() {
        $this->_isTearDownNecessary = true;
        $this->_buildfileRunner->runTarget(array('github'));
        $expectedGitRepository = "{$this->_buildfileBasedir}/build/"
            . "github-repos/phploc-phing/.git";
        $this->assertFileExists($expectedGitRepository, 
            "Github target doesn't fetch the expected 'phploc-phing' repository"
        );
    }
}
The outro screenshot shows the above stated test class run against the example buildfile on a Mac OS X system utilizing the --colors option; which by the way comes in really handy in combination with Stakeout.rb during the process of refactoring or extending/creating buildfiles the test-driven way.

PHPUnit console output

Saturday, 18 April 2009

Creating and using Phing ad hoc tasks

Sometimes there are build scenarios where you'll badly need a functionality, like adding a MD5 checksum file to a given project, that isn't provided neither by the available Phing core nor the optional tasks. Phing supports developers with two ways for extending the useable task pool: by writing 'outline' tasks that will end up in a directory of the Phing installation or by utilizing the AdhocTaskdefTask, which allows to define custom tasks in the buildfile itself. The following post will try to outline how to define and use these inline tasks, by sketching an ad hoc task that enables the build orchestra to clone Git repositories from GitHub during a hypothetical workbench setup.

Creating the inline/ad hoc task

The AdhocTaskdefTask expects a name attribute i.e. github-clone for the XML element which will later referr to the ad hoc task and a CDATA section hosting the task implementation. Similar to 'outline' tasks the ad hoc task extends Phing's Task class, configures the task via attributes and holds the logic to perform. Unfortunately inline task implementations don't allow to require or include external classes available in the include_path, like Zend_Http_Client which I initially tried to use for an example task fetching short Urls from is.gd. This limits the available functions and classes to craft the task from to the ones built into PHP. The following buildfile snippet shows the implementation of the github-clone ad hoc task which is wrapped by a private target to encourage reusability and limit it's callability.
<target name="-init-ad-hoc-tasks" 
description="Initializes the ad hoc task(s)">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {

private $repository = null;
private $destDirectory = null;

function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));

$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;

exec(escapeshellcmd($gitCommand), $output, $return);

if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Initialized github-clone ad hoc task." />
</target>

Using the ad hoc task

With the ad hoc task in the place to be, it's provided functionality can now be used from any target using the tasks XML element according to the given name i.e. github-clone in the AdhocTaskdefTask element earlier and by feeding it with the required attributes i.e. repos and dest. The next snippet allows you to take a peek at the complete buildfile with the ad hoc task in action.
<?xml version="1.0" encoding="UTF-8"?>
<project name="recordshelf" default="init-work-bench" basedir=".">

<property name="github.repos.dir" value="./github-repos" override="true" />

<target name="init-work-bench"
depends="-init-ad-hoc-tasks, -clone-git-repos"
description="Initializes the hypothetical workbench">
<echo message="Initialized workbench." />
</target>

<target name="-clean-git-repos"
description="Removes old repositories before initializing a new workbench">
<delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
</target>

<target name="-init-ad-hoc-tasks"
description="Initializes the ad hoc task(s)">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {

private $repository = null;
private $destDirectory = null;

function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));

$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;

exec(escapeshellcmd($gitCommand), $output, $return);

if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Initialized github-clone ad hoc task." />
</target>

<target name="-clone-git-repos" depends="-clean-git-repos"
description="Clones the needed Git repositories from GitHub">
<github-clone repos="git://github.com/abc/abc.git"
dest="${github.repos.dir}" />
<github-clone repos="git://github.com/xyz/xyz.git"
dest="${github.repos.dir}" />
</target>

</project>

Favouring inline over 'outline' tasks?

The one big advantage of using inline tasks over 'outline' tasks is that they are distributed with the buildfile and are instantly available without the need to modify the Phing installation. Some severe disadvantages of inline tasks are the limitation to use only the core PHP functions and classes for the implementation, the introduction of an additional hurdle to verify the task behaviour via PHPUnit as it's located in a CDATA section of the buildfile and the fact that the use of several inline tasks will blow up the buildfile, and thereby obfuscate the build flow.

Regrettably Phing doesn't provide an import task like Ant which might enable a refactoring to pull the ad hoc task definitions into a seperate XML file and include them at buildtime; in case you might have some expertise or ideas for a suitable workaround hit me with a comment. So far I tried to get it working, with no success, by utilizing Phing's PhingTask and XML's external entities declaration.

Sunday, 22 February 2009

Phplocing your projects with Phing

When I started to play around with Ruby on Rails, my attention got somehow soon drawn to it's Rake stats task, which provides developers or more likely project managers with an overview of the actual project size. Exactly one month ago Sebastian Bergmann, of PHPUnit fame, started to implement a similar tool dubbed phploc which can give you an overview of the size for any given PHP project. As I wanted to automate the invocation of this handy tool and collect it's report output out of a Phing buildfile, I invested some time to develop a custom Phing task doing so. Thereby the following post will show you a possible implementation of this task and it's use in a buildfile.

Installing phploc

To setup phploc on your system simply install the phploc PEAR package available from the pear.phpunit.de channel as shown in the next commands. In case you already have installed PHPUnit via PEAR you can omit the channel-discover command.
sudo pear channel-discover pear.phpunit.de
sudo pear install phpunit/phploc

Implementing the phploc task

As I already blogged about developing custom Phing task I'm only going to show the actual implementation and not dive into any details; alternatively you can also grab it from this public GitHub repository.
<?php
require_once 'phing/Task.php';
require_once 'phing/BuildException.php';
require_once 'PHPLOC/Analyser.php';
require_once 'PHPLOC/Util/FilterIterator.php';
require_once 'PHPLOC/TextUI/ResultPrinter.php';

class PHPLocTask extends Task
{
protected $suffixesToCheck = null;
protected $acceptedReportTypes = null;
protected $reportDirectory = null;
protected $reportType = null;
protected $fileToCheck = null;
protected $filesToCheck = null;
protected $reportFileName = null;
protected $fileSets = null;

public function init() {
$this->suffixesToCheck = array('php');
$this->acceptedReportTypes = array('cli', 'txt', 'xml');
$this->reportType = 'cli';
$this->reportFileName = 'phploc-report';
$this->fileSets = array();
$this->filesToCheck = array();
}
public function setSuffixes($suffixListOrSingleSuffix) {
if (stripos($suffixListOrSingleSuffix, ',')) {
$suffixes = explode(',', $suffixListOrSingleSuffix);
$this->suffixesToCheck = array_map('trim', $suffixes);
} else {
array_push($this->suffixesToCheck, trim($suffixListOrSingleSuffix));
}
}
public function setFile(PhingFile $file) {
$this->fileToCheck = trim($file);
}
public function createFileSet() {
$num = array_push($this->fileSets, new FileSet());
return $this->fileSets[$num - 1];
}
public function setReportType($type) {
$this->reportType = trim($type);
}
public function setReportName($name) {
$this->reportFileName = trim($name);
}
public function setReportDirectory($directory) {
$this->reportDirectory = trim($directory);
}
public function main() {
if (!isset($this->fileToCheck) && count($this->fileSets) === 0) {
$exceptionMessage = "Missing either a nested fileset or the "
. "attribute 'file' set.";
throw new BuildException($exceptionMessage);
}
if (count($this->suffixesToCheck) === 0) {
throw new BuildException("No file suffix defined.");
}
if (is_null($this->reportType)) {
throw new BuildException("No report type defined.");
}
if (!is_null($this->reportType) &&
!in_array($this->reportType, $this->acceptedReportTypes)) {
throw new BuildException("Unaccepted report type defined.");
}
if (!is_null($this->fileToCheck) && !file_exists($this->fileToCheck)) {
throw new BuildException("File to check doesn't exist.");
}
if ($this->reportType !== 'cli' && is_null($this->reportDirectory)) {
throw new BuildException("No report output directory defined.");
}
if (count($this->fileSets) > 0 && !is_null($this->fileToCheck)) {
$exceptionMessage = "Either use a nested fileset or 'file' "
. "attribute; not both.";
throw new BuildException($exceptionMessage);
}
if (!is_null($this->reportDirectory) && !is_dir($this->reportDirectory)) {
$reportOutputDir = new PhingFile($this->reportDirectory);
$logMessage = "Report output directory does't exist, creating: "
. $reportOutputDir->getAbsolutePath() . '.';
$this->log($logMessage);
$reportOutputDir->mkdirs();
}
if ($this->reportType !== 'cli') {
$this->reportFileName.= '.' . trim($this->reportType);
}
if (count($this->fileSets) > 0) {
$project = $this->getProject();
foreach ($this->fileSets as $fileSet) {
$directoryScanner = $fileSet->getDirectoryScanner($project);
$files = $directoryScanner->getIncludedFiles();
$directory = $fileSet->getDir($this->project)->getPath();
foreach ($files as $file) {
if ($this->isFileSuffixSet($file)) {
$this->filesToCheck[] = $directory . DIRECTORY_SEPARATOR
. $file;
}
}
}
$this->filesToCheck = array_unique($this->filesToCheck);
}
if (!is_null($this->fileToCheck)) {
if (!$this->isFileSuffixSet($file)) {
$exceptionMessage = "Suffix of file to check is not defined in"
. " 'suffixes' attribute.";
throw new BuildException($exceptionMessage);
}
}
$this->runPhpLocCheck();
}
protected function isFileSuffixSet($filename) {
$pathinfo = pathinfo($filename);
$fileSuffix = $pathinfo['extension'];
return in_array($fileSuffix, $this->suffixesToCheck);
}
protected function runPhpLocCheck() {
$files = $this->getFilesToCheck();
$result = $this->getCountForFiles($files);

if ($this->reportType === 'cli' || $this->reportType === 'txt') {
$printer = new PHPLOC_TextUI_ResultPrinter;
if ($this->reportType === 'txt') {
ob_start();
$printer->printResult($result);
file_put_contents($this->reportDirectory
. DIRECTORY_SEPARATOR . $this->reportFileName,
ob_get_contents());
ob_end_clean();
$reportDir = new PhingFile($this->reportDirectory);
$logMessage = "Writing report to: "
. $reportDir->getAbsolutePath() . DIRECTORY_SEPARATOR
. $this->reportFileName;
$this->log($logMessage);
} else {
$printer->printResult($result);
}
} elseif ($this->reportType === 'xml') {
$xml = $this->getResultAsXml($result);
$reportDir = new PhingFile($this->reportDirectory);
$logMessage = "Writing report to: " . $reportDir->getAbsolutePath()
. DIRECTORY_SEPARATOR . $this->reportFileName;
$this->log($logMessage);
file_put_contents($this->reportDirectory . DIRECTORY_SEPARATOR
. $this->reportFileName, $xml);
}
}
protected function getFilesToCheck() {
if (count($this->filesToCheck) > 0) {
$files = array();
foreach ($this->filesToCheck as $file) {
$files[] = new SPLFileInfo($file);
}
} elseif (!is_null($this->fileToCheck)) {
$files = array(new SPLFileInfo($this->fileToCheck));
}
return $files;
}
protected function getCountForFiles($files) {
$count = array('files' => 0, 'loc' => 0, 'cloc' => 0, 'ncloc' => 0,
'eloc' => 0, 'interfaces' => 0, 'classes' => 0, 'functions' => 0);
$directories = array();

foreach ($files as $file) {
$directory = $file->getPath();
if (!isset($directories[$directory])) {
$directories[$directory] = TRUE;
}
PHPLOC_Analyser::countFile($file->getPathName(), $count);
}

if (!function_exists('parsekit_compile_file')) {
unset($count['eloc']);
}
$count['directories'] = count($directories) - 1;
return $count;
}
protected function getResultAsXml($result) {
$newline = "\n";
$newlineWithSpaces = sprintf("\n%4s",'');
$xml = '<?xml version="1.0" encoding="UTF-8"?>';
$xml.= $newline . '<phploc>';

if ($result['directories'] > 0) {
$xml.= $newlineWithSpaces . '<directories>' . $result['directories'] . '</directories>';
$xml.= $newlineWithSpaces . '<files>' . $result['files'] . '</files>';
}
$xml.= $newlineWithSpaces . '<loc>' . $result['loc'] . '</loc>';

if (isset($result['eloc'])) {
$xml.= $newlineWithSpaces . '<eloc>' . $result['eloc'] . '</eloc>';
}
$xml.= $newlineWithSpaces . '<cloc>' . $result['cloc'] . '</cloc>';
$xml.= $newlineWithSpaces . '<ncloc>' . $result['ncloc'] . '</ncloc>';
$xml.= $newlineWithSpaces . '<interfaces>' . $result['interfaces'] . '</interfaces>';
$xml.= $newlineWithSpaces . '<classes>' . $result['classes'] . '</classes>';
$xml.= $newlineWithSpaces . '<methods>' . $result['functions'] . '</methods>' . $newline;
$xml.= '</phploc>';
return $xml;
}
}

Hooking the phploc task into Phing

To use the task in your Phing builds simply copy it into the phing/tasks/my directory and make it available via the taskdef task. The next table shows the available task attributes and the values they can take to configure it's behaviour and output. As you will see it also provides the ability to generate reports in a XML format; I chose to implement this feature to have the possibilty to transform the report results into HTML documents by applying for example a XSLT stylesheet. This way they can provide more value to non-technical project members or can be made accessible in a CI system dashboard if desired.

NameTypeDescriptionDefaultRequired
reportTypestringThe type of the report. Available types are cli|txt|xml.cliNo
reportNamestringThe name of the report type without a file extension.phploc-reportNo
reportDirectorystringThe directory to write the report file to.falseYes, when report type txt or xml is defined.
filestringThe name of the file to check.n/aYes, when no nested fileset is defined.
suffixesstringA comma-separated list of file suffixes to check.phpNo

Supported Nested Tags:
  • fileset
The closing buildfile extract shows an example phploc task configuration and is also available at the public GitHub repository. Happy phplocing!
<?xml version="1.0"?>
<project name="example" default="phploc" basedir=".">
<taskdef name="phploc" classname="phing.tasks.my.PHPLocTask" />
<target name="phploc">
<tstamp>
<format property="check.date.time" pattern="%Y%m%d-%H%M%S" locale="en_US"/>
</tstamp>
<phploc reportType="txt" reportName="${check.date.time}-report"
reportDirectory="phploc-reports">
<fileset dir=".">
<include name="**/*.php" />
<include name="*.php" />
</fileset>
</phploc>
</target>
</project>

Sunday, 26 October 2008

Getting a visualization of a Phing buildfile

Today I spent some time to get a tool running to visualize Phing buildfiles as this can come in handy for maintaing, refactoring or extending large buildfiles. Out of the box the Phing -l option can be used to get a first overview of all available targets in a given buildfile but it doesn't untangle the target dependencies and sometimes a picture is still worth a thousand words. Luckily the Ant community already provides several tools to accomplish the visualization of Ant buildfiles, reaching from solutions that apply a Xslt stylesheet upon a given buildfile e.g. ant2dot to those ones that take a programmatically approach e.g. Grand. All these solutions utilize Graphiz to generate a graphic from a DOT file representing the buildfile structure, it's targets and their dependencies. As Phing is a very close descendant of Ant the Xslt approach was best suited and the one with the least effort because their buildfile markup is very similar. The following post will walk you through on how to get a simple Phing buildfile visualization tool running in just a few minutes.

Grabbing the Xslt file

The first step is to get the ant2dot Xslt stylesheet and put it into the same directory as the visualization buildfile and target to come. Due to the aforementioned Phing and Ant buildfile markup similarities it can be used without any modfications.

Setting up the buildfile visualization target

The next step is to create a Phing target that utilizes the Xslt task to transfrom the fed buildfile into a DOT file which gets passed further to a platform dependent Exec task handling the final transformation into a PNG image. To make the visualization target independent from the buildfile to visualize it's hosted in an own buildfile and the target accepts the buildfile to be transformed as a property passed to the Phing Cli or if none given uses the default build.xml. Further the Xslt stylesheet accepts several parameters to add extended data to the resulting DOT file/PNG image which can be set in the <param> tags of the Xslt task. For a list of possible parameters have a look at the options section of ant2dot. The following codesnippet shows the visualization buildfile and the visualize target doing the Whodini like magic.
<?xml version="1.0"?>

<project name="buildfile-visualizer" default="visualize" basedir=".">

<target name="visualize"
description="Generates a visualization(PNG image) of a given buildfile">
<property name="buildfile" value="build.xml" />
<property name="phing2dot.xsl" value="${project.basedir}/ant2dot.xsl" />
<property name="dot.file" value="${buildfile}.dot" />
<property name="png.file" value="${buildfile}.png" />
<property name="dot.command.win" value="dot.exe -Tpng ${dot.file} -o ${png.file}" />
<property name="dot.command.mac" value="dot -Tpng ${dot.file} -o ${png.file}" />
<!-- Transform buildfile into DOT file -->
<xslt file="${buildfile}" tofile="${project.basedir}/${dot.file}"
style="${phing2dot.xsl}" overwrite="true">
<param name="graph.label" expression="${buildfile}" />
<param name="use.target.description" expression="true" />
</xslt>
<!-- Generate image from DOT file -->
<exec command="${dot.command.win}"
dir="${project.basedir}" os="WINNT" />
<exec command="${dot.command.mac}"
dir="${project.basedir}" os="Darwin" />

<delete file="${project.basedir}/${dot.file}" />
</target>

</project>

Running the buildfile visualization target

Now as mostly all necessary pieces are available it's time to check if the DOT command is available on the targeted platform by running a dot(.exe) -V on the console. If it isn't available it has to be installed, this might take several minutes depending on the given platform. Finally with everything in place the visualization process/target can be kicked off by calling Phing the
following way.
triton:tmp stolt$ phing -f buildfile-visualizer.xml [-Dbuildfile=<targeted-buildfile.xml>]
The last picture shows the visualization of the simple buildfile described in the Phing Userguide but it's also possible to get a meaningful visualization of larger buildfiles like the one I currently use for setting up Zend Framework based projects.

Visualization of a simple buildfile

Friday, 4 July 2008

Six valuable Phing build file refactorings

Some weeks months ago I finally got my hands on the ThoughtWorks Anthology and got immediately hooked on one of the featured essays called 'Refactoring Ant Build Files' contributed by Julian Simpson aka the build doctor. After absorbing and studying the provided catalogue of overall 24 refactorings, I spent some time to transform a few health-promoting ones to the Phing universe. So the following post will outline six five basic, but valuable Phing build file refactorings by showing the smelly example first, followed by the scentless one and a closing refactoring description.

Making build files 'first-class' codebase citizens

You might ask yourself if there even is any need to refactor and care about mostly poorly treated project artifacts like build files. Well according to the book market there are growing needs and catalogues for refactoring non-sourcecode matters like databases and nowadays even (X)HTML, and as build tools are often used to automate the whole build and delivery process of complete projects, their feed build files should be readable and clear, painless maintainable and easily customizable as the controlled project takes new directions and hurdles.

Today build files are also often used to drive the Continuous Integration(CI) process and are heavily used to run local development builds prior to commiting the finished development tasks, therefor messy and clotty build files will have a counterproductive impact on the build management lifecycle and on making required changes to it. So when striving for codebase citizen equality agree upon a build file coding standard (e.g. 5. Introduce distinct target naming or a common element indentation), reside all build files of a project in a SCM system like Subversion or Git and try not to neglect the build files constantly just because they aren't actual business logic.

Trapeze balancing without a safety net

While developing an applications business logic 'ideally' automated behaviour verifying tests are written, whether in a test-first or test-last approach, and these are building the implicit and required safety net for all follow-up refactorings. For build files there are currently no tailored testing tools/frameworks available to warrant their external behaviour during and after a refactoring, though it might be possible to create a basic safety net by using for example a combination of PHPUnit's assertFileExists and assertContains assertions. And even if there were such tools available, it's rather questionable that these would be applied to test-drive mostly simple starting and incremental evolving build files. So currently this flavour of refactoring needs to be applied with much more descipline and caution than classic sourcecode refactorings, even 'old-school' manual tests have to be run frequently and only with a proceeding practice more and more agressive refactorings will become an everyday tool. After this short note of caution, let us jump right into the refactoring catalogue extract.

1. Extract target

Take parts of a large target, declare them as independent targets and preserve dependencies by using the
target depends attribute.
Before
<target name="what-a-clew">

<phplint>
<fileset dir="${build.src}">
<include name="**/*.php"/>
</fileset>
</phplint>

<phpcodesniffer standard="PEAR" format="summary">
<fileset dir="${build.src}">
<include name="**/*.php"/>
</fileset>
</phpcodesniffer>

<phpunit>
<formatter todir="reports" type="xml"/>
<batchtest>
<fileset dir="="${build.tests}">
<include name="**/*Test*.php"/>
</fileset>
</batchtest>
</phpunit>

</target>
After:
<target name="phplint-report">
<phplint>
....
</phplint>
</target>

<target name="sniff-report" depends="phplint-report">
<phpcodesniffer standard="PEAR" format="summary">
....
</phpcodesniffer>
</target>

<target name="test-report" depends="sniff-report">
<phpunit>
....
</phpunit>
</target>
The Extract target refactoring is similar to the well-known Extract method refactoring catalogued by Martin Fowler and should be applied to unclutter long targets, which can become hard to understand and troubleshoot while maintaining or extending a build file. This refactoring is achieved by taken each atomic task (e.g. phplint) of the cluttered target and provide them each a own target (e.g. phplint-report) while the former tasks execution sequence can be obtained by utilizing the target depends attribute. You can compare this refactoring to the technique of tackling a method that's to large and infringes upon the single responsibility principle.

While twiddling with this refactoring I came up with a follow-up and hand in hand refactoring that might go by the name of Introduce facade target, which simply orchestrates the target execution sequence so you can remove the depends attribute of all orchestrated targets and thereby use them separately if needed and advisable. The following build file extract shows the result of this refactoring in action.

1.1 Introduce facade target

Provide a facade target to obtain the task execution sequence and to make each involved target a single callable unit.
After:
<target name="phplint-report">
<phplint>
....
</phplint>
</target>

<target name="sniff-report" depends="phplint-report">
<phpcodesniffer standard="PEAR" format="summary">
....
</phpcodesniffer>
</target>

<target name="test-report" depends="sniff-report">
<phpunit>
....
</phpunit>
</target>

<target name="quality-report" depends="lint-report, sniff-report, test-report"
description="Generates the overall projects quality report" />

2. Introduce property file

Move infrequently changing properties from the build file body to a flat file.
Before:
<property name="db.port" value="3306" />
<property name="db.name" value="example" />
<property name="db.user" value="funkdoc" />
....
<property name="runtime.property.x" value="default" />
....
After:

build.poperties file
[example properties]
db.port = 3306
db.name = example
db.user = funkdoc
....

build file
<property file="build.properties" />
<property name="runtime.property.x" value="default" />
....
The Introduce property file refactoring can be applied for moving infrequently changing or static properties out of the main build file body to raise the overall legibility and keep them distinct from runtime properties. The downside of this refactoring is a lost of property visibility and breaking up the former single build file into multiple units, which is contradictory to the third ANT best practice named 'Prefer a Single Buildfile' of an older best practice catalogue compiled by Eric M. Burke. So in this case, like in any case, you have to make your own choice based on your needs and requirements.

3. Replace comment with description

Annotate elements(targets) with the description attribute instead of XML comments.
Before:
<!-- This target runs the PHP_Codesniffer task and reports coding standard violations --!>
<target name="sniff-report">
....
</target>
After:
<target name="sniff-report" description="Runs the PHP_Codesniffer task and reports coding standard violations">
....
</target>
Often build files are accentuated with plain XML comments to retain the mechanics and purpose of build file elements(i.e. targets) and can become a diversionary/obscuring source while maintaining or extending a build file. By using the available description attribute of the target element to annotate its purpose it's possible to reduce that kind of noise and even better, if used constantly, they can provide valuable information about all accumulated targets of a build file when phing is called with the -l(ist) option. As you can see the Replace comment with description refactoring requires a minimum of effort/investment to achieve a very valuable impact.

4. Reuse elements by id

Declare an instance e.g. a fileset once and make references to it elsewhere to reduce duplication and increase clarity.
Before:
<target name="phplint-report">
<phplint>
<fileset dir="${build.src}">
<include name="**/*.php"/>
</fileset>
</phplint>
</target>

<target name="sniff-report" depends="phplint-report">
<phpcodesniffer standard="PEAR" format="summary">
<fileset dir="${build.src}">
<include name="**/*.php"/>
</fileset>
</phpcodesniffer>
</target>
After:
<fileset id="src_artifacts" dir="${build.src}">
<include name="**/*.php"/>
</fileset>

<target name="phplint-report">
<phplint>
<fileset refid="src_artifacts" />
</phplint>
</target>

<target name="sniff-report" depends="phplint-report">
<phpcodesniffer standard="PEAR" format="summary">
<fileset refid="src_artifacts" />
</phpcodesniffer>
</target>
The Reuse elements by id refactoring is, as the short description states, tailor-made to increase clarity while reducing code duplication, which is when present a risk that an alternation made to one element will be skipped for the other duplicates, by declaring top-level elements once by assigning an id attribute to it and then referring to it thoughout the rest of the build file. This refactoring is best compared to the classic sourcecode refactoring called Pull Up Method also catalogued by Martin Fowler and moreover it enforces the compliance with the DRY principle by providing a single point of change for futurities alternations.

5. Introduce distinct target naming

Use a different punctuation for targets and properties to enhance readability.
Before:
<property name="example.property1" value="abc" />
<property name="example_property2" value="def" />
<property name="example-Property3" value="ghi" />

<target name="example.target1">
<echo msg="${example.property1}" />
....
</target>

<target name="example-target2">
<echo msg="${example-Property3}" />
....
</target>
After:
<property name="example.property1" value="abc" />
<property name="example.property2" value="def" />
<property name="example.property3" value="ghi" />

<target name="example-target1">
<echo msg="${example.property1}" />
....
</target>

<target name="example-target2">
<echo msg="${example.property3}" />
....
</target>
The Introduce distinct target naming refactoring once again tackles the improvement of readability in a build file by applying a constant and different punctuation on the common elements: targets and properties. The appliance of this refactoring leaves you and coworkers with an immediate reply whether you're looking at a property value or a target and can lead towards an agreed upon in-house/project build file coding standard. For target names underscores and dashes are suitable, although dashes are preferred by me as a hypen is used when enforcing internal targets, while for the build properties dots should be considered to build namespaces and their names should 'always' be lowercased except for environment variables/properties.

In case this blog post whetted your appetite for more, heavier and here uncovered build file refactorings, you might consider picking up a copy of the ThoughtWorks Anthology book. Last but not least a shout out goes to the build doctor for the remix permission and until the next post I'm ghost like dog.

Saturday, 29 March 2008

Getting an overview of all targets accumulated in a Phing build file

Looking for an equivalent to Ant's -p(rojecthelp) command-line option in Phing, I dug up that you can use Phing's -l(ist) option to get a quick overview of all targets hosted in a given build file. This comes in handy when you are maintaining build files or have to get a raw picture of the provided targets of a project specific build file. The following console output shows the targets hosted in a example build file. As you will see, by using the -l command-line option you get an overview of the default target, the main targets and further subtargets.

triton:work stolt$ phing -l
Buildfile: /Volumes/USB DISK/work/build.xml
Build file for recordshelf project
Default target:
-------------------------------------------------------------------------------
help Displays the help for this build file

Main targets:
-------------------------------------------------------------------------------
build Builds the project
clean Removes data left over by former build
cs-coding-standard Runs the coding standard code inspection
help Display the help for this build file
init Initializes the build process
layout Creates the project layout for the build
lint-project Lints all code artifacts
test-components Runs the automated component tests
test-integration Runs the automated integration tests
test-units Runs the automated unit tests

Subtargets:
-------------------------------------------------------------------------------
build-initial-model
A target becomes a main target when it has an non-empty description attribute, otherwise it will be a subtarget. This can be used to define the visibility and thereby 'callability' of targets, similar as you would do when defining access specifiers for class members of a PHP class. Main targets would become the public targets of your build file and all subtargets could be considered as private targets, which shouldn't be called directly as they might/should be used to provide specific helper functionality to public main targets or target wrappers. To definitively veto the call to a target, which never should be run directly via the Phing CLI(e.g. phing tragetname), just define an invalid target name(i.e. -build-initial-model instead of build-initial-model) so the Phing CLI will respond with an error and thereby not run the build file. The next console output shows the Phing CLI response when calling a private target.
triton:work stolt$ phing -build-initial-model
Unknown argument: -build-initial-model
phing [options] [target [target2 [target3] ...]]
Options:
-h -help print this message
-l -list list available targets in this project
-v -version print the version information and exit
-q -quiet be extra quiet
-verbose be extra verbose
-debug print debugging information
-logfile <file> use given file for log
-logger <classname> the class which is to perform logging
-f -buildfile <file> use given buildfile
-D<property>=<value> use value for given property
-find <file> search for buildfile towards the root of the
filesystem and use it
-inputhandler <file> the class to use to handle user input

Report bugs to <[email protected]>
This post is somehow a partial and PHP flavoured remix of a post by Julian Simpson, who runs the build doctor blog and is also the author of the 'Refactoring Ant Build Files' contribution to the upcoming ThoughtWorks Anthology book.

Friday, 8 February 2008

PHP_CodeSniffer task is in the current Phing branch

A few days ago I was evaluating the need for writing a Phing task for the awesome PHP_CodeSniffer Pear package and noticed that Dirk Thomas blessedly already added one to the current Phing Svn branch. That's again extending the range of available tools that PHP's Continuous Integration(CI) toolchain has to offer and I quess there are more to follow, as Manuel Pichler just announced the development start of PHP_Depend.

To get the 2.3 Phing branch including the mentioned PHP_CodeSniffer task and it's documentation simply run a svn checkout http://svn.phing.info/branches/2.3 /path/to/phing-checkout-dir and you're ready to add continuous monitoring of coding standard adherence to your builds.

For simplifying the integration of the just checked out Phing 'branch' release into an existing Pear environment you can build a new Pear package by calling the buildfile in /path/to/phing-checkout-dir/pear. The built Pear package will be located in /path/to/phing-checkout-dir/pear/build and can than be used to switch the installed Phing Pear release.

Also make sure you have installed the underlying PHP_CodeSniffer Pear package via the Pear Installer.

Although the task is very well documented the next code snippet shows the shiny PHP_CodeSniffer task in an example build file.

<?xml version="1.0"?>
<project name="example" default="build" basedir=".">

<target name="build" depends="clean, svn-checkout, code-inspection, test">

...

</target>

...

<target name="code-inspection" description="runs code inspection on the stated directory">
<phpcodesniffer standard="PEAR"
format="summary"
tabWidth="4"
file="/path/to/source-files"
allowedFileExtensions="php"/>
</phpcodesniffer>
</target>

</project>

Sunday, 26 August 2007

Setting up Zend Framework applications with Phing

After spending too much time on directory and view script shifting to align an 'older' Zend Framework application to the very useful ViewRenderer Action Helper of the follow-up releases I crafted a single Phing buildfile to stick to the recommended conventions and to have a nearly 'one-button' setup solution for any upcoming projects.

There are already some solutions available ranging from prepacked application skeletons to programmatically creators. Of course this is a rude violation of the DRY principle, but you are welcome to read about the basic features the Phing based solution has to offer for now.

Requirements


Features

  • Building a default or modular application directory structure

  • Generating Controller, Model and View skeletons

  • Php linting/validation of generated skeletons

  • Customizable Controller, Model and bootstrap templates

  • Retrieval of a specific Zend Framework version via svn or get

  • Bootstrap file generation

  • Modular targets to generate and add Models and Controllers skeletons belated


Open/possible improvements

  • Couple Model generation with basic database table creating and seeding

  • Zend_Db, Zend_Registry and Zend_Log setup

  • Alignment to Zend_Application proposal, when it's moved into core


After having all requirements aboard it only comes down to the following two steps:

  1. Run the buildfile in the targeted directory

  2. Set the document root to the applications html directory e.g. by using Apaches VirtualHost capabilities

Although it's tested and I put some thoughts in it, any feedback about detected flaws and additional improvements are highly appreciated.

Friday, 10 August 2007

Using Phing to create and seed databases

Influenced by my current reading of "Continuous Integration, Improving Software Quality and Reducing Risk" by Paul M.Duval et al. and it's chapter about Continuous Database Integration I was curious if Phing provides any out-of-the-box tasks to enable the use of this process step in a Phing based buildfile. And it does via its PDOSQLExecTask which is available in the Phing 2.3.0beta1 release amongst other useful tasks e.g. the SvnCheckoutTask.

This task can be used to automate e.g. the following common and repeating scenarios:


  • Create and seed databases with clean test data prior to (component) tests.

  • Create and seed databases with default/standard data at deployment.

  • ...

The following buildfile shows the PDOSQLExecTask in action. The first target uses all Sql files located in ./test-sql as an input source whereas the second one uses inline Sql statements.
<?xml version="1.0" ?>
<project name="example" basedir="." default="prepare-test-databases-1">

<property name="pdo.driver" value="mysql" />
<property name="db.host" value="localhost" />
<property name="db.name" value="application_test" />

<!-- Create and seed databases from file -->
<target name="prepare-test-databases-1">
<pdo url="${pdo.driver}:host=${db.host};dbname=${db.name}" encoding="utf8"
userId="username" password="userpassword"
onerror="abort">
<fileset dir="test-sql">
<include name="*.sql"/>
</fileset>
</pdo>
</target>

<!-- Create and seed databases inline -->
<target name="prepare-test-databases-2">
<pdo url="${pdo.driver}:host=${db.host};dbname=${db.name}"
userId="username" password="userpassword"
onerror="abort">

DROP DATABASE IF EXISTS `${db.name}`;

CREATE DATABASE `${db.name}`;

USE `${db.name}`;

DROP TABLE IF EXISTS `users`;

CREATE TABLE `users` (`id` int(11) NOT NULL auto_increment COMMENT 'User Id',
`first_name` varchar(128) NOT NULL default '' COMMENT 'User first name',
`last_name` varchar(128) NOT NULL default '' COMMENT 'User last name',
`password` varchar(25) NOT NULL default '' COMMENT 'User password',
PRIMARY KEY (`id`))
ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Users test table';

INSERT INTO `users` (`id`, `first_name`, `last_name`, `password`)
VALUES (0, 'John', 'Doe', '3dHdju47');
INSERT INTO `users` (`id`, `first_name`, `last_name`, `password`)
VALUES (0, 'Jane', 'Doe', 'ZjG300irl');

<!-- Other DDL/DML Sql statements -->

</pdo>
</target>

<!-- Test part of the application -->
<target name="test-application-usermanagement" depends="prepare-test-databases-1">

<!-- PHPUnitTask Setup etc. -->

</target>

</project>

Sunday, 5 August 2007

Just another Phing task

Yesterday I was looking for a similarity to Ant's get task in the Phing build tool and as there was none available it was time to create on. The get task enables the using target to get a file from an Url and comes in handy when you have to add remote files to your build. The upcoming solution doesn't cover all available attributes of it's Ant relative task i.e. no Http authentication and progress output.

This table lists the available attributes of the get task and their possible default values.

NameTypeDescriptionDefaultRequired
srcstringThe source to 'get' fromn/aYes
deststringThe destination for the source filen/aYes
usetimestampstringEnables a conditional download of the source file when newer than the already 'getted' destination file. Available options are true|false.falseNo

The above table leads to the following Phing target layout where the custom get task is pulled in via the <taskdef> element.
<?xml version="1.0" ?>
<project name="example" basedir="." default="retrieve-library">

<!-- Pull in the custom get task -->
<taskdef name="get" classname="phing.tasks.my.GetTask" />

<property name="application.name" value="blogOrganizer"/>

<target name="retrieve-library">
<get src="http://framework.zend.com/images/PoweredBy_ZF_4LightBG.png"
dest="${project.basedir}/${application.name}/library/ZendFramework/PoweredBy_ZF_4LightBG.png"
usetimestamp="true"/>
</target>
</project>
The following code listing shows the internals of the GetTask class and goes to the directory /classes/phing/tasks/my and is also available for download. It utilizes the Php curl extension and provides a 'no curl extension enabled' fallback using only Php's built-in functions. Another uncovered approach for implementing this task would be to utilitize the wget command.
<?php

require_once('phing/Task.php');

class GetTask extends Task {

protected $src = null;
protected $dest = null;
protected $applyTimestampConditional = null;

protected $fpDest = null;

/**
* Initialize the GetTask.
*/
public function init() {
$this->applyTimestampConditional = false;
}
/**
* Set the source.
* @param string $src The source to get.
*/
public function setSrc($src) {
$this->src = $src;
}
/**
* Set the dest to copy/write the src to.
* @param string $src The dest for get src.
*/
public function setDest($dest) {
$this->dest = $dest;
}
/**
* Set conditional for a timestamp comparison.
* @param boolean $usetimestamp The conditional for the timestamp comparison.
*/
public function setUsetimestamp($usetimestamp) {
$this->applyTimestampConditional = $usetimestamp;
}
/**
* Get the src file and copy/write it to given dest.
* @throws BuildException
*/
public function main() {

$this->validate();

if(!file_exists($this->dest) || $this->applyTimestampConditional === false) {

try {

$kiloBytes = $this->getSrcFile();
print(" [get] Got file {$this->src} kB[{$kiloBytes}]\n");

} catch(Exception $e) {

throw new BuildException($e);

}

} else {

try {

if($this->applyTimestampConditional &&
filemtime($this->dest) === $this->getLastModificationForSrcFile($this->src)) {

print(" [get] The timestamps of src and existing dest are unique\n");

} else {

$kiloBytes = $this->getSrcFile();
print(" [get] Got file {$this->src} kB[{$kiloBytes}]\n");

}

} catch(Exception $e) {

throw new BuildException($e);

}
}

}
/**
* Validate that the required attributes have been set.
* @throws BuildException
*/
private function validate() {

if(!isset($this->src) && !isset($this->dest)) {

throw new BuildException("Required attributes 'src' and 'dest' are not set");

}

if(!isset($this->src)) {

throw new BuildException("Required attribute 'src' is not set");

}

if(!isset($this->dest)) {

throw new BuildException("Required attribute 'dest' is not set");

}
if(isset($this->applyTimestampConditional)) {

if($this->applyTimestampConditional !== false && $this->applyTimestampConditional !== true) {
throw new BuildException("Unsupported value for attribute usetimestamp set");
}
}

}
/**
* Get the src file and creates the dest file.
* Utilizes the curl extension first and uses php's default
* built-in funtions as a fallback.
* @return float The kilo bytes received from src.
* @throws Exception
*/
private function getSrcFile() {

if (function_exists('curl_init')) {

try {

$bytesCopied = $this->getSrcFileViaCurl();

} catch(Exception $e) {

throw $e;

}

} else {

try {

$bytesCopied = $this->getSrcFileViaPhpBuiltins();

} catch(Exception $e) {

throw $e;

}

}

try {

$this->touchDest();

} catch(Exception $e) {

//ignore, no rethrow as BuildException

}
return round($bytesCopied/1024, 2);

}
/**
* Get the src file and creates the dest file.
* Uses php's default built-in funtions.
* @return float The bytes received from src.
* @throws Exception
*/
private function getSrcFileViaPhpBuiltins() {

if(ini_get('allow_url_fopen') === '') {

throw new Exception("allow_url_fopen is disabled in php.ini");

}

if($src = @fopen($this->src, 'r')) {

try {

$this->createDestFile();

} catch(Exception $e) {

throw $e;
}

if($bytesCopied = stream_copy_to_stream($src, $this->fpDest)) {

fclose($src);
fclose($this->fpDest);
return $bytesCopied;

} else {

fclose($src);
fclose($this->fpDest);
throw new Exception("Unable to copy data from {$this->src} to {$this->dest}");

}

}
}
/**
* Get the src file and creates the dest file.
* Uses the curl extension.
* @return float The bytes received from src.
* @throws Exception
*/
private function getSrcFileViaCurl() {

if($ch = curl_init($this->src)) {

try {

$this->createDestFile();

} catch(Exception $e) {

throw $e;

}
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_FILE, $this->fpDest);
curl_exec($ch);

if($httpCode = (curl_getinfo($ch, CURLINFO_HTTP_CODE)) !== 200) {

curl_close($ch);
fclose($this->fpDest);
unlink($this->dest);
throw new Exception("Got Http Code {$httpCode} for {$this->src}");

}

$bytesCopied = curl_getinfo($ch, CURLINFO_SIZE_DOWNLOAD);
curl_close($ch);
fclose($this->fpDest);
return $bytesCopied;

} else {

throw new Exception("Unable to init curl with {$this->src}");

}

}
/**
* Touch the dest file.
* @throws Exception
* @see function touch()
*/
private function touchDest() {

try {

$lastModifikationOfSrcFile = $this->getLastModificationForSrcFile();
touch($this->dest, $lastModifikationOfSrcFile);

} catch(Exception $e) {

throw $e;

}
}
/**
* Get the last modification date from src file.
* @return string The timestamp of the src file.
* @throws Exception
* @note Remixed from a php.net filemtime user note.
*/
private function getLastModificationForSrcFile($uri = null) {

$timestamp = 0;

if($fp = @fopen($this->src, "r" )) {

$metaData = stream_get_meta_data($fp);
fclose($fp);

} else {

throw new Exception("Unable to open src {$this->src}");

}

if(!array_key_exists('wrapper_data', $metaData)) {

throw new Exception("Unable to read meta data section for src {$this->src}");

}

foreach($metaData['wrapper_data'] as $response) {

if(substr(strtolower($response), 0, 10) == 'location: ') {
$newUri = substr($response, 10);
return $this->getLastModificationForRemoteFile($newUri);

} elseif(substr(strtolower($response), 0, 15 ) == 'last-modified: ') {

$timestamp = strtotime(substr($response, 15));
return $timestamp;

}
}
}
/**
* Create the dest file.
* @throws Exception
*/
private function createDestFile() {

if($this->fpDest = fopen($this->dest, 'w')) {

return true;

} else {

throw new Exception("Unable to create {$this->dest}");

}

}

}