Showing posts with label Zend Framework. Show all posts
Showing posts with label Zend Framework. Show all posts

Tuesday, 16 March 2010

Using MongoHq in Zend Framework based applications

MongoHq logoAs the name slightly foreshadows MongoHq is a currently bit pricey cloud-based hosting solution for MongoDb databases provided by CommonThread. Since they went live a few weeks ago I signed up for the small plan and started to successfully re-thinker with it in an exploratory Zend Framework based application.

Therefore the following post will show how to bootstrap such an instance into a Zend Framework based application and how to use it from there in some simple scenarios like storing data coming from a Zend_Form into a designated collection and vice versa fetching it from there.

Bootstrapping a MongoHq enabled connection

To establish and make the MongoDb connection application-wide available the almighty Zend_Application component came to the rescue again. After reading Matthew Weier O'Phinney's enlightening blog post about creating re-usable Zend_Application resource plugins and deciding to use MongoDb in some more exploratory projects, I figured it would be best to create such a plugin and ditch the also possible resource method approach.

The next code listing shows a possible implementation of the MongoDb resource plugin initializing a Mongo instance for the given APPLICATION_ENV (i.e. production) mode.

For the other application environment modes (development | testing | staging) it's currently assumed that no database authentication is enabled, which is also the default when using MongoDb, so you might need to adapt the plugin to your differing needs; and since I'm currently only rolling on the small plan the support for multiple databases is also not accounted for.

library/Recordshelf/Resource/MongoDb.php
<?php

class Recordshelf_Resource_MongoDb
extends Zend_Application_Resource_ResourceAbstract
{
/**
* Definable Mongo options.
*
* @var array
*/
protected $_options = array(
'hostname' => '127.0.0.1',
'port' => '27017',
'username' => null,
'password' => null,
'databasename' => null,
'connect' => true
);
/**
* Initalizes a Mongo instance.
*
* @return Mongo
* @throws Zend_Exception
*/
public function init()
{
$options = $this->getOptions();

if (null !== $options['username'] &&
null !== $options['password'] &&
null !== $options['databasename'] &&
'production' === APPLICATION_ENV) {
// Database Dns with MongoHq credentials
$mongoDns = sprintf('mongodb://%s:%s@%s:%s/%s',
$options['username'],
$options['password'],
$options['hostname'],
$options['port'],
$options['databasename']
);
} elseif ('production' !== APPLICATION_ENV) {
$mongoDns = sprintf('mongodb://%s:%s/%s',
$options['hostname'],
$options['port'],
$options['databasename']
);
} else {
$exceptionMessage = sprintf(
'Recource %s is not configured correctly',
__CLASS__
);
throw new Zend_Exception($exceptionMessage);
}
try {
return new Mongo($mongoDns, array('connect' => $options['connect']));
} catch (MongoConnectionException $e) {
throw new Zend_Exception($e->getMessage());
}
}
}
With the MongoDb resource plugin in the place to be, it's time to make it known to the boostrapping mechanism which is done by registering the resource plugin in the application.ini.

Further the MongoHq credentials, which are available in the MongoHq > My Database section, and the main database name are added to the configuration file which will be used to set the definable resource plugin ($_)options and to connect to the hosted database.

application/configs/application.ini

[production]
pluginPaths.Recordshelf_Resource = "Recordshelf/Resource"
resources.mongodb.username = __MONGOHQ_USERNAME__
resources.mongodb.password = __MONGOHQ_PASSWORD__
resources.mongodb.hostname = __MONGOHQ_HOSTNAME__
resources.mongodb.port = __MONGOHQ_PORT__
resources.mongodb.databasename = __MONGOHQ_DATABASENAME__

...

Cloudifying documents into collections

Having the MongoHq enabled connection in the bootstrapping mechanism it can now be picked up from there and used in any Zend Framework application context.

The example action method (i.e. proposeAction) assumes data (i.e. a tech talk proposal to revive the example domain from my last blog post) coming from a Zend_Form which will be stored in a collection named proposals, a table in old relational database think.

The next code listings states the action method innards to do so by injecting the valid form values into a model class which provides accessors and mutators for the domain model's properties and can transform them into a proposal document aka an array structure.

application/controllers/ProposalController.php
<?php

class ProposalController extends Zend_Controller_Action
{
public function indexAction()
{
$this->view->form = new Recordshelf_Form_Proposal();
}
public function thanksAction()
{
}
public function proposeAction()
{
$this->_helper->viewRenderer->setNoRender();
$form = new Recordshelf_Form_Proposal();

$request = $this->getRequest();

if ($this->getRequest()->isPost()) {
if ($form->isValid($request->getPost())) {
$model = new Recordshelf_Model_Proposal($form->getValues());
$mapper = new Recordshelf_Model_ProposalMapper();
if ($mapper->insert($model)) {
return $this->_helper->redirector('thanks');
}
$this->view->form = $form;
return $this->render('index');
} else {
$this->view->form = $form;
return $this->render('index');
}
}
}
}
Next the model/data mappper is initialized, which triggers the picking of the MongoHq enabled Mongo connection instance and the auto-determination of the collection name to use based on the mapper's class name. Subsequently the populated model instance is passed into the mappper's insert method which is pulling the document (array structure) and doing the actual insert into the proposals collection.

To give you an idea of the actual document structure it's shown in the next listing, followed by the model/data mapper implementation.
Array
(
[state] => new
[created] => MongoDate Object
(
[sec] => 1268774242
[usec] => 360831
)

[submitee] => Array
(
[title] => Mr
[firstname] => John
[familyname] => Doe
[email] => [email protected]
[twitter] => johndoe
)

[title] => How to get a real name
[description] => Some descriptive text...
[topictags] => Array
(
[0] => John
[1] => Doe
[2] => Anonymous
)

)


application/models/ProposalMapper.php
<?php

class Recordshelf_Model_ProposalMapper
{
private $_mongo;
private $_collection;
private $_databaseName;
private $_collectionName;

public function __construct()
{
$frontController = Zend_Controller_Front::getInstance();
$this->_mongo = $frontController->getParam('bootstrap')
->getResource('mongoDb');
$config = $frontController->getParam('bootstrap')
->getResource('config');

$this->_databaseName = $config->resources->mongodb->get('databasename');

$replaceableClassNameparts = array(
'recordshelf_model_',
'mapper'
);
$this->_collectionName = str_replace($replaceableClassNameparts, '',
strtolower(__CLASS__) . 's');

$this->_collection = $this->_mongo->selectCollection(
$this->_databaseName,
$this->_collectionName
);
}
/**
* Inserts a proposal document/model into the proposals collection.
*
* @param Recordshelf_Model_Proposal $proposal The proposal document/model.
* @return MongoId
* @throws Zend_Exception
*/
public function insert(Recordshelf_Model_Proposal $proposal)
{
$proposalDocument = $proposal->getValues();
try {
if ($this->_collection->insert($proposalDocument, true)) {
return $proposalDocument['_id'];
}
} catch (MongoCursorException $mce) {
throw new Zend_Exception($mce->getMessage());
}
}
}

Querying and retrieving the cloudified data

As what comes in must come out, the next interaction with the Document Database Management System (DocDBMS) is about retrieving some afore-stored talk proposal documents from the collection so they can be rendered to the application's user. This isn't really MongoHq specific anymore, like most of the previous model parts, and is just here to round up this blog post and use some more of that MongoDb goodness. Looks like I have to look for an anonymous self-help group that stuff is highly addictive.

Anyway the next listing shows the action method fetching all stored documents available in the proposals collection. To save some CO2 on this blog post all documents are fetched, which ends up in the most trivial query but as you can figure the example domain provides a bunch of query examples like only proposals for a given topic tag, specific talk title or a given proposal state which can be easily created via passed-through Http request parameters.

application/controllers/ProposalController.php
<?php

class ProposalController extends Zend_Controller_Action
{
...

public function listAction()
{
$mapper = new Recordshelf_Model_ProposalMapper();
$proposals = $mapper->fetchAll();
// For iterating the Recordshelf_Model_Proposal's in the view
$this->view->proposals = $proposals;
}
}
The last code listing shows the above used fetchAll method of the data mapper class returning an array of stored proposal documents mapped to their domain model (i.e. Recordshelf_Model_Proposal) in the application.

application/models/ProposalMapper.php
<?php

class Recordshelf_Model_ProposalMapper
{
...

/**
* Fetches all stored talk proposals.
*
* @return array
*/
public function fetchAll()
{
$cursor = $this->_collection->find();
$proposals = array();

foreach ($cursor as $documents) {
$proposal = new Recordshelf_Model_Proposal();
foreach ($documents as $property => $value) {
if ('submitee' === $property) {
$proposal->submitee = new Recordshelf_Model_Submitee($value);
} else {
$proposal->$property = $value;
}
}
$proposals[] = $proposal;
}
return $proposals;
}
}

Friday, 5 February 2010

Utilizing Twitter lists with Zend_Service_Twitter

Twitter lists with the Zend FrameworkSeveral months ago Twitter added the list feature to it's public API. While debating some use cases for an event registration application I stumbled upon an interesting feature, which adds participants automatically to a Twitter list upon registration. This way registered and interested users can discover like-minded individuals and get in touch prior to any pre-social event activities. This post will show how this feature can be implemented by utilizing the Zend_Service_Twitter component, and how it then can be used in a Zend Framework based application.

Implementing the common list features

Looking at the three relevant parts of the Twitter list API some common features emerged and had to be supported to get the feature out of the door. These are namely the creation, deletion of new lists and the addition, removal of list members (i.e. event participants). Since the current Twitter component doesn't support these list operations out of the box it was time to put that develeoper hat on and get loose; which was actually a joy due to the elegance of the extended Zend_Service_Twitter component laying all the groundwork.

A non-feature-complete implementation is shown in the next code listing and can alternatively be pulled from GitHub. Currently it only supports the above stated common operations plus the ability to get the lists of a Twitter account and it's associated members; but feel free to fork it or even turn it into an official proposal.
<?php

require_once 'Zend/Service/Twitter.php';
require_once 'Zend/Service/Twitter/Exception.php';

class Recordshelf_Service_Twitter_List extends Zend_Service_Twitter
{
const LIST_MEMBER_LIMIT = 500;
const MAX_LIST_NAME_LENGTH = 25;
const MAX_LIST_DESCRIPTION_LENGTH = 100;

/**
* Initializes the service and adds the list to the method types
* of the parent service class.
*
* @param string $username The Twitter account name.
* @param string $password The Twitter account password.
* @see Zend_Service_Twitter::_methodTypes
*/
public function __construct($username = null, $password = null)
{
parent::__construct($username, $password);
$this->_methodTypes[] = 'list';
}
/**
* Creates a list associated to the current user.
*
* @param string $listname The listname to create.
* @param array $options The options to set whilst creating the list.
* Allows to set the list creation mode (public|private)
* and the list description.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function create($listname, array $options = array())
{
$this->_init();

if ($this->_existsListAlready($listname)) {
$exceptionMessage = 'List with name %s exists already';
$exceptionMessage = sprintf($exceptionMessage, $listname);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}

$_options = array('name' => $this->_validListname($listname));
foreach ($options as $key => $value) {
switch (strtolower($key)) {
case 'mode':
$_options['mode'] = $this->_validMode($value);
break;
case 'description':
$_options['description'] = $this->_validDescription($value);
break;
default:
break;
}
}
$path = '/1/%s/lists.xml';
$path = sprintf($path, $this->getUsername());

$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Deletes an owned list of the current user.
*
* @param string $listname The listname to delete.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function delete($listname)
{
$this->_init();

if (!$this->_isListAssociatedWithUser($listname)) {
$exceptionMessage = 'List %s is not associate with user %s ';
$exceptionMessage = sprintf($exceptionMessage,
$listname,
$this->getUsername()
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
$_options['_method'] = 'DELETE';
$path = '/1/%s/lists/%s.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Adds a member to a list of the current user.
*
* @param integer $userId The numeric user id of the member to add.
* @param string $listname The listname to add the member to.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function addMember($userId, $listname)
{
$this->_init();

if (!$this->_isListAssociatedWithUser($listname)) {
$exceptionMessage = 'List %s is not associate with user %s ';
$exceptionMessage = sprintf($exceptionMessage,
$listname,
$this->getUsername()
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}

$_options['id'] = $this->_validInteger($userId);
$path = '/1/%s/%s/members.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);

if ($this->_isListMemberLimitReached($listname)) {
$exceptionMessage = 'List can contain no more than %d members';
$exceptionMessage = sprintf($exceptionMessage,
self::LIST_MEMBER_LIMIT
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}

$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Removes a member from a list of the current user.
*
* @param integer $userId The numeric user id of the member to remove.
* @param string $listname The listname to remove the member from.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function removeMember($userId, $listname)
{
$this->_init();

if (!$this->_isListAssociatedWithUser($listname)) {
$exceptionMessage = 'List %s is not associate with user %s ';
$exceptionMessage = sprintf($exceptionMessage,
$listname,
$this->getUsername()
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}

$_options['_method'] = 'DELETE';
$_options['id'] = $this->_validInteger($userId);
$path = '/1/%s/%s/members.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Fetches the list members of the current user.
*
* @param string $listname The listname to fetch members from.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function getMembers($listname) {
$this->_init();
$path = '/1/%s/%s/members.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
$response = $this->_get($path);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Fetches the list of the current user or any given user.
*
* @param string $username The username of the list owner.
* @return Zend_Rest_Client_Result
*/
public function getLists($username = null)
{
$this->_init();
$path = '/1/%s/lists.xml';
if (is_null($username)) {
$path = sprintf($path, $this->getUsername());
} else {
$path = sprintf($path, $username);
}
$response = $this->_get($path);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Checks if the list exists already to avoid number
* indexed recreations.
*
* @param string $listname The list name.
* @return boolean
* @throws Zend_Service_Twitter_Exception
*/
private function _existsListAlready($listname)
{
$_listname = $this->_validListname($listname);
$lists = $this->getLists();
$_lists = $lists->lists;
foreach ($_lists->list as $list) {
if ($list->name == $_listname) {
return true;
}
}
return false;
}
/**
* Checks if the list is associated with the current user.
*
* @param string $listname The list name.
* @return boolean
*/
private function _isListAssociatedWithUser($listname)
{
return $this->_existsListAlready($listname);
}
/**
* Checks if the list member limit is reached.
*
* @param string $listname The list name.
* @return boolean
*/
private function _isListMemberLimitReached($listname)
{
$members = $this->getMembers($listname);
return self::LIST_MEMBER_LIMIT < count($members->users->user);
}
/**
* Returns the list creation mode or returns the private mode when invalid.
* Valid values are private or public.
*
* @param string $creationMode The list creation mode.
* @return string
*/
private function _validMode($creationMode)
{
if (in_array($creationMode, array('private', 'public'))) {
return $creationMode;
}
return 'private';
}
/**
* Returns the list name or throws an Exception when invalid.
*
* @param string $listname The list name.
* @return string
* @throws Zend_Service_Twitter_Exception
*/
private function _validListname($listname)
{
$len = iconv_strlen(trim($listname), 'UTF-8');
if (0 == $len) {
$exceptionMessage = 'List name must contain at least one character';
throw new Zend_Service_Twitter_Exception($exceptionMessage);
} elseif (self::MAX_LIST_NAME_LENGTH < $len) {
$exceptionMessage = 'List name must contain no more than %d characters';
$exceptionMessage = sprintf($exceptionMessage,
self::MAX_LIST_NAME_LENGTH
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
return trim($listname);
}
/**
* Returns the list description or throws an Exception when invalid.
*
* @param string $description The list description.
* @return string
* @throws Zend_Service_Twitter_Exception
*/
private function _validDescription($description)
{
$len = iconv_strlen(trim($description), 'UTF-8');
if (0 == $len) {
return '';
} elseif (self::MAX_LIST_DESCRIPTION_LENGTH < $len) {
$exceptionMessage = 'List description must contain no more than %d characters';
$exceptionMessage = sprintf($exceptionMessage,
self::MAX_LIST_DESCRIPTION_LENGTH
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
return trim(strip_tags($description));
}
}

Adding the 'auto list' feature

For using the above implemented add member feature it's assumed that a participant has provided a valid and existing Twitter username, his approval of being added to the event list (i.e. zfweekend) and that he further has been registered effectively. To have the name of the Twitter list to act on and the account credentials available corresponding configuration entries are set as shown next.

application/configs/application.ini

[production]
twitter.username = __USERNAME__
twitter.password = __PASSWORD__
twitter.auto.listname = zfweekend
With the Twitter credentials and the list name available it's now possible to pull this feature into the register method of the register Action Controller, where it's applied as shown in the outro listing. As you will see, besides some bad practices due to demonstration purposes, the register Form makes use of a custom TwitterScreenName validator and filter which are also available via GitHub. Happy Twitter listing!
<?php

class RegisterController extends Zend_Controller_Action
{
/**
* @badpractice Push this into a specific Form class.
* @return Zend_Form
*/
private function _getForm()
{
$form = new Zend_Form();
$form->setAction('/register/register')
->setMethod('post');
$twitterScreenName = $form->createElement('text', 'twitter_screen_name',
array('label' => 'Twittername: ')
);
$twitterScreenName->addValidator(new Recordshelf_Validate_TwitterScreenName())
->setRequired(true)
->setAllowEmpty(false)
->addFilter(new Recordshelf_Filter_TwitterScreenName());

$autoListApproval = $form->createElement('checkbox', 'auto_list_approval',
array('label' => 'I approved to be added to the event Twitter list: ')
);

$form->addElement($twitterScreenName)
->addElement($autoListApproval)
->addElement('submit', 'register', array('label' => ' Register '));

return $form;
}
public function indexAction()
{
$this->view->form = $this->_getForm();
}
public function thanksAction()
{
}
/**
* @badpractice Handle possible Exception of
* Recordshelf_Service_Twitter_List::addMember.
* @return Recordshelf_Service_Twitter_List
*/
public function registerAction()
{
$this->_helper->viewRenderer->setNoRender();
$form = $this->_getForm();
$request = $this->getRequest();

if ($this->getRequest()->isPost()) {
if ($form->isValid($request->getPost())) {

$model = new Recordshelf_Model_Participant($form->getValues());
$model->save();

if ($form->getElement('auto_list_approval')->isChecked()) {
$twitterScreenName = $form->getValue('twitter_screen_name');
$twitter = $this->_getTwitterListService();
$response = $twitter->user->show($twitterScreenName);
$userId = (string) $response->id;
$response = $twitter->list->addMember($userId,
$this->_getTwitterListName());

$model->hasBeenAddedToTwitterList(true);
$model->update();

return $this->_helper->redirector('thanks');
}
} else {
return $this->_helper->redirector('index');
}
}
}
/**
* @badpractice Push this into a dedicated Helper or something similar.
* @return Recordshelf_Service_Twitter_List
*/
private function _getTwitterListService()
{
$config = Zend_Registry::get('config');
return new Recordshelf_Service_Twitter_List(
$config->twitter->get('username'),
$config->twitter->get('password')
);
}
/**
* @badpractice Push this into a dedicated Helper or something similar.
* @return string
*/
private function _getTwitterListName()
{
$config = Zend_Registry::get('config');
return $config->twitter->auto->get('listname');
}
}


Wednesday, 14 October 2009

Zend Framework 1.8 Web Application Development book review

Zend Framework 1.8 Web Application DevelopmentAs the days are rapidly getting shorter, my reading appetite grows potentially and this evening I finished the 'Zend Framework 1.8 Web Application Development' book written by Keith Pope. While Keith worked on the book, I peeked several times at it's tutorial application, dubbed the Storefront, to get me going with the new Zend_Application component. Looking at it's code made me feel certain to get another great digest of the new features and components of version 1.8, and also a different practical perspective on web application development with the Zend Framework, once the book has been published. Therefor I got in touch with the publisher Packt and fortunately got a copy of which I'd like to share a personal review in this blog post.

What's in it?

The book opens with a quick run-through of the Model-View-Controller (MVC) architecture by creating a project structure via Zend_Tool and building a first very basic web application. While this introduction intentionally skips over a lot of details, the following chapter provides very detailed insights into the Zend Framework's MVC components by explaining the surrounded objects, the Design Patterns they are based upon and their interactions.

After laying out that hefty block of theory the aforementioned tutorial application is introduced and built incrementally over several chapters; each one going into more detail for the specific application aspect. The highlight content of these chapters reach from introducing the Fat Model Skinny Controller concept, thoughts on Model design strategies which are reflected in a custom Storefront Model design, to developing application specific Front Controller Plugins, Action-Helpers, and View-Helpers. The application walk-through is completed by looking at general techniques to optimize the Storefront application and by building an automated PHPUnit Test Suite of functional tests utilizing Zend_Test to keep the Zend Framework based application self-reliant and refactorable.

Conclusion

The book by Keith Pope provides any interested PHP developer, who's not already sold on a specific framework, a thorough introduction to the vivid Zend Framework and it's use in a MVC based web application development context. The content of the book is delivered in a fluent, very enthusiastic and 'knowledge-pillowed' writing tone. By implementing or working through the Storefront application seasoned web developers using older versions of the Framework will get a good blue sheet on new components like Zend_Application and it's implication in the bootstrapping process; while new developers tending towards picking up the Zend Framework will get a current and well compiled guide, which might first start off with a steep learning-curve but will turn into profund knowledge once hanging in there.

The only thing that seemed a bit odd to me, was the utilization of Ant instead of Phing as the build tool for the Storefront application to set the application environment, to remove all require_once statements from the framework library and to run the PHPUnit Test Suite; but this might also be inflicted by my Phing nuttiness.

Saturday, 19 September 2009

Logging to MongoDb and accessing log collections with Zend_Tool

Influenced by a recent blog post of a colleague of mine and by being kind of broke on a Saturday night; I tinkered with the just recently discovered MongoDb and hooked it into the Zend_Log environment by creating a dedicated Zend_Log_Writer. The following post will therefore present a peek at a prototypesque implementation of this writer and show how the afterwards accumulated log entries can be accessed and filtered with a custom Zend_Tool project provider.

Logging to a MongoDb database

The following steps assume that an instance of a MongoDb server is running and that the required PHP MongoDb module is also installed and loaded. To by-pass log entries to a MongoDb database there is a need to craft a proper Zend_Log_Writer. This can be achieved by extending the Zend_Log_Writer_Abstract class, injecting a Mongo connection instance and implementing the actual write functionality as shown in the next listing.
<?php
require_once 'Zend/Log/Writer/Abstract.php';

class Recordshelf_Log_Writer_MongoDb extends Zend_Log_Writer_Abstract
{
    private $_db;
    private $_connection;

   /**
    * @param Mongo $connection The MongoDb database connection
    * @param string $db The MongoDb database name
    * @param string $collection The collection name string the log entries 
    */
    public function __construct(Mongo $connection, $db, $collection)
    {
        $this->_connection = $connection;
        $this->_db = $this->_connection->selectDB($db)->createCollection(
            $collection
        );
    }
    public function setFormatter($formatter)
    {
        require_once 'Zend/Log/Exception.php';
        throw new Zend_Log_Exception(get_class() . ' does not support formatting');
    }
    public function shutdown()
    {
        $this->_db = null;
        $this->_connection->close();
    }
    protected function _write($event)
    {
        $this->_db->insert($event);
    }
   /**
    * Create a new instance of Recordshelf_Log_Writer_MongoDb
    * 
    * @param  array|Zen_Config $config
    * @return Recordshelf_Log_Writer_MongoDb
    * @throws Zend_Log_Exception
    * @since  Factory Interface available since release 1.10.0
    */
    static public function factory($config) 
    {
        $exceptionMessage = 'Recordshelf_Log_Writer_MongoDb does not currently '
            . 'implement a factory';
        throw new Zend_Exception($exceptionMessage);
    }
}
With the MongoDb writer available and added to the library directory of the application it's now possible to utilize this new storage backend as usual with the known Zend_Log component. The Mongo connection injected into the writer is configured via Zend_Config and initialized via the Zend_Application bootstrapping facility as shown in the listings below.

application/configs/application.ini
[production]
app.name = recordshelf

....

log.mongodb.db = zf_mongo
log.mongodb.collection = recordshelf_log
log.mongodb.server = localhost
log.priority = Zend_Log::CRIT

....
application/Bootstrap.php
<?php

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    protected $_logger;

    protected function _initConfig()
    {
        Zend_Registry::set('config', new Zend_Config($this->getOptions()));
    }

    protected function _initLogger()
    {
        $this->bootstrap(array('frontController', 'config'));
        $config = Zend_Registry::get('config');     

        $applicationName = $config->app->get('name', 'recordshelf');
        $mongoDbServer = $config->log->mongodb->get('server', '127.0.0.1');
        $mongoDbName = $config->log->mongodb->get('db', "{$applicationName}_logs");
        $mongoDbCollection = $config->log->mongodb->get('collection', 'entries');

        $logger = new Zend_Log();
        $writer = new Recordshelf_Log_Writer_MongoDb(new Mongo($mongoDbServer), 
        $mongoDbName, $mongoDbCollection);

        if ('production' === $this->getEnvironment()) {
            $priority = constant($config->log->get('priority', Zend_Log::CRIT));
            $filter = new Zend_Log_Filter_Priority($priority);
            $logger->addFilter($filter);
        }
        $logger->addWriter($writer);
        $this->_logger = $logger;
        Zend_Registry::set('log', $logger);
    }
}
controllers/ExampleController.php
<?php

class ExampleController extends Zend_Controller_Action
{
    private $_logger = null;

    public function init()
    {
        $this->_logger = Zend_Registry::get('log');
    }

    public function fooAction()
    {
        $this->_logger->log('A debug log message from within action ' . 
            $this->getRequest()->getActionName(), Zend_Log::DEBUG);
    }

    public function barAction()
    {
        $this->_logger->log('A debug log message from within ' . 
            __METHOD__, Zend_Log::DEBUG);
    }
}

Accessing the log database with a Zend_Tool project provider

After handling the application-wide logging with the MongoDb writer sooner or later the issue to access the gathered log entries will rise. For this mundane and recurring use case the ProjectProvider provider of the Zend_Tool framework is an acceptable candidate to hook a custom action into the Zend_Tool environment of a given project. Therefor a new Zend_Tool_Project Project provider is first scaffolded via the forthcoming command.
sudo zf create project-provider mongodb-logs filter
Second the generated provider skeleton its filter action is enliven with the logic to query the MongoDb database and the stored log collection. The action to come accepts three arguments to filter the stored log entry results by a specific date in the format of 'YYYY-MM-DD' and a given Zend_Log priority (currently limited to the constants defined in Zend_Log) in a specific application environment. The next listing shows the implementation of the import action of the MongodbLogsProvider project provider; which is clearly, as it's length indicates, in need for a clean-up task.

providers/Mongodb-logsProvider.php
<?php
require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';
require_once 'Zend/Date.php';
require_once 'Zend/Validate/Date.php';
require_once 'Zend/Log.php';
require_once 'Zend/Config/Ini.php';

class MongodbLogsProvider extends Zend_Tool_Project_Provider_Abstract
{

    public function filter($date = null, $logPriority = null, 
        $env = 'development')
    {
        $ref = new Zend_Reflection_Class('Zend_Log');
        $logPriorities = $ref->getConstants();

        if (in_array(strtoupper($date), array_keys($logPriorities)) || 
            in_array(strtoupper($date), array_values($logPriorities))) {
            $logPriority = $date;
            $date = null;
        }
        if (!is_null($date)) {
            $validator = new Zend_Validate_Date();
            if (!$validator->isValid($date)) {
                $exceptionMessage = "Given date '{$date}' is not a valid date.";
                throw new Zend_Tool_Project_Provider_Exception($exceptionMessage);
            }
            $dateArray = array();
            list($dateArray['year'], $dateArray['month'], $dateArray['day']) = 
                explode('-', $date);
            $date = new Zend_Date($dateArray);
        } else {
            $date = new Zend_Date();
        }
        $date = $date->toString('Y-MM-dd');

        if (!is_null($logPriority)) {
            if (!is_numeric($logPriority)) {
                $logPriority = strtoupper($logPriority);
                if (!in_array($logPriority, array_keys($logPriorities))) {
                    $exceptionMessage = "Given priority '{$logPriority}' is not defined.";
                    throw new Zend_Tool_Project_Provider_Exception($exceptionMessage);
                } else {
                    $logPriority = $logPriorities[$logPriority];
                }
            }
        if (!in_array($logPriority, array_values($logPriorities))) {
            $exceptionMessage = "Given priority '{$logPriority}' is not defined.";
            throw new Zend_Tool_Project_Provider_Exception();
        }
            $priorities = array_flip($logPriorities);
            $priorityName = $priorities[$logPriority];
        }

        if ($env !== 'development' && $env !== 'production') {
            $exceptionMessage = "Unsupported environment '{$env}' provided.";
            throw new Zend_Tool_Project_Provider_Exception();
        }
        $config = new Zend_Config_Ini('./application/configs/application.ini', 
            $env);

        $applicationName = $config->app->get('name', 'recordshelf');
        $mongoDbServer = $config->log->mongodb->get('server', '127.0.0.1');
        $mongoDbName = $config->log->mongodb->get('db', "{$applicationName}_logs");
        $mongoDbCollection = $config->log->mongodb->get('collection', 'entries');

        try {
            $connection = new Mongo($mongoDbServer);
            $db = $connection->selectDB($mongoDbName)->createCollection(
            $mongoDbCollection);
        } catch (MongoConnectionException $e) {
            throw new Zend_Tool_Project_Provider_Exception($e->getMessage());
        }
        $dateRegex = new MongoRegex("/$date.*/i");

        if (is_null($logPriority)) {
            $query = array('timestamp' => $dateRegex);
            $appendContentForResults = "Found #amountOfEntries# log entrie(s) "
                . "on {$date}";
            $appendContentForNoResults = "Found no log entries on {$date}";
        } else {            
            $query = array('priority' => (int) $logPriority, 
                           'timestamp' => $dateRegex
                     );
            $appendContentForResults = "Found #amountOfEntries# log entrie(s) "
                . "for priority {$priorityName} on {$date}";
            $appendContentForNoResults = "Found no log entries for priority "
                . "{$priorityName} on {$date}";
        }

        $cursor = $db->find($query);
        $amountOfEntries = $cursor->count();

        if ($amountOfEntries > 0) {
            $content = str_replace('#amountOfEntries#', $amountOfEntries, 
                $appendContentForResults);
            $this->_registry->getResponse()->appendContent($content);
            foreach ($cursor as $id => $value) {
                $content = "{$id}: {$value['timestamp']} > ";
                if (is_null($logPriority)) {
                    $content.= "[{$value['priorityName']}] ";
                }
                $content.= "{$value['message']}";
                $this->_registry->getResponse()->appendContent($content);
            }
        } else {
            $content = $appendContentForNoResults;
            $this->_registry->getResponse()->appendContent($content);
        }
        $connection->close();
    }
}
The coming outro screenshots show two use cases for the filter action of the MongodbLogsProvider issued against the zf command line client. The first screenshot shows the use case where all log entries for the current day are queried, while the second one shows the use case where all log entries for a specific date and log priority are queried and fed back to the user.

All log entries of the current day

All CRIT log entries for a specific date

Saturday, 4 July 2009

Scaffolding, implementing and using project specific Zend_Tool_Project_Providers

Working on a project involving several legacy data migration tasks, I got curious what the Zend_Tool_Project component of the Zend Framework offers to create project specific providers for the above mentioned tasks or ones of similar nature. Therefore the following post will try to show how these providers can be developed in an iterative manner by scaffolding them via the capabilities of the Zend_Tool_Project ProjectProvider provider, enlived with action/task logic, and be used in the project scope.

Scaffolding project specific providers

All following steps assume there is a project available i.e. recordshelf initially created with the Zend_Tool_Project Project provider and that the forthcoming commands are issued from the project root directory against the zf command line client. The scaffolding of a project specific provider can be triggered via the create action of the ProjectProvider provider by passing in the name of the provider i.e. csv and it's intended actions. As the next console snippet shows it's
possible to specify several actions as a comma separated list.
sudo zf create project-provider csv importSpecials,importSummersale
After running the command the project's profile .zfproject.xml has been modified and a new providers directory exists in the project root directory containing the scaffolded Csv provider. The next code snippet shows the initial Csv provider class skeleton and its two empty action methods named importSpecials and importSummersale. At the point of this writing, using the Zend Framework 1.8.4 and PHP 5.2.10 on a Mac OS X system the generated Csv provider code or the mapping in the .zfproject.xml is incorrect, but can be fixed by renaming the class from CsvProvider to Csv.
<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class CsvProvider extends Zend_Tool_Project_Provider_Abstract
{

public function importSpecials()
{
/** @todo Implementation */
}

public function importSummersale()
{
/** @todo Implementation */
}


}

Implementing the action logic

Having the project provider class skeleton ready to get going, it's time to enliven the actions with their intended features by using either other components of the Zend Framework, any suitable third party library or plain-vanilla PHP. For the sake of brevity I decided to implement only the importSpecials action which transforms the data of a known CSV file structure into a relevant database table. The CSV parsing steps shown next might not be that sophisticated, as their sole purpose is to illustrate an exemplary implementation of a project specific provider action.
<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class Csv extends Zend_Tool_Project_Provider_Abstract
{
private function _isProjectProviderSupportedInProject(Zend_Tool_Project_Profile $profile,
$projectProviderName)
{
$projectProviderResource = $this->_getProjectProfileResource($profile,
$projectProviderName);
return $projectProviderResource instanceof Zend_Tool_Project_Profile_Resource;
}

private function _isActionSupportedByProjectProvider(Zend_Tool_Project_Profile $profile,
$projectProviderName, $actionName)
{
$projectProviderResource = $this->_getProjectProfileResource($profile,
$projectProviderName);
$projectProviderAttributes = $projectProviderResource->getContext()
->getPersistentAttributes();
return in_array($actionName, explode(',', $projectProviderAttributes['actionNames']));
}

private function _getProjectProfileResource(Zend_Tool_Project_Profile $profile,
$projectProviderName)
{
$profileSearchParams[] = 'ProjectProvidersDirectory';
$profileSearchParams['ProjectProviderFile'] =
array('projectProviderName' => strtolower($projectProviderName));
return $profile->search($profileSearchParams);
}

public function importSpecials($csvFile, $env = 'development')
{
$relatedTablename = 'specials';

if (!$this->_isProjectProviderSupportedInProject($profile, __CLASS__)) {
throw new Exception("ProjectProvider Csv is not supported in this project.");
}
if (!$this->_isActionSupportedByProjectProvider($profile, __CLASS__, __FUNCTION__)) {
$exceptionMessage = "Action 'importSpecials' is not supported by "
. "the Csv ProjectProvider in this project.";
throw new Exception($exceptionMessage);
}

if (!file_exists($csvFile)) {
throw new Exception("Given csv-file '{$csvFile}' doesn't exist.");
}

$importEnvironment = trim($env);
if ($importEnvironment !== 'development' && $importEnvironment !== 'production') {
throw new Exception("Unsupported environment '{$importEnvironment}' provided.");
}

$csvHandle = fopen($csvFile, "r");

if (!$csvHandle) {
throw new Exception("Unable to open given csv-file '{$csvFile}'.");
}

$config = new Zend_Config_Ini('./application/configs/application.ini',
$importEnvironment);
$db = Zend_Db::factory($config->database);

$db->query("TRUNCATE TABLE {$relatedTablename}");
echo "Truncated the project '{$relatedTablename}' database table." . PHP_EOL;

$rowCount = $insertCount = 0;

while (($csvLine = fgetcsv($csvHandle)) !== false) {
if ($rowCount > 0) {
$insertRow = array(
'product_name' => $csvLine[0],
'product_image_path' => $csvLine[1],
'price' => $csvLine[2],
'special_until' => $csvLine[3]
);
$db->insert($relatedTablename, $insertRow);
++$insertCount;
}
++$rowCount;
}
fclose($csvHandle);
$importMessage = "Imported {$insertCount} rows into the project "
. "'{$relatedTablename}' database table.";
echo $importMessage;
}

...
}

Making providers and actions pretendable

To make project specific providers its actions pretendable and thereby providing some kind of user documentation the provider classes have to implement a marker interface called Zend_Tool_Framework_Provider_Pretendable. For making a action of a provider pretendable and giving some feedback to the user, the request is checked if the action has been issued in the pretend mode; which is possible by adding -p option to the issued zf command line client command. The next code snippet shows how the above stated Csv provider and its importSpecials action is made pretendable.
<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class Csv extends Zend_Tool_Project_Provider_Abstract implements
Zend_Tool_Framework_Provider_Pretendable

{

public function importSpecials($csvFile, $env = 'development')
{
...

if ($this->_registry->getRequest()->isPretend()) {
$pretendMessage = "I would import the specials data provided in {$csvFile} "
. "into the project '{$relatedTablename}' database table.";
echo $pretendMessage;
} else {
...
}

}
...
}

Using project specific providers

To use the bundled up capabilities of project specific providers, these have to made accessable to the zf command line client by putting them in the include_path. Currently I discovered no best practice for doing so only for single project scopes and simply added the path to the project to my php.ini and thereby global include_path; another approach might be to add the project name as a prefix to the Provider. After doing so it's possible to get an overview of all with the Zend_Tool_Project shipped providers plus the project specific providers and their offered actions by issuing the zf --help command as shown in the next screenshot. To ensure that project specific providers and its actions are only runnable in projects which support them, it is necessary to check if these and the offered action exists as resources in the project its profile .zfproject.xml file as shown in the implementation of the importSpecials action in one of above code snippets.

Provider overview

As shown in the previous screenshot the first character of the project specific providers are omitted, this is another minor bug which might be fixed in one of the forthcoming Zend Framework releases. The current workaround for this issue is simply to type the command exactly as shown in the help. The outro screenshot shows how the import-specials action of the project specific Csv provider is issued against the zf command line client and its provided user feedback after an successfull import against the projects development database.

Calling the import-specials action

Friday, 23 January 2009

Installing Zend_Tool on Mac OS X

Yesterday I decided to tiptoe into the development of custom Zend_Tool Providers as the introductional article series by Ralph Schindler motivated me to learn more about it and I already have some useful use cases on my mind. Therefor I prior had to install the Zend_Tool component and it's driving CLI scripts on my MacBook. The following brief instruction describes a possible approach that got me running in no time on a Mac OS X system. Once the Zend Framework has an official PEAR channel most of the forthcoming steps should be obsolete and entirely performed by the PEAR package installer command.

Fetching and installing the Zend_Tool component

First I tried to install the 1.8.0(devel) version of the Zend Framework via the pear.zfcampus.org PEAR channel but it currently only delivers the 1.7.3PL1(stable) package; even after switching the stability state of the PEAR config. To dodge the include_path setting hassle and for a further use when customizing other tools like Phing tasks I decided to keep the installed package.
sudo pear channel-discover pear.zfcampus.org
sudo pear install zfcampus/zf-devel
The next commands are showing the footwork I had to do to get the Zend_Tool component into the PEAR Zend Framework package installed in /opt/local/lib/php/Zend.
sudo svn co http://framework.zend.com/svn/framework/standard/incubator/library/Zend/Tool/ $HOME/Cos/Zend/Tool
sudo rsync -r --exclude=.svn $HOME/Cos/Zend/Tool /opt/local/lib/php/Zend

Putting the Zend_Tool CLI scripts to work

The next steps were to fetch the CLI scripts from the public Subversion repository and to link them into the system path /opt/local/bin as shown in the next commands.
sudo svn co http://framework.zend.com/svn/framework/standard/incubator/bin $HOME/Cos/Zend/bin
sudo ln $HOME/Cos/Zend/bin/zf.sh /opt/local/bin/zf
sudo ln $HOME/Cos/Zend/bin/zf.php /opt/local/bin/zf.php

Checking the installation

With everything hopefully in place it was time to verify the success of the installation via the below stated provider action call; and as I got the version of the installed Zend Framework as a response of the executed action/command I'm good to go.
zf show version

Friday, 31 October 2008

Tinyizing URLs with Zend_Http_Client

While doing some initial research for a blog related automation task to implement I learned some more about services which transform long URLs into short ones. The well-knownst of these services, due to the Twitter hype, is probably TinyURL which can be accessed via a classic webinterface or by calling a public API. In a recent blog post Dave Marshall outlined a quick workaround for tweeting via the Zend_Http_Client component which is a reasonable approach for calling services that aren't in the Zend Framework core yet like Zend_Service_Twitter or are not supported out of the box. Therefore this post will try to describe a Zend Framework way of creating tinyized URLs.

Getting tiny tiny y'all

According to Wikipedia there are numerous services available e.g. RubyUrl providing the same feature as TinyURL, so to be prepared for the future and thereby maybe violating the YAGNI principle I decided to declare a very basic interface first in case of switching the service provider someday.
<?php
/**
* 'Interface-level' PHPDoc Block
*/
interface Recordshelf_Service_UrlShortener_Interface
{
public function __construct($serviceEndpoint = '');
public function shortenize($url);
}
The next code snippet shows the implementation for the TinyURL service programmed against the interface and hosting an additional alias method called tinyize which is simply wrapping the actual worker method. The service utilizes Zend_Http_Client by setting the endpoint of the service, transmitting a GET request parameterized with the URL to shorten against it and returning the response containing the tinyized URL.
<?php
require_once('Zend/Http/Client.php');
require_once('Recordshelf/Service/UrlShortener/Interface.php');
/**
* 'Class-level' PHPDoc Block
*/
class Recordshelf_Service_TinyUrl implements
Recordshelf_Service_UrlShortener_Interface
{
/**
* The service endpoint
*
* @var string
*/
private $_serviceEndpoint = null;
/**
* Recordshelf service tinyURL constructor
*
* @param string $serviceEndpoint
*/
public function __construct(
$serviceEndpoint = 'http://tinyurl.com/api-create.php')
{
$this->_serviceEndpoint = $serviceEndpoint;
}
/**
* Shortenizes a given Url
*
* @param string $url
* @return string
* @throws Exception
*/
public function shortenize($url) {
if (is_null($this->_serviceEndpoint)) {
throw new Exception('No service endpoint set');
}
$client = new Zend_Http_Client($this->_serviceEndpoint);
$client->setParameterGet('url', $url)
->setMethod(Zend_Http_Client::GET);
try {
$response = $client->request();
} catch (Exception $e) {
throw $e;
}

if (200 === $response->getStatus()) {
return $response->getBody();
} else {
throw new Exception($response->getStatus() . ": " .
$response->getMessage());
}
}
/**
* Alias method for the shortenize method
*
* @param string $url
* @throws Exception
* @see shortenize
*/
public function tinyize($url)
{
return $this->shortenize($url);
}
}
Now with everything hopefully operating smoothly it's time for a test-drive, yeah I'm lazy and cut that development approach called TDD, by creating a service instance and requesting a TinyURL for the Zend Framework website as shown in the outro listing.
<?php
$service = new Recordshelf_Service_TinyUrl();
$service->tinyize('http://framework.zend.com');
// => http://tinyurl.com/nf8kf
In case off considering or favouring a more framework independent approach there are also other blends available like one via file_get_contents or via curl. Happy tinyizing!

Thursday, 16 October 2008

Scraping websites with Zend_Dom_Query

Today I stumbled upon an interesting and reportable scenario where I had to extract information of the weekly published Drum and Bass charts provided by BBC 1Xtra. As this information currently isn't available in any consumer friendly format like for example a RSS feed, I had to go that scraping route but didn't want to hustle with a regex approach. Since version 1.6.0 the Zend_Dom_Query component has been added to the framework mainly to support functional testing of MVC applications, but it also can be used for rolling custom website scrapers in a snap. Woot, perfect match!

The following code snippets are showing the Bbc_DnbCharts_Scraper class I came up with and an example of its usage. The class utilizes curl to read the website holding the desired data, which will be passed to Zend_Dom_Query to execute queries upon it. For querying the former loaded XHTML Document Object Model it's possible to either utilize XPath or CSS selectors. So I had to pick my poison, and decided to go with the CSS selectors as them were best suited for the document to query and will be more familiar to most jQuery or Prototype users. The query returns a result set of all matching DOMElements which are further unpuzzled via a private helper method returning just the desired charts data as shown in the closing listing. As you can see the implementation of the scraping can be done with a minimum of effort and these are exactly the moments I love the Zend Framework for.

<?php
require_once('Zend/Dom/Query.php');
/**
* 'Class-level' PHPDoc Block
*/
class Bbc_DnbCharts_Scraper
{
private $_url = null;
private $_xhtml = null;

/**
* @param string $url
*/
public function __construct($url)
{
$this->_url = $url;
}
/**
* Scrapes off the drum and bass charts content from the BBC 1Xtra website.
*
* @return array
* @throws Exception
*/
public function scrape()
{
try {
$dom = new Zend_Dom_Query($this->_getXhtml());
} catch (Exception $e) {
throw $e;
}
$results = $dom->query('div.chart div');
$chartDetails = array();
foreach ($results as $index => $result) {
/* @var $result DOMElement */
if ($result->nodeValue !== '') { //filter out <br /> element
$chartDetails[] = $result->nodeValue;
}
}
return $this->_unpuzzleChartDetails($chartDetails, true);
}
/**
* Unpuzzles the chart details and groups them by their chart position,
* if desired with associative keys.
*
* @param array $details
* @param boolean $associative
* @return array
*/
private function _unpuzzleChartDetails(array $details, $associative = false)
{
if (0 === count($details)) {
return array();
} else {
$nextChartRank = 2;
$charts = array();
$groupedChartDetails = array();

foreach ($details as $index => $chartDetail)
{
if ($index <= $nextChartRank) {
$groupedChartDetails[] = $chartDetail;
}
if ($index == $nextChartRank) {
$nextChartRank+=3;
$charts[] = $groupedChartDetails;
unset($groupedChartDetails);
}
}
if ($associative) {
$associatives = array('artist', 'tune', 'label');
foreach ($charts as $chartsIndex => $chart) {
unset($charts[$chartsIndex]);
foreach ($chart as $chartIndex => $chartDetails) {
$charts[$chartsIndex][$associatives[$chartIndex]] =
$chartDetails;
}
}
}
return $charts;
}
}
/**
* Gets the XHTML document via curl
*
* @return string
* @throws Exception
*/
private function _getXhtml()
{
$curl = curl_init();
if (!$curl) {
throw new Exception('Unable to init curl. ' . curl_error($curl));
}
curl_setopt($curl, CURLOPT_URL, $this->_url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
// Faking user agent
curl_setopt($curl, CURLOPT_USERAGENT,
'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');
$xhtml = curl_exec($curl);
if (!$xhtml) {
throw new Exception('Unable to read XHTML. ' . curl_error($curl));
}
curl_close($curl);
return $xhtml;
}
}

// Usage demo
$scraper = new Bbc_DnbCharts_Scraper('http://www.bbc.co.uk/1xtra/drumbass/chart/');
$charts = $scraper->scrape();
The closing code snippet shows an extract of the Drum and Bass charts from BBC 1Xtra scraped off around the 16th October 2008.
Array
(
[0] => Array
(
[artist] => Chase & Status Ft Plan B
[tune] => Pieces
[label] => Ram Records
)

...

[9] => Array
(
[artist] => Zen
[tune] => Full Effect
[label] => Flipmode Audio
)

)

Thursday, 28 February 2008

'Zend Framework Das Entwickler-Handbuch' Buch Rezension

Zend Framework Entwickler HandbuchYesterday I received the book 'Zend Framework Das Entwickler-Handbuch' written by Carsten Möhrke and published via Galileo Press. As the targeted audience of this book will be mainly german-speaking readers and the publisher has currently no intention to publish it in other languages, I will switch the language for this book review.

'Zend Framework Das Entwickler-Handbuch' bietet auf insgesamt 420 Seiten einen guten Einstieg in die Anwendungsentwicklung mit dem Zend Framework bis zu der Version 1.0.3. In den einführenden Kapiteln werden grundlegende Themen wie die Installation des Frameworks, die Theorie des MVC Patterns und dessen Realisierung mit den entsprechenden Komponenten des Zend Framework behandelt. Die folgenden Kapitel des Buches sind dann meist vom MVC Kontext losgelöst und bieten daher gerade für Einsteiger die Möglichkeit weitere Komponenten wie z.B. Zend_Db und Zend_Db_Table für den Datenbankzugriff oder Zend_Auth für die Benutzerauthentifikation isoliert zu erkunden und deren Prinzipien zu verstehen. Das Spektrum der im Buch behandelten Komponenten erstreckt sich von Zend_Acl bis Zend_XmlRpc und bietet somit einen guten Gesamtüberblick auf die vorhandenen und nutzbaren Funktionalitäten des Frameworks. Die einzelnen Komponenten werden dem Leser jeweils durch meist adäquate Code-Listings und die in den Kontext passende Theorie nähergebracht. Aufgrund der raschen technischen Entwicklung und dem etwas unglücklichen gewählten Veröffentlichungstermins werden Komponenten wie z.B. Zend_Layout und Zend_Form aus dem Zend Framework 1.5.0 Preview Release leider nicht behandelt.

Das Buch wendet sich meiner Meinung nach in erster Linie an Einsteiger, welche am Zend Framework interessiert sind und einen schnellen, portablen und deutschsprachigen Einstieg in dessen Magie haben wollen. Für diese Zielgruppe ist das Buch eine klare Empfehlung.

Für Leser die sich schon gute Kenntnisse im Umgang mit dem Framework erarbeitet haben und Besitzer eines permanenten Onlinezugangs sind bietet dieses Buch meiner Meinung nach leider keinen großen Mehrwert, da es ein sehr gutes und kostenloses Referenzhandbuch gibt und auftretende Detailfragen meist mittels diesem, der vorbildlichen API Dokumentation oder in letzter Instanz mit der Hilfe der rasch reagierend Mailinglist Community geklärt werden können. Für diese Zielgruppe ist meine Empfehlung es einmal Probe zu lesen und dann selbst zu entscheiden.

Als einen kaufentscheidenden Mehrwert für erfahrene Leser hätte ich mir z.B. ein grundlegendes Kapitel über das Testen von Zend Framework Anwendungen mit einem xUnit Framework z.B. PHPUnit gewünscht, dieser Aspekt wird von dem Autor leider total ausgeklammert.

Monday, 18 February 2008

Creating Zend Framework snippets for TextMate

After finally converting to Mac OS X, I really couldn't resist to fall for the famous TextMate editor. The editor comes with a nice PHP bundle authored by Ciarán Walsh that eases the typing and creation of common language constructs like classes and control structures by entering pre-defined keys and hitting the tab key to insert and customize them. To reduce the typing effort for the most common tasks in creating a Zend Framework based application, which are creating action controllers including their hosted actions and creating new models for accessing the underlying database, I spent some minutes to figure out how to create and add these valuable snippets to the default PHP bundle.

Adding the controller and model snippets

The Bundle Editor of TextMate is a starter aid that can be used to create the intended Zend Framework snippets and categorize them. To add a new snippet simply select the PHP bundle and hit the most left button in the lower left of the Bundle Editor window. Now you are able to create new snippets, edit their content and assign an activation key to them i.e. zfc and also to define the snippet scope i.e. source.php.

To trigger the automatic insertion of the new snippet you just have to type zfc and hit the tab key in an openend PHP file. The next listing shows the snippet code for the creation of a Zend Framework action controller and as you will notice it contains several at first glance variable look-a-likes. All variables prefixed with TM are definable via the TextMate preferences while the numeric indexed variables are used to tab through the code after insertion and to modify the custom parts like the name for the controller class. They start by an index of 1 for the first tab position after the snippet insertion and end with an index of 0 for the last tab position. Thereby the first two tab strikes will allow you to modify the PHPDoc block while the third tab strike will allow you to edit the controller name e.g. News, which whould make the current code artifact your NewsController class.
/**
* ${1:'File-level' PHPDoc Block}
*
* @author ${PHPDOC_AUTHOR:$TM_FULLNAME} <$TM_ORGANIZATION_EMAIL>
*/

/**
* ${2:'Class-level' PHPDoc Block}
*
* @author ${PHPDOC_AUTHOR:$TM_FULLNAME} <$TM_ORGANIZATION_EMAIL>
*/
class ${3:Name}Controller extends Zend_Controller_Action
{
/**
* ${4:'Method-level' PHPDoc Block}
*/
public function ${5:name}Action()
{
$6
}
$0
}//Activation key: zfc tab
As an action contoller surely will increase all allong by several more actions the next listing shows the snippet code to add a single action to a currently opened action controller class by simply entering zfca and striking the tab key.
/**
* ${1:'Method-level' PHPDoc Block}
*/
public function ${2:name}Action()
{
$0
}//Activation key: zfca tab
The next last valuable snippet would refer to the model part of a Zend Framework application, so the listing shows a snippet for a table class, which automatically sets the table name property to the lowercased and prior entered class name. This snippet would be inserted into the current PHP file by entering the activation key zfm and striking the tab key again.
/**
* ${1:'File-level' PHPDoc Block}
* @author ${PHPDOC_AUTHOR:$TM_FULLNAME} <$TM_ORGANIZATION_EMAIL>
*/

/**
* ${2:'Class-level' PHPDoc Block}
* @author ${PHPDOC_AUTHOR:$TM_FULLNAME} <$TM_ORGANIZATION_EMAIL>
*/
class ${3:Name} extends Zend_Db_Table_Abstract
{
protected \$_name = '${3/./\l$0/}';
}//Activation key: zfm tab

Adding a View Helper snippet

The next snippet was provided by a reader called Jeff Federmann, who contacted me via mail as Google ate his code he tried to contribute in a comment. This snippet can get you up to speed when developing custom View Helpers. The workflow for this snippet is to create a new file for the helper, saving it with a purpose indicating name, which will auto-determine the View Helper class name on code insertion, and trigger the snippet by entering the activation key i.e. zfvh plus hitting the tab again. So nuff talk. Here is the snippet contributed by Jeef.
/**
* ${1:'File-level' PHPDoc Block}
*
* @author ${PHPDOC_AUTHOR:$TM_FULLNAME} <$TM_ORGANIZATION_EMAIL>
*/

/**
* ${2:'Class-level' PHPDoc Block}
*
* @author ${PHPDOC_AUTHOR:$TM_FULLNAME} <$TM_ORGANIZATION_EMAIL>
*/
class Zend_View_Helper_${3:`#!/usr/bin/env php
$filename = $_ENV['TM_FILENAME'];
$filename = str_replace('.php', '', $filename);
echo $filename;
?>`} extends Zend_View_Helper_Abstract
{
/**
* ${4:'Method-level' PHPDoc Block}
*/
public function ${3/./\l$0/}()
{
$5
}
$0
}

Housing the Zend Framework snippets

To finally categorize the Zend Framework snippets and avoid bloating the default PHP Bundle the Bundle Editor comes to rescue again, as it allows to add new categories to an existing bundle in which the shown snippets and other compatible and common snippets, e.g. Zend_Registry access to acquire the application's Zend_Log instance, can be housed.

Monday, 11 February 2008

Zend Framework coding standards on one page

Before jumping into the development of a Zend Framework coding standard for the PHP_CodeSniffer Pear package, I spent some time revisiting and compiling the available Zend Framework coding standards into a handy one-paged Pdf document.

In case somebody is interrested it's available here.

Monday, 3 December 2007

Zend Framework in Action Pre-review

Zend Framework in ActionFor the last few days I've been reading the MEAP release of Zend Framework in Action from Rob Allen and Nick Lo published by Manning Publications and as a pre-review I d'like to share my reading experience. Currently there are 6 of totally 16 intended chapters available, to find out if them are already worth the money you might want to read further.

What's in it?
The first two chapters are introductional and outline what's in the Zend Framework and how it can support and ease the development of (enterprise) web applications. The obligatory Hello World application is used to provide a quick introduction to the MVC pattern and how it's implemented by the framework's controller system and how each part of the MVC pattern is represented in the code artifacts of a application. Next the anatomy of a Zend Framework based application (directory structure, bootstrapping and the request routing to the various components) is layed out to the reader.

After prodviding this 3,000 feet view a community application is introduced, providing the stories and models which will be implemented with the appropriate Zend Framework components and evolve with the coming close-up view chapters. The book carries on to get the guiding application initially going and introduces techniques how to unclutter views through the Two-Step View design pattern and how to build a growing safety-net for your application and it's further evolution via unit tests written in PHPUnit.

The next chapter opens the close-up views by looking at Ajax and how it fits within Zend Framework based applications. The connection of the views to the server/model code is demonstrated first via handcrafted code and next by using two common JavaScript libraries named Prototype and Yahoo! User Interface (YUI). Additional JSON, an important connector in nowadays Ajax application, is covered by taking a look at the Zend_Json component.

Next the focus descends two layers deeper into the model, hosting the specific business logic of an application, by covering the way to establish a connection to the database via the Zend_Db_Adapter and how to perform CRUD actions against it with the Zend_Db_Table component. The linking of tables together is explained by a closer look at the table relationship feature of the Zend_Db_Table component. As a very valuable goodie the authors provide their additional knowledge about testing Zend Framework models.

The last available chapter introduces the concepts of authorisation and authentication and goes on providing the needed theoretical background in addition with explanatory implementations for both scenarios.

Conclusion
As this book is co-authored by Rob Allen, who provides the very valuable introduction I assume everyone getting started with the Zend Framework has used or at least noticed, Zend Framework in Action is the more detailed and naturally matured sequel of it. Though the book isn't released yet the provided knowledge is already valueable whether for novice or more advanced learners/developers whose thirst for knowledge isn't fully covered by the Zend Framework manual. Personally I liked the fact that the authors share their knowledge about unit testing Zend Framework applications and that the reader is guided along an example application, which might motivate to further exploratory coding activities. Reading only these first available chapters leaves you waiting for the rest of it.

Sunday, 26 August 2007

Setting up Zend Framework applications with Phing

After spending too much time on directory and view script shifting to align an 'older' Zend Framework application to the very useful ViewRenderer Action Helper of the follow-up releases I crafted a single Phing buildfile to stick to the recommended conventions and to have a nearly 'one-button' setup solution for any upcoming projects.

There are already some solutions available ranging from prepacked application skeletons to programmatically creators. Of course this is a rude violation of the DRY principle, but you are welcome to read about the basic features the Phing based solution has to offer for now.

Requirements


Features

  • Building a default or modular application directory structure

  • Generating Controller, Model and View skeletons

  • Php linting/validation of generated skeletons

  • Customizable Controller, Model and bootstrap templates

  • Retrieval of a specific Zend Framework version via svn or get

  • Bootstrap file generation

  • Modular targets to generate and add Models and Controllers skeletons belated


Open/possible improvements

  • Couple Model generation with basic database table creating and seeding

  • Zend_Db, Zend_Registry and Zend_Log setup

  • Alignment to Zend_Application proposal, when it's moved into core


After having all requirements aboard it only comes down to the following two steps:

  1. Run the buildfile in the targeted directory

  2. Set the document root to the applications html directory e.g. by using Apaches VirtualHost capabilities

Although it's tested and I put some thoughts in it, any feedback about detected flaws and additional improvements are highly appreciated.

Tuesday, 26 June 2007

Turning a Zend_Log log file into a RSS feed

Whilst touring the web I found an interesting project for turning Apache Web Server log files into RSS feeds. This approach can be adjusted to monitor the maintenance needs of a web application deployed on an assumed productive system. Therefor a XML capable Zend_Log instance will be set up and the resulting log file will be transformed into a RSS feed via a custom Action Helper wrapping a XSLT transformation.

Setting up the XML logger
To reduce the transformation effort a XML logger can be used for accumulating all log entries possibly raised by the application. The following code listing outlines the way to do so in the applications bootstrap file.

...

$writer = new Zend_Log_Writer_Stream('../logs/recordshelf-log.xml');
$writer->setFormatter(new Zend_Log_Formatter_Xml());
$logger = new Zend_Log($writer);
$logger->setEventItem('timestamp', date('D, j M Y H:i:s', time()));

Zend_Registry::set('logger', $logger);
Zend_Registry::set('log', '../logs/recordshelf-log.xml');

...
After setting up the XML logger instance all entries to the log file will look like the following ones, assuming the default format has been kept. It's a bit weird that the resulting log file is not a valid XML document because of a missing XML declaration and a non present root element.
...
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:00:02</timestamp>
<message>Log message 1</message>
<priority>3</priority>
<priorityName>ERR</priorityName>
</logEntry>
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:00:25</timestamp>
<message>Log message 2</message>
<priority>7</priority>
<priorityName>DEBUG</priorityName>
</logEntry>
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:01:30</timestamp>
<message>#0 Exception stacktrace part 1
#1 Exception stacktrace part 2
#2 Exception stacktrace part 3</message>
<priority>3</priority>
<priorityName>ERR</priorityName>
</logEntry>
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:01:30</timestamp>
<message>Log message 3</message>
<priority>3</priority>
<priorityName>ERR</priorityName>
</logEntry>
...
Shifting formats
The desired format transformation can be achieved via a custom Action Helper which simply wraps up a XSLT transformation and can be reused for further 'transformation scenarios' in any Action Controller. It's arguments are the source XML and the XSL stylesheet to apply. The following code shows the Recordshelf_Controller_Action_Helper_Xslt located in the applications custom library.
<?php

require_once 'Zend/Controller/Action/Helper/Abstract.php';

class Recordshelf_Controller_Action_Helper_Xslt extends Zend_Controller_Action_Helper_Abstract {

public function direct(SimpleXMLElement $xml, $xslFile) {

if(!file_exists($xslFile)) {

throw new Zend_Exception("Unable to load xsl stylesheet '{$xslFile}'");

}

$doc = new DOMDocument();
$xslt = new XSLTProcessor();

$doc->load($xslFile);
$xslt->registerPHPFunctions();

// Needed for <pubDate> value in <channel> element
$xslt->setParameter('', 'creationDate', date('D, j M Y H:i:s', time()));
$xslt->importStyleSheet($doc);

$doc->loadXML($xml->asXML());

$this->getResponse()->setHeader('Content-Type', 'text/xml')
->setBody($xslt->transformToXML($doc));

}

}
The XSL stylesheet responsible for the log to RSS transformation is located in an application-root/xsl directory and looks like the following.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:php="http://php.net/xsl">

<xsl:template match="/">
<rss version="2.0">
<channel>
<title>recordshelf.org maintenance feed</title>
<link>http://recordshelf.org/</link>
<description>RSS representation of the recordshelf.org application log file</description>
<language>en-US</language>

<!-- Get creationDate value set in Action Helper -->

<pubDate><xsl:value-of select="$creationDate"/></pubDate>

<xsl:for-each select="logEntries/logEntry">

<item>
<title>
<xsl:variable name="priority" select="priority"/>

<!-- Choose log priority type as <title> value -->

<xsl:choose>
<xsl:when test='$priority = 0'>Emergency message</xsl:when>
<xsl:when test='$priority = 1'>Alert message</xsl:when>
<xsl:when test='$priority = 2'>Critical message</xsl:when>
<xsl:when test='$priority = 3'>Error message</xsl:when>
<xsl:when test='$priority = 4'>Warning message</xsl:when>
<xsl:when test='$priority = 5'>Notice message</xsl:when>
<xsl:when test='$priority = 6'>Informational message</xsl:when>
<xsl:otherwise>Debug message</xsl:otherwise>
</xsl:choose>

</title>

<!-- Raise exception stacktrace readability -->

<description><xsl:value-of select="php:function('nl2br',string(message))"/></description>
<pubDate><xsl:value-of select="timestamp"/></pubDate>
</item>
</xsl:for-each>
</channel>
</rss>
</xsl:template>
</xsl:stylesheet>
Gluing the Action Controller with the Action Helper
The last step to finally achieve the transformation is adding a Action Controller to the application which delegates the work to the afore crafted XSLT Action Helper. This Action Controller should be access restricted to prevent revealing sensitive data.
<?php

class MaintenanceController extends Zend_Controller_Action {

public function init() {

Zend_Controller_Action_HelperBroker::addPath('Recordshelf/Controller/Action/Helper/',
'Recordshelf_Controller_Action_Helper');

}

public function rssAction() {

// Load XML log file and turn it into a valid XML document
$xmlLog = '<?xml version="1.0" encoding="UTF-8"?><logEntries>';
$xmlLog.= file_get_contents(Zend_Registry::get('log'));
$xmlLog.= '</logEntries>';

// Call Action Helper for the transformation
$this->_helper->xslt(new SimpleXMLElement($xmlLog), '../xsl/logToRss.xsl');

}

...

}
Finally polling the applications health
With everything in place the applications 'maintenance/monitor' RSS feed is now available at http://domain.org/maintenance/rss/ for aggregation in a feed reader. I currently use the FeedReader3 since it allows the definition of custom polling intervals.

feedreader