AUTOMATED TESTING WITH
CODECEPTION
PHP automated testing framework	

Jonathan Lau

Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
MOTIVATION + BIO
•

Quick start guide for codeception	


•

Experience with running it in a Cake based project	


•

Introduce it as a alternative choice

Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
TEST STRATEGY
What do we use these days?
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
WAYS TO TEST
•

Unit test

•

Functional test

•

White box testing	


•

•

Full knowledge of the code
base	


Knowledge of input and
expected output	


•

No browser emulation	


•

!

Run script during build /
check out	


•

Acceptance test
•

Yes, still need this for sanity
sake

Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
EFFORT VS RETURN
•

•

Unit tests are easy to write
but it can’t cover integration
issue	

Functional / acceptance
tests can cover the end to
end behavior but it can take
a while to write and some
effort to maintain

Maintenance effort
Acceptance test

Functional test
Unit test

Coverage

Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
ACCEPTANCE TEST
Let the robot do the clicking
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
ACCEPTANCE TEST
•

Browser emulator: Selenum or Mink	


•

Select on-screen elements by text or path	


•

Supported actions: click, enter text, drag drop etc.	


•

Coverage can be an issue	


•

Really SLOW
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
EXAMPLE CODE
<?php	

$I = new WebGuy($scenario);	

$I->amOnPage('/');	

$I->click('Sign Up');	

$I->submitForm('#signup', array('username' => 'MilesDavis', 'email' =>
'miles@davis.com'));	

$I->see('Thank you for Signing Up!');	

?>
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
FUNCTIONAL TEST
•

Very similar concept as acceptance test	


•

No browser emulation 	


•

Emulate the actual web request instead	


•

Might need hook into the framework to introduce a test
mode.	


•

Can’t test javascript / AJAX
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
EXAMPLE CODE
<?php	

$I = new TestGuy($scenario);	

$I->amOnPage('/');	

$I->click('Sign Up');	

$I->submitForm('#signup', array('username' => 'MilesDavis', 'email' => 'miles@davis.com'));	

$I->see('Thank you for Signing Up!');	

$I->seeEmailSent('miles@davis.com', 'Thank you for registration');	

$I->seeInDatabase('users', array('email' => 'miles@davis.com'));	

?>
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
UNIT TEST
•

Friendly only to developer	


•

Running the test subject in isolation and surrounding pieces are
stubbed out	


•

It’s good for validating logic correctness 	


•

Can’t validate integration issues	


•

Built on top of PHP Unit and has more tooling to help write test
faster
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
EXAMPLE CODE
<?php	

// we are testing the public method of User class.	

$I = new CodeGuy($scenario);	

$I->testMethod('User.update');	

$I->haveStubClass($unit = Stub::make('User'));	

$I->dontSeeInDatabase('users', array('id' => 1, 'username' => 'miles'));	

$I->executeTestedMethodOn($unit, 1, array('username' => 'miles'));	

$I->seeMethodInvoked($unit, 'save');	

$I->seeInDatabase('users', array('id' => 1, 'username' => 'miles'));	

?>
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
GENERAL PITFALLS
•

Javascript pop up box	


•

Drag and drop - takes a bit more work to get it to
work	


•

Delay tuning - variance in the latency on the web
servers	


•

Browser session is refreshed between test files
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
WHAT WORKS FOR US
•

Integration risk is always
larger than algorithmic
correctness	


•

Automating acceptance test
is a huge win	


•

Automating functional test
should be next	


•

Unit test… meh…
Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com
KILL BUGS
LIKE THE
TERRAINS
Good luck and thanks	

!

jon@smokehousesoftware.com
!

Contributors: Kenneth Chiu	


Smokehouse Software | Jonathan Lau | jon@smokehousesoftware.com

Codeception

  • 1.
    AUTOMATED TESTING WITH CODECEPTION PHPautomated testing framework Jonathan Lau Smokehouse Software | Jonathan Lau | [email protected]
  • 2.
    MOTIVATION + BIO • Quickstart guide for codeception • Experience with running it in a Cake based project • Introduce it as a alternative choice Smokehouse Software | Jonathan Lau | [email protected]
  • 3.
    TEST STRATEGY What dowe use these days? Smokehouse Software | Jonathan Lau | [email protected]
  • 4.
    WAYS TO TEST • Unittest • Functional test • White box testing • • Full knowledge of the code base Knowledge of input and expected output • No browser emulation • ! Run script during build / check out • Acceptance test • Yes, still need this for sanity sake Smokehouse Software | Jonathan Lau | [email protected]
  • 5.
    EFFORT VS RETURN • • Unittests are easy to write but it can’t cover integration issue Functional / acceptance tests can cover the end to end behavior but it can take a while to write and some effort to maintain Maintenance effort Acceptance test Functional test Unit test Coverage Smokehouse Software | Jonathan Lau | [email protected]
  • 6.
    ACCEPTANCE TEST Let therobot do the clicking Smokehouse Software | Jonathan Lau | [email protected]
  • 7.
    ACCEPTANCE TEST • Browser emulator:Selenum or Mink • Select on-screen elements by text or path • Supported actions: click, enter text, drag drop etc. • Coverage can be an issue • Really SLOW Smokehouse Software | Jonathan Lau | [email protected]
  • 8.
    EXAMPLE CODE <?php $I =new WebGuy($scenario); $I->amOnPage('/'); $I->click('Sign Up'); $I->submitForm('#signup', array('username' => 'MilesDavis', 'email' => '[email protected]')); $I->see('Thank you for Signing Up!'); ?> Smokehouse Software | Jonathan Lau | [email protected]
  • 9.
    FUNCTIONAL TEST • Very similarconcept as acceptance test • No browser emulation • Emulate the actual web request instead • Might need hook into the framework to introduce a test mode. • Can’t test javascript / AJAX Smokehouse Software | Jonathan Lau | [email protected]
  • 10.
    EXAMPLE CODE <?php $I =new TestGuy($scenario); $I->amOnPage('/'); $I->click('Sign Up'); $I->submitForm('#signup', array('username' => 'MilesDavis', 'email' => '[email protected]')); $I->see('Thank you for Signing Up!'); $I->seeEmailSent('[email protected]', 'Thank you for registration'); $I->seeInDatabase('users', array('email' => '[email protected]')); ?> Smokehouse Software | Jonathan Lau | [email protected]
  • 11.
    UNIT TEST • Friendly onlyto developer • Running the test subject in isolation and surrounding pieces are stubbed out • It’s good for validating logic correctness • Can’t validate integration issues • Built on top of PHP Unit and has more tooling to help write test faster Smokehouse Software | Jonathan Lau | [email protected]
  • 12.
    EXAMPLE CODE <?php // weare testing the public method of User class. $I = new CodeGuy($scenario); $I->testMethod('User.update'); $I->haveStubClass($unit = Stub::make('User')); $I->dontSeeInDatabase('users', array('id' => 1, 'username' => 'miles')); $I->executeTestedMethodOn($unit, 1, array('username' => 'miles')); $I->seeMethodInvoked($unit, 'save'); $I->seeInDatabase('users', array('id' => 1, 'username' => 'miles')); ?> Smokehouse Software | Jonathan Lau | [email protected]
  • 13.
    GENERAL PITFALLS • Javascript popup box • Drag and drop - takes a bit more work to get it to work • Delay tuning - variance in the latency on the web servers • Browser session is refreshed between test files Smokehouse Software | Jonathan Lau | [email protected]
  • 14.
    WHAT WORKS FORUS • Integration risk is always larger than algorithmic correctness • Automating acceptance test is a huge win • Automating functional test should be next • Unit test… meh… Smokehouse Software | Jonathan Lau | [email protected]
  • 15.
    KILL BUGS LIKE THE TERRAINS Goodluck and thanks ! [email protected] ! Contributors: Kenneth Chiu Smokehouse Software | Jonathan Lau | [email protected]