{"id":604,"date":"2011-10-25T20:43:00","date_gmt":"2011-10-25T20:43:00","guid":{"rendered":"http:\/\/www.javacodegeeks.com\/2012\/10\/code-coverage-with-unit-integration-tests.html"},"modified":"2012-10-21T20:24:19","modified_gmt":"2012-10-21T20:24:19","slug":"code-coverage-with-unit-integration","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2011\/10\/code-coverage-with-unit-integration.html","title":{"rendered":"Code coverage with unit & integration tests"},"content":{"rendered":"
On a pet project recently I set out to build automated UI (integration) tests as well as the normal unit tests. I wanted to get all of this integrated into my maven build, with code coverage reports so I could get an idea of areas with insufficient test coverage. Rather than just publish the source code for the project, I\u2019ve put together a simple example to demonstrate how I got all this setup; so if you\u2019re looking to integrate maven<\/a>, junit<\/a>, webdriver <\/a>(now selenium) and emma <\/a>– read on to find out how I went about it.<\/p>\n

First off, all the source code for this is available on github: https:\/\/github.com\/activelylazy\/coverage-example<\/a>. I\u2019ll show key snippets, but obviously there\u2019s lots of detail omitted that (hopefully) isn\u2019t relevant.<\/p>\n

The Example App<\/h1>\n

Rather than break with tradition, the example application is a simple, if slightly contrived, hello world:<\/p>\n

\"\"<\/a><\/div>\n
\n<\/div>\n

How It Works<\/h2>\n

The start page is a simple link to the hello world page:<\/p>\n

<h1>Example app<\/h1>\r\n<p>See the <a id=\"messageLink\" href=\"helloWorld.html\">message<\/a><\/p>\r\n<\/pre>\n
The hello world page just displays the message:<\/p>\n
<h1>Example app<\/h1>\r\n<p id=\"message\"><c:out value=\"${message}\"\/><\/p>\r\n<\/pre>\n
The hello world controller renders the view, passing in the message:<\/p>\n
public class HelloWorldController extends ParameterizableViewController {\r\n    \/\/ Our message factory\r\n    private MessageFactory messageFactory;\r\n    @Override\r\n    protected ModelAndView handleRequestInternal(HttpServletRequest request,\r\n        HttpServletResponse response) throws Exception {\r\n        \/\/ Get the success view\r\n        ModelAndView mav = super.handleRequestInternal(request, response);\r\n        \/\/ Add our message\r\n        mav.addObject(\"message\",messageFactory.createMessage());\r\n        return mav;\r\n    }\r\n    @Autowired\r\n    public void setMessageFactory(MessageFactory messageFactory) {\r\n        this.messageFactory = messageFactory;\r\n    }\r\n}<\/pre>\n
Finally the MessageFactory simply returns the hard-coded message:<\/p>\n
public String createMessage() {\r\n    return \"Hello world\";\r\n}\r\n<\/pre>\n
The unit test<\/h2>\n
We define a simple unit test to verify that the MessageFactory behaves as expected:<\/p>\n
public class MessageFactoryTest {\r\n    \/\/ The message factory\r\n    private MessageFactory messageFactory;\r\n    @Test\r\n    public void testCreateMessage() {\r\n        assertEquals(\"Hello world\",messageFactory.createMessage());\r\n    }\r\n    @Autowired\r\n    public void setMessageFactory(MessageFactory messageFactory) {\r\n        this.messageFactory = messageFactory;\r\n    }\r\n}\r\n<\/pre>\n
Build<\/h2>\n
A basic maven pom file is sufficient to build this and run the unit test. At this point we have a working app, with a unit test for the core functionality (such as it is) that we can build and run.<\/p>\n
<project>\r\n    <modelVersion>4.0.0<\/modelVersion>\r\n    <groupId>com.example<\/groupId>\r\n    <artifactId>helloworld<\/artifactId>\r\n    <packaging>war<\/packaging>\r\n    <version>1.0-SNAPSHOT<\/version>\r\n    <name>helloworld Maven Webapp<\/name>\r\n    <build>\r\n        <finalName>helloworld<\/finalName>\r\n    <\/build>\r\n    <dependencies>\r\n        ...omitted...\r\n    <\/dependencies>\r\n<\/project>\r\n<\/pre>\n
Code Coverage<\/h1>\n
Now let\u2019s integrate Emma so we can get some code coverage reports. First, we define a new Maven profile, this allows us to control whether or not we use emma on any given build.<\/p>\n
<profile>\r\n    <id>with-emma<\/id>\r\n    <build>\r\n        <plugins>\r\n            <plugin>\r\n                <groupId>org.codehaus.mojo<\/groupId>\r\n                <artifactId>emma-maven-plugin<\/artifactId>\r\n                <inherited>true<\/inherited>\r\n                <executions>\r\n                    <execution>\r\n                        <id>instrument<\/id>\r\n                        <phase>process-test-classes<\/phase>\r\n                        <goals>\r\n                            <goal>instrument<\/goal>\r\n                        <\/goals>\r\n                    <\/execution>\r\n                <\/executions>\r\n            <\/plugin>\r\n        <\/plugins>\r\n    <\/build>\r\n<\/profile>\r\n<\/pre>\n
This simply invokes the \u201cinstrument\u201d goal during the Maven \u201cprocess-test-classes\u201d phase; i.e. once we\u2019ve compiled our class files, use emma to instrument them. We can run this by invoking maven with the new profile:<\/p>\n
mvn clean install -Pwith-emma\r\n<\/pre>\n
Once the build has completed, we can run Emma to generate code coverage reports:
\nOn Windows:<\/p>\n
java -cp %USERPROFILE%\/.m2\/repository\/emma\/emma\/2.0.5312\/emma-2.0.5312.jar emma report -r xml,html -in coverage.ec -in target\/coverage.em\r\n<\/pre>\n
On Linux:
 
<\/div> <\/div><\/p>\n
java -cp ~\/.m2\/repository\/emma\/emma\/2.0.5312\/emma-2.0.5312.jar emma report -r xml,html -in coverage.ec -in target\/coverage.em\r\n<\/pre>\n
We can now view the HTML coverage report in coverage\/index.html. At this point, it shows we have 50% test coverage (by classes). MessageFactory is fully covered, but the HelloWorldController doesn\u2019t have any tests at all.<\/p>\n
Integration Test<\/h1>\n
To test our controller and JSP, we\u2019ll use WebDriver to create a simple integration test; this is a JUnit test that happens to launch a browser.<\/p>\n
public class HelloWorldIntegrationTest {\r\n    \/\/ The webdriver\r\n    private static WebDriver driver;\r\n    @BeforeClass\r\n    public static void initWebDriver() {\r\n        driver = new FirefoxDriver();\r\n    }\r\n    @AfterClass\r\n    public static void stopSeleniumClent() {\r\n        try {\r\n            driver.close();\r\n            driver.quit();\r\n        } catch( Throwable t ) {\r\n            \/\/ Catch error & log, not critical for tests\r\n            System.err.println(\"Error stopping driver: \"+t.getMessage());\r\n            t.printStackTrace(System.err);\r\n        }\r\n    }\r\n    @Test\r\n    public void testHelloWorld() {\r\n        \/\/ Start from the homepage\r\n        driver.get(\"http:\/\/localhost:9080\/helloworld\/\");\r\n        HomePage homePage = new HomePage(driver);\r\n        HelloWorldPage helloWorldPage = homePage.clickMessageLink();\r\n        assertEquals(\"Hello world\",helloWorldPage.getMessage());\r\n    }\r\n}\r\n<\/pre>\n
Lines 4-18 simply start Web Driver before the test and shut it down (closing the browser window) once the test is finished.
\nOn line 22 we navigate to the homepage with a hard-coded URL.
\nOn line 23 we initialise our Web Driver page object<\/a> for the homepage. This encapsulates all the details of how the page works, allowing the test to interact with the page functionally, without worrying about the mechanics (which elements to use etc).
\nOn line 24 we use the homepage object to click the \u201cmessage\u201d link; this navigates to the hello world page.
\nOn line 25 we confirm that the message shown on the hello world page is what we expect.
\nNote: I\u2019m using page objects to separate test specification<\/em> (what to do) from test implementation<\/em> (how to do it). For more on why this is important see keeping tests from being brittle<\/a>.<\/p>\n
Homepage<\/h2>\n
The homepage object is pretty simple:<\/p>\n
public HelloWorldPage clickMessageLink() {\r\n    driver.findElement(By.id(\"messageLink\")).click();\r\n    return new HelloWorldPage(driver);\r\n}\r\n<\/pre>\n
HelloWorldPage<\/h2>\n
The hello world page is equally simple:<\/p>\n
public String getMessage() {\r\n    return driver.findElement(By.id(\"message\")).getText();\r\n}\r\n<\/pre>\n
Running the Integration Test<\/h2>\n
To run the integration test during our Maven build we need to make a few changes. First, we need to exclude integration tests from the unit test phase:<\/p>\n
<plugin>\r\n    <groupId>org.apache.maven.plugins<\/groupId>\r\n    <artifactId>maven-surefire-plugin<\/artifactId>\r\n    ...\r\n    <configuration>\r\n        ...\r\n        <excludes>\r\n            <exclude>**\/*IntegrationTest.java<\/exclude>\r\n            <exclude>**\/common\/*<\/exclude>\r\n        <\/excludes>\r\n    <\/configuration>\r\n<\/plugin>\r\n<\/pre>\n
Then we define a new profile, so we can optionally run integration tests:<\/p>\n
<profile>\r\n    <id>with-integration-tests<\/id>\r\n    <build>\r\n        <plugins>\r\n            <plugin>\r\n                <groupId>org.mortbay.jetty<\/groupId>\r\n                <artifactId>maven-jetty-plugin<\/artifactId>\r\n                <version>6.1.22<\/version>\r\n                <configuration>\r\n                    <scanIntervalSeconds>5<\/scanIntervalSeconds>\r\n                    <stopPort>9966<\/stopPort>\r\n                    <stopKey>foo<\/stopKey>\r\n                    <connectors>\r\n                        <connector implementation=\"org.mortbay.jetty.nio.SelectChannelConnector\">\r\n                            <port>9080<\/port>\r\n                            <maxIdleTime>60000<\/maxIdleTime>\r\n                        <\/connector>\r\n                    <\/connectors>\r\n                <\/configuration>\r\n                <executions>\r\n                    <execution>\r\n                        <id>start-jetty<\/id>\r\n                        <phase>pre-integration-test<\/phase>\r\n                        <goals>\r\n                            <goal>run<\/goal>\r\n                        <\/goals>\r\n                        <configuration>\r\n                            <daemon>true<\/daemon>\r\n                        <\/configuration>\r\n                    <\/execution>\r\n                    <execution>\r\n                        <id>stop-jetty<\/id>\r\n                        <phase>post-integration-test<\/phase>\r\n                        <goals>\r\n                            <goal>stop<\/goal>\r\n                        <\/goals>\r\n                    <\/execution>\r\n                <\/executions>\r\n            <\/plugin>\r\n            <plugin>\r\n                <groupId>org.apache.maven.plugins<\/groupId>\r\n                <artifactId>maven-surefire-plugin<\/artifactId>\r\n                <version>2.5<\/version>\r\n                <inherited>true<\/inherited>\r\n                <executions>\r\n                    <execution>\r\n                        <id>integration-tests<\/id>\r\n                        <phase>integration-test<\/phase>\r\n                        <goals>\r\n                            <goal>test<\/goal>\r\n                        <\/goals>\r\n                        <configuration>\r\n                            <excludes>\r\n                                <exclude>**\/common\/*<\/exclude>\r\n                            <\/excludes>\r\n                            <includes>\r\n                                <include>**\/*IntegrationTest.java<\/include>\r\n                            <\/includes>\r\n                        <\/configuration>\r\n                    <\/execution>\r\n                <\/executions>\r\n            <\/plugin>\r\n        <\/plugins>\r\n    <\/build>\r\n<\/profile>\r\n<\/pre>\n
<profile><\/p>\n
<id>with-integration-tests<\/id><\/p>\n
<build><\/p>\n
<plugins><\/p>\n
<plugin><\/p>\n
<groupId>org.mortbay.jetty<\/groupId><\/p>\n
<artifactId>maven-jetty-plugin<\/artifactId><\/p>\n
<version>6.1.22<\/version><\/p>\n
<configuration><\/p>\n
<scanIntervalSeconds>5<\/scanIntervalSeconds><\/p>\n
<stopPort>9966<\/stopPort><\/p>\n
<stopKey>foo<\/stopKey><\/p>\n
<connectors><\/p>\n
<connector implementation=\u201dorg.mortbay.jetty.nio.SelectChannelConnector\u201d><\/p>\n
<port>${test.server.port}<\/port><\/p>\n
<maxIdleTime>60000<\/maxIdleTime><\/p>\n
<\/connector><\/p>\n
<\/connectors><\/p>\n
<\/configuration><\/p>\n
<executions><\/p>\n
<execution><\/p>\n
<id>start-jetty<\/id><\/p>\n
<phase>pre-integration-test<\/phase><\/p>\n
<goals><\/p>\n
<goal>run<\/goal><\/p>\n
<\/goals><\/p>\n
<configuration><\/p>\n
<daemon>true<\/daemon><\/p>\n
<\/configuration><\/p>\n
<\/execution><\/p>\n
<execution><\/p>\n
<id>stop-jetty<\/id><\/p>\n
<phase>post-integration-test<\/phase><\/p>\n
<goals><\/p>\n
<goal>stop<\/goal><\/p>\n
<\/goals><\/p>\n
<\/execution><\/p>\n
<\/executions><\/p>\n
<\/plugin><\/p>\n
<plugin><\/p>\n
<groupId>org.apache.maven.plugins<\/groupId><\/p>\n
<artifactId>maven-surefire-plugin<\/artifactId><\/p>\n
<version>2.5<\/version><\/p>\n
<inherited>true<\/inherited><\/p>\n
<executions><\/p>\n
<execution><\/p>\n
<id>integration-tests<\/id><\/p>\n
<phase>integration-test<\/phase><\/p>\n
<goals><\/p>\n
<goal>test<\/goal><\/p>\n
<\/goals><\/p>\n
<configuration><\/p>\n
<excludes><\/p>\n
<exclude>**\/common\/*<\/exclude><\/p>\n
<\/excludes><\/p>\n
<includes><\/p>\n
<include>**\/*IntegrationTest.java<\/include><\/p>\n
<\/includes><\/p>\n
<\/configuration><\/p>\n
<\/execution><\/p>\n
<\/executions><\/p>\n
<\/plugin><\/p>\n
<\/plugins><\/p>\n
<\/build><\/p>\n
<\/profile><\/p>\n<\/div>\n
This may look complex, but really we\u2019re just configuring jetty to run while we run our integration tests; then configuring how to run the integration tests themselves.
\nIn lines 9-19 configure jetty \u2013 the port to run on and how to stop it.
\nLines 21-30 configure jetty to run during the \u201cpre-integration-test\u201d phase of the maven build.
\nLines 31-37 configure jetty to be stopped during the \u201cpost-integration-test\u201d phase of the maven build.
\nIn lines 40-62 we use the maven-surefire-plugin again, this time to run during the \u201cintegration-test\u201d phase of the build, only running our integration test classes.<\/p>\n
We can run this build with:<\/p>\n
mvn clean install -Pwith-emma -Pwith-integration-tests\r\n<\/pre>\n
This will build everything, run the unit tests, build the war, fire up jetty to host the war, run our integration tests (you\u2019ll see a firefox window popup while the rest runs) then shut down jetty. Because the war is built with instrumented classes, Emma also tracks code coverage while we run our integration tests.<\/p>\n
We can now build our application, running unit tests and integration tests, gathering combined code coverage reports. If we re-run the emma report and check code coverage we now see we have 100% test coverage \u2013 since the controller is also being covered through tests.<\/p>\n
Issues<\/h1>\n
What are the outstanding issues with this, what further extensions can be made?<\/p>\n