Dockerfile<\/h2>\n
Okay, now that we have a codebase and a Codeship-specific branch, we need to prepare it to run our tests using Docker and Codeship. (I\u2019m not going to cover getting Docker and the Jet CLI tool installed, so I suggest you follow Codeship\u2019s own Jet installation tutorial<\/a> before moving forward.)<\/p>\n The first thing we need to do is set up a Dockerfile. If you are new to Docker, this file is used to configure a container that is managed by Docker.<\/p>\n Now, with our Dockerfile, we are able to spin up a suitable Docker container for building and testing our codebase. However, the Dockerfile is only part of the solution. We also need to tell Codeship about our Dockerfile, as well as which steps to take in order to run our test suite. This is accomplished using two files: codeship_services.yml<\/a> and codeship_steps.yml<\/a>.<\/p>\n The codeship-services.yml file is essentially a Docker Compose file, which defines any services, networks, and volumes our build process might need. To get started, we\u2019ll keep our services file pretty lightweight by only building one image. It is important to note here that Codeship supports a \u2018cached\u2019 directive<\/a> that attempts to reuse previous builds in order to speed up the build process.<\/p>\n Next, the Now that we have our three configuration settings (and hopefully have Jet and Docker configured locally), we\u2019re ready to put it all together! To run our test process, simply call the As you can see, excluding the build time (which I precached to speed subsequent tests up), the entire test suite of this project takes about 20 seconds. Not a bad base benchmark, but we can do better.<\/p>\n So, how do we fix the long build time? This is where the \u2018parallel\u2019 directive<\/a> from the What this directive does is allow us to run steps in parallel, which in this case means running each subdirectory in our tests directory independently of the others. We can therefore run multiple tests at once, which will in turn let us drastically reduce the length of our test time. This is achieved by spinning up separate containers, in order to allow us to retain control over performance and resource use.<\/p>\n With just a little tweak to our Up to this point, we\u2019ve been focusing on speeding up our test suite through parallelization, but what about the rest of the process? How can we optimize everything from building to deploying? This is where the The syntax of this file is based on Docker Compose<\/a> (meaning most<\/a> syntax supported by Docker Compose is supported by Codeship Services), which is used to define multiple Docker containers for use in our build process. With a few small changes to this file, we can persist data across our containers, which will allow us to reduce unnecessary overhead and streamline each step.<\/p>\n To accomplish this, the first thing we need to do is split our build and test steps into two separate services. To keep this example straightforward, we will be using the same Dockerfile; however in the event that different processes require different packages, using multiple Dockerfiles would be appropriate.<\/p>\n Let\u2019s break this file down a bit.<\/p>\n As you can see, the original app service has been duplicated into two separate services: build and test. What\u2019s different about these two services \u2014 the I should note that because this is a backend PHP framework we are working with, the build process is a fairly simple installation of third-party PHP packages. In more complicated codebases, however, the build step could include anything from binary compilation to asset uglification and minification (the sky\u2019s really the limit here). This flexibility is advantageous from a DevOps perspective because it assures that you are free to use whichever frameworks and development tools are the best fit \u2014 which is one of the key reasons for using Docker and a microservices architecture in the first place.<\/p>\n Up until now, we have been doing all of our work locally via the Jet CLI tool, but at some point we need to actually start pushing data somewhere. Thankfully, Codeship supports deploying containers to remote registries<\/a>, which allows us to deploy a built container wherever we want. While deployments are pretty par for the course when it comes to CI\/CD services, what makes this unique is our ability to control exactly what gets deployed, and when.<\/p>\n For example, we might want to push our container to the remote registry before updating a code coverage report, or run database migrations in a parallel process. Or maybe we don\u2019t want to deploy to a container image at all. Deployments could also be artifacts, APIs, or even to SFTP. This is a major advantage, since it means you can deploy to whatever infrastructure you already have. You don\u2019t have to change your infrastructure to match your new development workflow.<\/p>\n Here, though, I\u2019ll cover the basics of deploying to a container registry. I won\u2019t go over the remote registry pushing configuration process because Codeship explains it<\/a> much more effectively than I would, but in the context of this demo, it\u2019s as simple as adding an additional step to our codeship-steps.yml<\/a> file.<\/p>\n If you followed the Codeship documentation for enabling repository pushes, then you can see that getting it hooked into this demo was surprisingly easy. One quick push to GitHub later, and we now have an end-to-end build, test, and deploy process split across multiple containers and steps!<\/p>\n While this is a very limited demo, the potential here should be self-evident. The power of Codeship\u2019s Docker support is incredible, and with the addition of their Jet CLI tool<\/a>, you can incorporate it into your development process right from the start.<\/p>\n# Start with the offical image for PHP 5.6\r\nFROM php:5.6\r\n\r\n# Install additonal packages\r\nRUN apt-get update\r\nRUN apt-get install -y git unzip libmcrypt-dev libmemcached-dev libz-dev\r\nRUN pecl install memcached\r\n\r\n# Install\/Enable PHP extensions\r\nRUN docker-php-ext-install pcntl mcrypt\r\nRUN echo extension=memcached.so >> \/usr\/local\/etc\/php\/conf.d\/memcached.ini\r\n\r\n# Install Composer and make it available in the PATH\r\nRUN curl -sS https:\/\/getcomposer.org\/installer | php -- --install-dir=\/usr\/bin\/ --filename=composer\r\n\r\n# Set the current working directory\r\nWORKDIR \/app\r\n\r\n# Copy the source code onto the container (required by composer)\r\nCOPY src .\/src\r\n\r\n# Copy over the composer.json and run composer install.\r\n# This is done as a separate steps so the image can be cached this step won't be\r\n# rerun unless you change composer.json\r\nCOPY composer.json .\/\r\nRUN composer install --prefer-source --no-interaction\r\n\r\n# Finally copy the tests onto the container\r\nCOPY phpunit.xml .\/\r\nCOPY tests .\/tests<\/pre>\n
Services and Steps<\/h2>\n
app:\r\n build:\r\n image: app\r\n dockerfile_path: Dockerfile\r\n cached: true<\/pre>\n
codeship-steps.yml<\/code> file is a configuration file that defines the steps of the build, test, and deploy process. Again, to get started, we\u2019re going to keep this file lightweight. For now, we\u2019ll run our entire test suite all at once. Also, while we are only running one step at first, I have directed Codeship and Jet to run my steps in parallel. This will come in handy in a little bit.<\/p>\n- type: parallel\r\n steps:\r\n - service: app\r\n command: vendor\/bin\/phpunit<\/pre>\n
Putting It All Together<\/h2>\n
jet steps<\/code> command from within the project directory. Here\u2019s a quick screenshot of the results of the first run process:<\/p>\n![\"image01\"](\"http:\/\/www.webcodegeeks.com\/wp-content\/uploads\/2016\/11\/image01.png\")
<\/a><\/p>\nParallelization<\/h2>\n
codeship-steps.yml<\/code> file comes into play.<\/p>\n- type: parallel\r\n steps:\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Auth\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Broadcasting\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Bus\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Cache\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Console\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Container\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Cookie\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Database\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Encryption\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Events\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Filesystem\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Foundation\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Hashing\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Http\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Log\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Mail\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Pagination\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Pipeline\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Queue\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Redis\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Routing\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Session\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Support\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Translation\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/Validation\/\r\n - service: app\r\n command: vendor\/bin\/phpunit tests\/View\/<\/pre>\n
codeship-steps.yml<\/code> file, we can split up our test suite into multiple parallel processes, which reduces our total test time to 14 seconds. In a test suite that takes less than a minute, the speedup might feel negligible, but in a large suite with 10 minutes\u2019 worth of tests, this can save us a ton of time in the long run.<\/p>\n![\"image02\"](\"http:\/\/www.webcodegeeks.com\/wp-content\/uploads\/2016\/11\/image02.png\")
<\/a><\/p>\nMultiple Services<\/h2>\n
codeshipservices.yml<\/code> file comes into play.<\/p>\nBuild and Test<\/h2>\n
build:\r\n build:\r\n image: app\r\n dockerfile_path: Dockerfile\r\n cached: true\r\n volumes_from:\r\n - data\r\ntest:\r\n build:\r\n image: app\r\n dockerfile_path: Dockerfile\r\n cached: true\r\n volumes_from:\r\n - data\r\ndata:\r\n image: busybox\r\n volumes:\r\n - .\/vendor:\/app\/vendor<\/pre>\n
volumes<\/code><\/a> and volumes_from<\/code> directives. Because each service will require a fully built application, the test service inherits mounted volumes from the build service. What this means is that when the build process is run (via Composer), the resulting packages are persisted across each container.<\/p>\nDeploy<\/h2>\n
- type: serial\r\n steps:\r\n - type: serial\r\n steps:\r\n - service: build\r\n command: composer install --prefer-source --no-interaction\r\n - type: parallel\r\n steps:\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Auth\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Broadcasting\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Bus\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Cache\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Console\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Container\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Cookie\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Database\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Encryption\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Events\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Filesystem\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Foundation\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Hashing\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Http\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Log\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Mail\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Pagination\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Pipeline\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Queue\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Redis\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Routing\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Session\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Support\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Translation\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/Validation\/\r\n - service: test\r\n command: vendor\/bin\/phpunit tests\/View\/\r\n - type: serial\r\n steps:\r\n - service: build\r\n type: push\r\n image_name: zachflower\/codeship-jet-parallel-ci\r\n registry: https:\/\/index.docker.io\/v1\/\r\n encrypted_dockercfg_path: dockercfg.encrypted<\/pre>\n
![\"image00\"](\"http:\/\/www.webcodegeeks.com\/wp-content\/uploads\/2016\/11\/image00.png\")
<\/a><\/p>\nConclusion<\/h2>\n