Docker is an amazing tool for developers. It allows us to build and replicate images on any host, removing the inconsistencies of dev environments and reducing onboarding timelines considerably.<\/p>\n
To provide an example of how you might move to containerized development, I built a simple In a two-part series, I will cover the development and pipeline creation steps. In this post, I will cover the first part: developing and testing with Docker Compose.<\/p>\n This tutorial requires you to have a few items before you can get started.<\/p>\n The todo app here is essentially a stand-in, and you could replace it with your own application. Some of the setup here is specific for this application, and the needs of your application may not be covered, but it should be a good starting point for you to get the concepts needed to Dockerize your own applications.<\/p>\n Once you have everything set up, you can move on to the next section.<\/p>\n At the foundation of any Dockerized application, you will find a In the root directory of the application, create a new Open the newly created If you run With only one instruction in the Dockerfile, this doesn\u2019t do too much, but it does show you the build process without too much happening. At this point, you now have an image created, and running The This particular Docker file uses Let\u2019s add the instructions to the Here is what is happening:<\/p>\n We initially install into You can now run You have now successfully created the application image using Docker. Currently, however, our app won\u2019t do much since we still need a database, and we want to connect everything together. This is where Docker Compose will help us out.<\/p>\n Now that you know how to create an image with a Create the file The Docker Compose file<\/a> will define and run the containers based on a configuration file. We are using compose file version 2<\/a> syntax, and you can read up on it on Docker\u2019s site.<\/p>\n An important concept to understand is that Docker Compose spans \u201cbuildtime\u201d and \u201cruntime.\u201d Up until now, we have been building images using Compose triggers \u201cbuildtime\u201d \u2014 instructing our images and containers to build \u2014 but it also populates data used at \u201cruntime,\u201d such as env vars and volumes. This is important to be clear on. For instance, when we add things like Open your This will take a bit to unpack, but let\u2019s break it down by service.<\/p>\n The first directive in the web service is to The This Compose file uses \u201clinks.\u201d The preference here is Like the NodeJS image we used, the Docker Store has a prebuilt image for PostgreSQL<\/a>. Instead of using a Now that we have our services defined, we can build the application using At this point, the application is running, and you will see log output in the console. You can also run the services as a background process, using At a new command prompt, you can run This will tell you the name of the services, the command used to start it, its current state, and the ports. Notice The Eventually, however, it would be good to check how your app will build with a clean setup. You can run Feel free to check out the source code, play around a bit, and see how things go for you.<\/p>\n The application itself includes some integration tests built using The current containers are running using the project name Instead, we will use a different project name to run the application, isolating the tests into their own environment. Since containers are ephemeral (short-lived), running your tests in a separate set of containers makes certain that your app is behaving exactly as it should in a clean environment.<\/p>\n In your terminal, run the following command:<\/p>\n You should see The Since the At this point, you should have a solid start using Docker Compose for local app development. In the next part of this series about using Docker Compose for NodeJS development, I will cover integration and deployments of this application using Codeship.<\/p>\n Is your team using Docker in its development workflow? If so, I would love to hear about what you are doing and what benefits you see as a result.<\/p>\ntodo<\/code> API using NodeJS, Express, and PostgreSQL using Docker Compose<\/a> for development, testing, and eventually in my CI\/CD pipeline.<\/p>\nRequirements for This Tutorial<\/h2>\n
\n
Creating the Dockerfile<\/h2>\n
Dockerfile<\/code><\/a>. The Dockerfile<\/code> contains all of the instructions used to build out the application image. You can set this up by installing NodeJS and all of its dependencies; however the Docker ecosystem has an image repository (the Docker Store) with a NodeJS<\/a> image already created and ready to use.<\/p>\nDockerfile<\/code>.<\/p>\n\/> touch Dockerfile<\/pre>\n
Dockerfile<\/code> in your favorite editor. The first instruction, FROM<\/code><\/a>, will tell Docker to use the prebuilt NodeJS image. There are several choices, but this project uses the node:7.7.2-alpine<\/code> image. For more details about why I\u2019m using alpine<\/code> here over the other options, you can read this post<\/a>.<\/p>\nFROM node:7.7.2-alpine<\/pre>\n
docker build .<\/code>, you will see something similar to the following:<\/p>\nSending build context to Docker daemon 249.3 kB\r\nStep 1\/1 : FROM node:7.7.2-alpine\r\n7.7.2-alpine: Pulling from library\/node\r\n709515475419: Pull complete\r\n1a7746e437f7: Pull complete\r\n662ac7b95f9d: Pull complete\r\nDigest: sha256:6dcd183eaf2852dd8c1079642c04cc2d1f777e4b34f2a534cc0ad328a98d7f73\r\nStatus: Downloaded newer image for node:7.7.2-alpine\r\n ---> 95b4a6de40c3\r\nSuccessfully built 95b4a6de40c3<\/pre>\n
docker images<\/code> will show you the images you have available:<\/p>\nREPOSITORY TAG IMAGE ID CREATED SIZE\r\nnode 7.7.2-alpine 95b4a6de40c3 6 weeks ago 59.2 MB<\/pre>\n
Dockerfile<\/code> needs more instructions to build out the application. Currently it\u2019s only creating an image with NodeJS installed, but we still need our application code to run inside the container. Let\u2019s add some more instructions to do this and build this image again.<\/p>\nRUN<\/code><\/a>, COPY<\/code><\/a>, and WORKDIR<\/code><\/a>. You can read more about those on Docker\u2019s reference page to get a deeper understanding.<\/p>\nDockerfile<\/code> now:<\/p>\nFROM node:7.7.2-alpine\r\n\r\nRUN apk update\r\nCOPY package.json \/tmp\/package.json\r\nRUN cd \/tmp && npm install --quiet\r\nRUN mkdir -p \/usr\/app && cp -a \/tmp\/node_modules \/usr\/app\r\n\r\nWORKDIR \/usr\/app\r\nCOPY .\/ \/usr\/app\/<\/pre>\n
\n
apk<\/code> packages<\/li>\npackage.json<\/code> file to tmp<\/code><\/li>\nnode_modules<\/code> in tmp<\/code><\/li>\nnode_modules<\/code> to \/usr\/app<\/code><\/li>\n\/usr\/app<\/code><\/li>\n\/usr\/app<\/code><\/li>\n<\/ul>\ntmp<\/code> to make use of the cache layers I talked about earlier. By doing it this way, we are able to cache node_modules<\/code> and not install them every time we build the image unless there are new ones to be installed.<\/p>\ndocker build .<\/code> again and see the results:<\/p>\nSending build context to Docker daemon 249.3 kB\r\nStep 1\/7 : FROM node:7.7.2-alpine\r\n ---> 95b4a6de40c3\r\nStep 2\/7 : RUN apk update\r\n ---> Running in 9e1ba27dcd8d\r\nfetch http:\/\/dl-cdn.alpinelinux.org\/alpine\/v3.4\/main\/x86_64\/APKINDEX.tar.gz\r\nfetch http:\/\/dl-cdn.alpinelinux.org\/alpine\/v3.4\/community\/x86_64\/APKINDEX.tar.gz\r\nv3.4.6-101-g85afbbc [http:\/\/dl-cdn.alpinelinux.org\/alpine\/v3.4\/main]\r\nv3.4.6-83-g67e50bc [http:\/\/dl-cdn.alpinelinux.org\/alpine\/v3.4\/community]\r\nOK: 5972 distinct packages available\r\n ---> fe85afcf457d\r\nRemoving intermediate container 9e1ba27dcd8d\r\nStep 3\/7 : COPY package.json \/tmp\/package.json\r\n ---> 5bcbc25f16f1\r\nRemoving intermediate container 7bdfe092ce5e\r\nStep 4\/7 : RUN cd \/tmp && npm install --quiet\r\n ---> Running in 57f5ac4618ec\r\n\r\n ### NPM MODULES INSTALLED ###\r\n\r\n ---> 525f662aeacf\r\nRemoving intermediate container 57f5ac4618ec\r\nStep 5\/7 : RUN mkdir -p \/usr\/app && cp -a \/tmp\/node_modules \/usr\/app\r\n ---> Running in 05529fe28f41\r\n ---> 58dc99530a26\r\nRemoving intermediate container 05529fe28f41\r\nStep 6\/7 : WORKDIR \/usr\/app\r\n ---> 394a19b73f96\r\nRemoving intermediate container be09c6bcae84\r\nStep 7\/7 : COPY .\/ \/usr\/app\/\r\n ---> 7b591e02ded3\r\nRemoving intermediate container 7bfb004ad51f\r\nSuccessfully built 7b591e02ded3<\/pre>\n
Docker Compose Services<\/h2>\n
Dockerfile<\/code>, let\u2019s create an application as a service and connect it to a database. Then we can run some setup commands and be on our way to creating that new todo list.<\/p>\ndocker-compose.yml<\/code>:<\/p>\n\/> touch docker-compose.yml<\/pre>\n
docker build .<\/code>, which is \u201cbuildtime.\u201d This is when our containers are actually built. We can think of \u201cruntime\u201d as what happens once our containers are built and being used.<\/p>\nvolumes<\/code> and command<\/code>, they will override the same things that may have been set up via the Dockerfile at \u201cbuildtime.\u201d<\/p>\ndocker-compose.yml<\/code> file in your editor and copy\/paste the following lines:<\/p>\nversion: '2'\r\nservices:\r\n web:\r\n build: .\r\n command: npm run dev\r\n volumes:\r\n - .:\/usr\/app\/\r\n - \/usr\/app\/node_modules\r\n ports:\r\n - \"3000:3000\"\r\n links:\r\n - postgres\r\n environment:\r\n DATABASE_URL: postgres:\/\/todoapp@postgres\/todos\r\n postgres:\r\n image: postgres:9.6.2-alpine\r\n environment:\r\n POSTGRES_USER: todoapp\r\n POSTGRES_DB: todos<\/pre>\n
The web service<\/h3>\n
build<\/code> the image based on our Dockerfile<\/code>. This will recreate the image we used before, but it will now be named according to the project we are in, nodejsexpresstodoapp<\/code>. After that, we are giving the service some specific instructions on how it should operate:<\/p>\n\n
command: npm run dev<\/code> \u2013 Once the image is built, and the container is running, the npm run dev<\/code> command will start the application.<\/li>\nvolumes:<\/code> \u2013 This section will mount paths between the host and the container.<\/li>\n.:\/usr\/app\/<\/code> \u2013 This will mount the root directory to our working directory in the container.<\/li>\n\/usr\/app\/node_modules<\/code> \u2013 This will mount the node_modules<\/code> directory to the host machine using the buildtime directory.<\/li>\nlinks:<\/code> \u2013 This will create a dependency and create the environment variable to network the services together.<\/li>\nenvironment:<\/code> \u2013 The application itself expects the environment variable DATABASE_URL<\/code> to run. This is set in db.js<\/code>.<\/li>\nports:<\/code> \u2013 This will publish the container\u2019s port, in this case 3000<\/code>, to the host as port 3000<\/code>.<\/li>\n<\/ul>\nDATABASE_URL<\/code> is the connection string. postgres:\/\/todoapp@postgres\/todos<\/code> connects using the todoapp<\/code> user, on the host postgres<\/code>, using the database todos<\/code>.<\/p>\ndepends_on<\/code><\/a>, however, Codeship currently doesn\u2019t support depends_on<\/code>. This will make the transition from local to CI\/CD easier at a later step.<\/p><\/blockquote>\nThe Postgres service<\/h3>\n
build<\/code> directive, we can use the name of the image, and Docker will grab that image for us and use it. In this case, we are using postgres:9.6.2-alpine<\/code>. We could leave it like that, but it has environment<\/code> variables to let us customize it a bit.<\/p>\nenvironment:<\/code> \u2013 This particular image accepts a couple environment variables so we can customize things to our needs. POSTGRES_USER: todoapp<\/code> \u2013 This creates the user todoapp<\/code> as the default user for PostgreSQL. POSTGRES_DB: todos<\/code> \u2013 This will create the default database as todos<\/code>.<\/p>\nRunning The Application<\/h2>\n
docker-compose up<\/code>. This will show the images being built and eventually starting. After the initial build, you will see the names of the containers being created:<\/p>\nPulling postgres (postgres:9.6.2-alpine)...\r\n9.6.2-alpine: Pulling from library\/postgres\r\n627beaf3eaaf: Pull complete\r\ne351d01eba53: Pull complete\r\ncbc11f1629f1: Pull complete\r\n2931b310bc1e: Pull complete\r\n2996796a1321: Pull complete\r\nebdf8bbd1a35: Pull complete\r\n47255f8e1bca: Pull complete\r\n4945582dcf7d: Pull complete\r\n92139846ff88: Pull complete\r\nDigest: sha256:7f3a59bc91a4c80c9a3ff0430ec012f7ce82f906ab0a2d7176fcbbf24ea9f893\r\nStatus: Downloaded newer image for postgres:9.6.2-alpine\r\nBuilding web\r\n...\r\nCreating nodejsexpresstodoapp_postgres_1\r\nCreating nodejsexpresstodoapp_web_1\r\n...\r\nweb_1 | Your app is running on port 3000<\/pre>\n
docker-compose up -d<\/code>. During development, I prefer to run without -d<\/code> and create a second terminal window to run other commands. If you want to run it as a background process and view the logs, you can run docker-compose logs<\/code>.<\/p>\ndocker-compose ps<\/code> to view your running containers. You should see something like the following:<\/p>\nName Command State Ports\r\n------------------------------------------------------------------------------------------------\r\nnodejsexpresstodoapp_postgres_1 docker-entrypoint.sh postgres Up 5432\/tcp\r\nnodejsexpresstodoapp_web_1 npm run dev Up 0.0.0.0:3000->3000\/tcp<\/pre>\n
nodejsexpresstodoapp_web_1<\/code> has listed the port as 0.0.0.0:3000->3000\/tcp<\/code>. This tells us that you can access the application using localhost:3000\/todos<\/code> on the host machine.<\/p>\n\/> curl localhost:3000\/todos\r\n\r\n[]<\/pre>\n
package.json<\/code> file has a script to automatically build the code and migrate the schema to PostgreSQL. The schema and all of the data in the container will persist as long as the postgres:9.6.2-alpine<\/code> image is not removed.<\/p>\ndocker-compose down<\/code>, which will clear things that are built and let you see what is happening with a fresh start.<\/p>\nTesting the Application<\/h2>\n
jest<\/code><\/a>. There are various ways to go about testing, including creating something like Dockerfile.test<\/code> and docker-compose.test.yml<\/code> files specific for the test environment. That\u2019s a bit beyond the current scope of this article, but I want to show you how to run the tests using the current setup.<\/p>\nnodejsexpresstodoapp<\/code>. This is a default from the directory name. If we attempt to run commands, it will use the same project, and containers will restart. This is what we don\u2019t want.<\/p>\n\/> docker-compose -p tests run -p 3000 --rm web npm run watch-tests<\/pre>\n
jest<\/code> run through integration tests and wait for changes.<\/p>\ndocker-compose<\/code> command accepts several options<\/a>, followed by a command. In this case, you are using -p tests<\/code> to run the services under the tests<\/code> project name. The command being used is run<\/code><\/a>, which will execute a one-time command against a service.<\/p>\ndocker-compose.yml<\/code> file specifies a port, we use -p 3000<\/code> to create a random port to prevent port collision. The --rm<\/code> option will remove the containers when we stop the containers. Finally, we are running in the web<\/code> service npm run watch-tests<\/code>.<\/p>\nConclusion<\/h2>\n