Introduction to Docker

Compose
Content source: https://www.baeldung.com/docker-compose
Estimated reading time : 15 Minutes

1. Overview
When using Docker extensively, the management of several different containers quickly
becomes cumbersome.

Docker Compose is a tool that helps us overcome this problem and easily handle multiple
containers at once.

In this tutorial, we'll have a look at its main features and powerful mechanisms.

2. The YAML Configuration Explained

In short, Docker Compose works by applying many rules declared within a single docker-
compose.yml configuration file.

These YAML rules, both human-readable and machine-optimized, provide us an effective way
to snapshot the whole project from ten-thousand feet in a few lines.

Almost every rule replaces a specific Docker command so that in the end we just need to run:

1 docker-compose up

We can get dozens of configurations applied by Compose under the hood. This will save us
the hassle of scripting them with Bash or something else.

In this file, we need to specify the version of the Compose file format, at least one service, and
optionally volumes and networks:

1 version: "3.7"
2 services:
3 ...
4 volumes:
5 ...
networks:
6 ...
7

Let's see what these elements actually are.

Cloud Computing – Docker Compose 1

 2.1. Services

First of all, services refer to containers' configuration.

For example, let's take a dockerized web application consisting of a front end, a back end, and
a database: We'd likely split those components into three images and define them as three
different services in the configuration:

1
2 services:
frontend:
3 image: my-vue-app
4 ...
5 backend:
6 image: my-springboot-app
...
7
db:
8 image: postgres
9 ...
10

There are multiple settings that we can apply to services, and we'll explore them deeply later
on.

2.2. Volumes & Networks

Volumes, on the other hand, are physical areas of disk space shared between the host and a
container, or even between containers. In other words, a volume is a shared directory in
the host, visible from some or all containers.

Similarly, networks define the communication rules between containers, and between a
container and the host. Common network zones will make containers' services discoverable
by each other, while private zones will segregate them in virtual sandboxes.

Again, we'll learn more about them in the next section.

3. Dissecting a Service
Let's now begin to inspect the main settings of a service.

3.1. Pulling an Image

Sometimes, the image we need for our service has already been published (by us or by others)
in Docker Hub, or another Docker Registry.

If that's the case, then we refer to it with the image attribute, by specifying the image name
and tag:

Cloud Computing – Docker Compose 2

 1 services:
2 my-service:
3 image: ubuntu:latest
4 ...

3.2. Building an Image

Instead, we might need to build an image from the source code by reading its Dockerfile.

This time, we'll use the build keyword, passing the path to the Dockerfile as the value:

1 services:
2 my-custom-app:
3 build: /path/to/dockerfile/
4 ...

We can also use a URL instead of a path:

1 services:
2 my-custom-app:
3 build: https://github.com/my-company/my-project.git
...
4

Additionally, we can specify an image name in conjunction with the build attribute, which will
name the image once created, making it available to be used by other services:

1 services:
2 my-custom-app:
3 build: https://github.com/my-company/my-project.git
4 image: my-project-image
...
5

3.3. Configuring the Networking

Docker containers communicate between themselves in networks created, implicitly

or through configuration, by Docker Compose. A service can communicate with another
service on the same network by simply referencing it by container name and port (for
example network-example-service:80), provided that we've made the port accessible through
the expose keyword:

1 services:
2 network-example-service:
3 image: karthequian/helloworld:latest
4 expose:
- "80"
5

In this case, by the way, it would also work without exposing it, because the expose directive
is already in the image Dockerfile.

Cloud Computing – Docker Compose 3

 To reach a container from the host, the ports must be exposed declaratively through
the ports keyword, which also allows us to choose if exposing the port differently in the host:

1
2 services:
3 network-example-service:
4 image: karthequian/helloworld:latest
5 ports:
- "80:80"
6 ...
7 my-custom-app:
8 image: myapp:latest
9 ports:
- "8080:3000"
10
...
11 my-custom-app-replica:
12 image: myapp:latest
13 ports:
14 - "8081:3000"
...
15
16

Port 80 will now be visible from the host, while port 3000 of the other two containers will be
available on ports 8080 and 8081 in the host. This powerful mechanism allows us to run
different containers exposing the same ports without collisions.

Finally, we can define additional virtual networks to segregate our containers:

1
2
services:
3 network-example-service:
4 image: karthequian/helloworld:latest
5 networks:
6 - my-shared-network
...
7 another-service-in-the-same-network:
8 image: alpine:latest
9 networks:
10 - my-shared-network
11 ...
another-service-in-its-own-network:
12 image: alpine:latest
13 networks:
14 - my-private-network
15 ...
16 networks:
my-shared-network: {}
17 my-private-network: {}
18
19

In this last example, we can see that another-service-in-the-same-network will be able to ping
and to reach port 80 of network-example-service, while another-service-in-its-own-
network won't.

Cloud Computing – Docker Compose 4

 3.4. Setting Up the Volumes

There are three types of volumes: anonymous, named, and host ones.

Docker manages both anonymous and named volumes, automatically mounting them in
self-generated directories in the host. While anonymous volumes were useful with older
versions of Docker (pre 1.9), named ones are the suggested way to go nowadays. Host
volumes also allow us to specify an existing folder in the host.

We can configure host volumes at the service level and named volumes in the outer level of
the configuration, in order to make the latter visible to other containers and not only to the
one they belong:

1
2 services:
3 volumes-example-service:
4 image: alpine:latest
volumes:
5 - my-named-global-volume:/my-volumes/named-global-volume
6 - /tmp:/my-volumes/host-volume
7 - /home:/my-volumes/readonly-host-volume:ro
8 ...
9 another-volumes-example-service:
image: alpine:latest
10 volumes:
11 - my-named-global-volume:/another-path/the-same-named-global-volume
12 ...
13 volumes:
14 my-named-global-volume:
15

Here, both containers will have read/write access to the my-named-global-volume shared
folder, no matter the different paths they've mapped it to. The two host volumes, instead, will
be available only to volumes-example-service.

The /tmp folder of the host's file system is mapped to the /my-volumes/host-volume folder of
the container.
This portion of the file system is writeable, which means that the container can not only read
but also write (and delete) files in the host machine.

We can mount a volume in read-only mode by appending :ro to the rule, like for
the /home folder (we don't want a Docker container erasing our users by mistake).

3.5. Declaring the Dependencies

Often, we need to create a dependency chain between our services, so that some services get
loaded before (and unloaded after) other ones. We can achieve this result through
the depends_on keyword:

Cloud Computing – Docker Compose 5

 1
services:
2
kafka:
3 image: wurstmeister/kafka:2.11-0.11.0.3
4 depends_on:
5 - zookeeper
6 ...
zookeeper:
7 image: wurstmeister/zookeeper
8 ...
9

We should be aware, however, that Compose will not wait for the zookeeper service to finish
loading before starting the kafka service: it will simply wait for it to start. If we need a service
to be fully loaded before starting another service, we need to get deeper control of startup
and shutdown order in Compose.

4. Managing Environment Variables

Working with environment variables is easy in Compose. We can define static environment
variables, and also define dynamic variables with the ${} notation:

1 services:
2 database:
3 image: "postgres:${POSTGRES_VERSION}"
4 environment:
DB: mydb
5
USER: "${USER}"
6

There are different methods to provide those values to Compose.

For example, one is setting them in a .env file in the same directory, structured like
a .properties file, key=value:

1 POSTGRES_VERSION=alpine
2 USER=foo

Otherwise, we can set them in the OS before calling the command:

1 export POSTGRES_VERSION=alpine
2 export USER=foo
3 docker-compose up

Finally, we might find handy using a simple one-liner in the shell:

1 POSTGRES_VERSION=alpine USER=foo docker-compose up

We can mix the approaches, but let's keep in mind that Compose uses the following priority
order, overwriting the less important with the higher ones:

1. Compose file
2. Shell environment variables

Cloud Computing – Docker Compose 6

 3. Environment file
4. Dockerfile
5. Variable not defined

5. Scaling & Replicas

In older Compose versions, we were allowed to scale the instances of a container through
the docker-compose scale command. Newer versions deprecated it and replaced it with the ––
scale option.

On the other side, we can exploit Docker Swarm – a cluster of Docker Engines – and autoscale
our containers declaratively through the replicas attribute of the deploy section:

1
2 services:
3 worker:
4 image: dockersamples/examplevotingapp_worker
5 networks:
- frontend
6 - backend
7 deploy:
8 mode: replicated
9 replicas: 6
10 resources:
limits:
11 cpus: '0.50'
12 memory: 50M
13 reservations:
14 cpus: '0.25'
15 memory: 20M
...
16
17

Under deploy, we can also specify many other options, like the resources thresholds.
Compose, however, considers the whole deploy section only when deploying to Swarm,
and ignores it otherwise.

6. A Real-World Example: Spring Cloud Data Flow

While small experiments help us understanding the single gears, seeing the real-world code
in action will definitely unveil the big picture.

Spring Cloud Data Flow is a complex project, but simple enough to be understandable.
Let's download its YAML file and run:

1 DATAFLOW_VERSION=2.1.0.RELEASE SKIPPER_VERSION=2.0.2.RELEASE docker-compose up

Compose will download, configure, and start every component, and then intersect the
container's logs into a single flow in the current terminal.

Cloud Computing – Docker Compose 7

 It'll also apply unique colors to each one of them for a great user experience:

We might get the following error running a brand new Docker Compose installation:

1 lookup registry-1.docker.io: no such host

While there are different solutions to this common pitfall, using 8.8.8.8 as DNS is probably the
simplest.

7. Lifecycle Management
Let's finally take a closer look at the syntax of Docker Compose:

1 docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]

While there are many options and commands available, we need at least to know the ones to
activate and deactivate the whole system correctly.

7.1. Startup

We've seen that we can create and start the containers, the networks, and the volumes
defined in the configuration with up:

1 docker-compose up

After the first time, however, we can simply use start to start the services:

1 docker-compose start

Cloud Computing – Docker Compose 8

 In case our file has a different name than the default one (docker-compose.yml), we can exploit
the -f and ––file flags to specify an alternate file name:

1 docker-compose -f custom-compose-file.yml start

Compose can also run in the background as a daemon when launched with the -d option:

1 docker-compose up -d

7.2. Shutdown

To safely stop the active services, we can use stop, which will preserve containers, volumes,
and networks, along with every modification made to them:

1 docker-compose stop

To reset the status of our project, instead, we simply run down, which will destroy
everything with only the exception of external volumes:

1 docker-compose down

8. Conclusion
In this tutorial, we've learned about Docker Compose and how it works.

As usual, we can find the source docker-compose.yml file on GitHub, along with a helpful
battery of tests immediately available in the following image:

Cloud Computing – Docker Compose 9

Cloud Computing – Docker Compose 10