⚠️ 👉 NOTE: this proposal has been replaced by #9694.

This proposal, replacing #9175, is to bring straightforward, Fig-inspired stack composition to the Docker client. The ultimate goal is to provide an out-of-the-box Docker development experience that is:

1. delightful
2. eventually suitable for production orchestration in the “80% case”
3. compatible with Docker clustering

(The previous proposal built on the docker groups proposal, #8637. This proposal does not, as I've determined that - since groups aren't necessary for implementing composition - it's preferable to avoid making changes or additions to the Docker API.)

I’ve already implemented an alpha of the required functionality on my composition branch - though it’s not ready for prime time, everyone is very much encouraged to try it out, especially Fig users. Scroll down for test builds!

The basic idea of stack composition is that with a very simple configuration file which describes what containers you want your application to consist of, you can type a single command and Docker will do everything necessary to get it running.

Configuration file

A group.yml is used to describe your application. It looks a lot like fig.yml, except that what Fig calls the “project name” is specified explicitly:

name: rails_example

containers:
  db:
    image: postgres:latest
  web:
    build: .
    command: bundle exec rackup -p 3000
    volumes:
      - .:/myapp
    ports:
      - "3000:3000"
    links:
      - db

A container entry must specify either an image to be pulled or a build directory (but not both). Other configuration options mostly map to their docker run counterparts.

docker up

There’s a new docker up command. It performs the following steps:

1. Parse group.yml
2. For each defined container, check whether we have an image for it. If not, either build or pull it as needed.
3. Create/recreate the defined containers. (For now, containers are always recreated if they already exist - this is the simplest way to ensure changes to group.yml are picked up.)
4. Start the containers in dependency order (based on links and volumes_from declarations in group.yml).
5. Unless docker up was invoked with -d, attach to all containers and stream their aggregated log output until the user sends Ctrl-C, at which point attempt to stop all containers with SIGTERM. Subsequent Ctrl-Cs result in a SIGKILL.

Enhancements to existing CLI commands

An optional NAME_PREFIX argument is added to docker ps to allow filtering of containers based on name prefix (on the client side, initially).

A new syntax is introduced as a shorthand for referring to containers, images and build directories:

• :web designates the container named web in group.yml.
• : designates all containers defined in group.yml. Depending on the exact command being invoked, this is restricted to containers which currently exist on the host, or those whose image has been built.

Here are some example commands with their longwinded/non-portable equivalents.

List our containers:

$ docker ps rails_example_
$ docker ps :

Rebuild the web image:

$ docker build -t rails_example_web .
$ docker build :web

Re-pull the db image:

$ docker pull postgres:latest
$ docker pull :db

Kill the web container:

$ docker kill rails_example_web
$ docker kill :web

Kill all containers:

$ docker kill rails_example_web rails_example_db
$ docker kill :

Kill and remove all containers:

$ docker rm -f rails_example_web rails_example_db
$ docker rm -f :

Delete the web image:

$ docker rmi rails_example_web
$ docker rmi :web

Open a bash shell in the web container:

$ docker exec -ti rails-example/web bash
$ docker exec :web bash

Run a one-off container using web’s image and configuration:

$ docker build -t rails_example_web . && docker run -ti -v `pwd`:/myapp --link rails_example_db:db rails_example_web bash
$ docker run -ti :web bash

Topics for discussion: an inexhaustive list

Including the app name in the file. I’m unsure about making this the default - lots of Fig users want to be able to do it, but I’m worried that it’ll hurt portability (we don’t do it with Dockerfile, and in my opinion it’s better off for it). Alternate approaches include using the basename of the current directory (like Fig does), or generating a name and storing it in a separate, unversioned file.

Clustering and production. People are already deploying single-host production sites with fig up -d, validating this general approach in simple scenarios, but we need to be sure that it’ll port well to a clustered Docker instance.

Scaling. I don't think an equivalent to fig scale is necessary on day one, but it will eventually be needed as Docker becomes a multi-host platform, so there shouldn't be anything in the design that'll make that difficult to implement later.

Test builds

• linux/amd64: http://cl.ly/0Q3G1l2t301S/download/docker-1.3.2-dev-linux
• darwin/386: http://cl.ly/1J1h0p2Q0J2m/download/docker-1.3.2-dev-darwin-386
• darwin/amd64: http://cl.ly/1J1G323h3d3T/download/docker-1.3.2-dev-darwin-amd64

Here's how to test it out if you're running boot2docker. First, replace the binary in your VM:

$ boot2docker ssh
docker@boot2docker:~$ sudo -i
root@boot2docker:~# curl -LO http://cl.ly/0Q3G1l2t301S/download/docker-1.3.2-dev-linux
root@boot2docker:~# /etc/init.d/docker stop
root@boot2docker:~# mv /usr/local/bin/docker ./docker-stable
root@boot2docker:~# mv ./docker-1.3.2-dev-linux /usr/local/bin/docker
root@boot2docker:~# chmod +x /usr/local/bin/docker
root@boot2docker:~# /etc/init.d/docker start
root@boot2docker:~# docker version   # both "Git commit"s should be c6bf574
root@boot2docker:~# exit
docker@boot2docker:~$ exit

Next, replace your client binary:

$ curl -LO http://cl.ly/1J1G323h3d3T/download/docker-1.3.2-dev-darwin-amd64
$ mv /usr/local/bin/docker ./docker-stable
$ mv ./docker-1.3.2-dev-darwin-amd64 /usr/local/bin/docker
$ chmod +x /usr/local/bin/docker
$ docker version                     # both "Git commit"s should be c6bf574

Not yet implemented

There are a few things left to implement:

• Supporting volumes_from
• Validation of the YAML file
• Code cleanup
• Test coverage

Example app 1: Python/Redis counter

Here’s a sample app you can try:

app.py:

from flask import Flask
from redis import Redis

app = Flask(__name__)
redis = Redis(host="redis", port=6379)

@app.route('/')
def hello():
    redis.incr('hits')
    return 'Hello World! I have been seen %s times.' % redis.get('hits')

if __name__ == "__main__":
    app.run(host="0.0.0.0", debug=True)

requirements.txt:

flask
redis

Dockerfile:

FROM python:2.7
ADD . /code
WORKDIR /code
RUN pip install -r requirements.txt

group.yml:

name: counter

containers:
  web:
    build: .
    command: python app.py
    ports:
      - "5000:5000"
    volumes:
      - .:/code
    links:
      - redis
    environment:
      - PYTHONUNBUFFERED=1
  redis:
    image: redis:latest
    command: redis-server --appendonly yes

If you put those four files in a directory and type docker up, you should see everything start up:

It'll build the web image, pull the redis image, start both containers and stream their aggregated output. If you Ctrl-C, it'll shut them down.

Example app 2: Fresh Rails app

I’ve ported the Rails example from Fig. See composing_rails.md.

Code

To get hacking, check out the composition branch on my fork:

# if you don’t already have Docker cloned
$ git clone [email protected]:docker/docker
$ cd docker

$ git remote add aanand [email protected]:aanand/docker.git
$ git fetch --all
$ git checkout -b composition aanand/composition