Prerequisites<\/h2>\n
To follow this post and execute all the examples, you will need the following:<\/p>\n
- \n
- Docker Machine >= 0.7.0<\/strong>: to provision Docker engines. You can obtain Docker Machine here<\/a>.<\/li>\n
- Docker Compose >= 1.7.1<\/strong>: to orchestrate the application\u2019s services.<\/li>\n
- Docker client >= 1.11.1<\/strong>: for talking with the Swarm manager. You can obtain the Docker client here<\/a>.<\/li>\n
- Digital Ocean API Token<\/strong>: to allow docker-machine to create machines on which to provision its engines. You can generate your token here<\/a>.<\/li>\n<\/ul>\n
Creating the Swarm cluster<\/h3>\n
To begin, we need a Swarm cluster with these characteristics:<\/p>\n
- \n
- At least one public-facing node to host the NGINX proxy. Multiple nodes can be created and balanced using DNS.<\/li>\n
- At least one node to host the Swarm manager. Swarm has built-in HA, which is not needed for this tutorial. Implementing it is left as an exercise for the reader; see here<\/a>.<\/li>\n
- At least one node to host the key\/value datastore. Here I used Consul. As for the Swarm manager, you might want to have at least three nodes for this in production. This kind of node is not in the Swarm cluster.<\/li>\n<\/ul>\n
To make this step easier, I created a script that creates all the required nodes on Digital Ocean.<\/p>\n
#!\/bin\/bash\r\n\r\n# nodes to create configuration\r\npublic_nodes=\"public01\"\r\nnodes=\"node01 node02 node03\"\r\n\r\n# Create the KV node using consul\r\ndocker-machine create \\\r\n -d digitalocean \\\r\n --digitalocean-access-token=$DO_ACCESS_TOKEN \\\r\n consul\r\n\r\ndocker-machine ssh consul docker run -d \\\r\n -p \"8500:8500\" \\\r\n -h \"consul\" \\\r\n progrium\/consul -server -bootstrap\r\n\r\nKV_IP=$(docker-machine ip consul)\r\nKV_ADDR=\"consul:\/\/${KV_IP}:8500\"\r\n\r\n# Create the Swarm Manager\r\necho \"Create swarm manager\"\r\ndocker-machine create \\\r\n -d digitalocean \\\r\n --digitalocean-access-token=$DO_ACCESS_TOKEN \\\r\n --swarm --swarm-master \\\r\n --swarm-discovery=$KV_ADDR \\\r\n --engine-opt=\"cluster-store=${KV_ADDR}\" \\\r\n --engine-opt=\"cluster-advertise=eth0:2376\" \\\r\n manager\r\n\r\n# Create Public facing Swarm nodes\r\nfor node in $public_nodes; do\r\n (\r\n echo \"Creating ${node}\"\r\n\r\n docker-machine create \\\r\n -d digitalocean \\\r\n --digitalocean-size \"4gb\" \\\r\n --engine-label public=yes \\\r\n --digitalocean-access-token=$DO_ACCESS_TOKEN \\\r\n --swarm \\\r\n --swarm-discovery=$KV_ADDR \\\r\n --engine-opt=\"cluster-store=${KV_ADDR}\" \\\r\n --engine-opt=\"cluster-advertise=eth0:2376\" \\\r\n $node\r\n ) &\r\ndone\r\nwait\r\n\r\n# Create other Swarm nodes\r\nfor node in $nodes; do\r\n (\r\n echo \"Creating ${node}\"\r\n\r\n docker-machine create \\\r\n -d digitalocean \\\r\n --digitalocean-size \"2gb\" \\\r\n --engine-label public=no \\\r\n --digitalocean-access-token=$DO_ACCESS_TOKEN \\\r\n --swarm \\\r\n --swarm-discovery=$KV_ADDR \\\r\n --engine-opt=\"cluster-store=${KV_ADDR}\" \\\r\n --engine-opt=\"cluster-advertise=eth0:2376\" \\\r\n $node\r\n ) &\r\ndone\r\nwait\r\n\r\n# Print Cluster Information\r\necho \"\"\r\necho \"CLUSTER INFORMATION\"\r\necho \"Consul UI: http:\/\/${KV_IP}:8500\"\r\necho \"Environment variables to connect trough docker cli\"\r\ndocker-machine env --swarm manager<\/pre>\nJust copy the script to a file named
create-swarm-cluster.sh<\/code> and give execution permissions withchmod +x create-swarm-cluster.sh<\/code>. To execute the script, you will need to give it the previously generated Digital Ocean API token.<\/p>\nexport DO_ACCESS_TOKEN=<your-digitalocean-api-token>\r\n.\/create-swarm-cluster.sh<\/pre>\n
This is how our cluster looks:<\/p>\n
![\"cluster\"](\"http:\/\/www.systemcodegeeks.com\/wp-content\/uploads\/2016\/06\/cluster-1024x588.png\")
<\/a><\/p>\nNow that we have our cluster, we have to export the right environment variables to connect to it using our local Docker client. To do that:<\/p>\n
eval $(docker-machine env --swarm manager)<\/pre>\n
To verify that you are connected to the Swarm cluster:<\/p>\n
docker info | grep \"Server Version:\"<\/pre>\n
That should output something like:<\/p>\n
Server Version: swarm\/1.2.3<\/pre>\n
Running Your Application into the Cluster<\/h2>\n
Now that our Swarm cluster is ready, we just need to start our application. For this purpose, I chose the super cool Cats vs Dogs Voting Demo Application<\/a>.<\/p>\n
The application consists of three parts:<\/p>\n
- \n
- The Voting Application, where you actually choose between cats and dogs<\/li>\n
- The Result Application<\/li>\n
- The worker, in charge of persisting votes in the Postgres database<\/li>\n<\/ul>\n
For dependencies, it has Postgres and Redis databases.<\/p>\n
In order to run our application, we\u2019re going to need a
docker-compose.yml<\/code> file that will start an instance of each microservice and the dependencies.<\/p>\nversion: \"2\"\r\n\r\nservices:\r\n voting-app:\r\n image: docker\/example-voting-app-voting-app:latest\r\n ports:\r\n - \"80\"\r\n links:\r\n - redis\r\n networks:\r\n - front-tier\r\n - back-tier\r\n\r\n result-app:\r\n image: docker\/example-voting-app-result-app:latest\r\n ports:\r\n - \"80\"\r\n links:\r\n - db\r\n networks:\r\n - front-tier\r\n - back-tier\r\n\r\n worker:\r\n image: docker\/example-voting-app-worker:latest\r\n networks:\r\n - back-tier\r\n\r\n redis:\r\n image: redis:alpine\r\n ports: [\"6379\"]\r\n networks:\r\n - back-tier\r\n\r\n db:\r\n image: postgres:9.5\r\n volumes:\r\n - \"db-data:\/var\/lib\/postgresql\/data\"\r\n networks:\r\n - back-tier\r\n\r\nvolumes:\r\n db-data:\r\n\r\nnetworks:\r\n front-tier:\r\n back-tier:<\/pre>\n
If you start it, you should obtain something like this:<\/p>\n
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\r\n266c4ec6c51d docker\/example-voting-app-result-app:latest \"node server.js\" 14 seconds ago Up 10 seconds 104.236.XXX.XXX:32768->80\/tcp node01\/nginxreverseproxy_result-app_1\r\n8cefe6ad38b9 docker\/example-voting-app-voting-app:latest \"python app.py\" 14 seconds ago Up 11 seconds 104.236.XXX.XXX:32769->80\/tcp node02\/nginxreverseproxy_voting-app_1\r\ne04fa43c9909 redis:alpine \"docker-entrypoint.sh\" 17 seconds ago Up 14 seconds 104.236.XXX.XXX:32768->6379\/tcp node02\/nginxreverseproxy_redis_1\r\n8e802ff973aa postgres:9.5 \"\/docker-entrypoint.s\" 17 seconds ago Up 14 seconds 5432\/tcp node01\/nginxreverseproxy_db_1\r\ne434f6a9ff9c docker\/example-voting-app-worker:latest \"java -jar target\/wor\" 17 seconds ago Up 15 seconds public01\/nginxreverseproxy_worker_1<\/pre>\n
But hang on. We\u2019re not going to start
docker-compose.yml<\/code> because it\u2019s not suitable for a cluster. In fact:<\/p>\n- \n
- We don\u2019t have an entry point or a defined set of entry points which point to our DNS. If you look at the above output, you should note that things were scheduled without any specific constraint.<\/li>\n
- Our Postgres database has a mounted volume bound to the node on which the container is running. But what happens if the container is rescheduled on another node?<\/li>\n
- Front-end applications should be on the 80 or 443 ports, not on casual ones.<\/li>\n
- If you scale web applications with
docker-compose scale<\/code>, those are not load balanced.<\/li>\n<\/ul>\n!New Call-to-action<\/p>\n
So, one thing at a time: To solve the problem of the single entry point for our DNS servers, we are going to need an automated way to register our services into a proxy. In our case, we\u2019ll use NGINX and ehazlett\/interlock<\/a> for this purpose.<\/p>\n
With Interlock, each time that a service is added or scaled, it will be added into the pool of the NGINX
proxy_pass<\/code> so that NGINX can route and balance request to the right containers. In this way, the single entry point for the HTTP requests made to the cluster will be the NGINX container that\u2019s bound to thepublic01<\/code> node using the constraint:constraint:node==public01<\/code>. This also solves the problem that, when scaling containers usingdocker-compose scale<\/code>, requests across containers are balanced. You can always start more public nodes and balance them using DNS balancing.<\/p>\nFor the Postgres database, the only thing we can do here is to bind it to a specific node so that we can be sure that it is not rescheduled on another machine.<\/p>\n
Obviously this will increase the chances of failure \u2014 it\u2019s creating a single point of failure on the cluster. However, there\u2019s a way to run stateful services like databases in production by allowing your volumes to follow your containers. It\u2019s called Flocker, and unfortunately it was not suitable for this post, but you can learn more here<\/a>.<\/p>\n
To set up this interlock, you will need this
docker-compose.yml<\/code>:<\/p>\ninterlock:\r\n image: ehazlett\/interlock:master\r\n command: -D run -c \/etc\/interlock\/config.toml\r\n tty: true\r\n ports:\r\n - 8080\r\n environment:\r\n INTERLOCK_CONFIG: |\r\n ListenAddr = \":8080\"\r\n DockerURL = \"${SWARM_HOST}\"\r\n TLSCACert = \"\/etc\/docker\/ca.pem\"\r\n TLSCert = \"\/etc\/docker\/server.pem\"\r\n TLSKey = \"\/etc\/docker\/server-key.pem\"\r\n [[Extensions]]\r\n Name = \"nginx\"\r\n ConfigPath = \"\/etc\/nginx\/nginx.conf\"\r\n PidPath = \"\/var\/run\/nginx.pid\"\r\n TemplatePath = \"\"\r\n MaxConn = 1024\r\n Port = 80\r\n volumes:\r\n - \/etc\/docker:\/etc\/docker:ro\r\n\r\nnginx:\r\n image: nginx:latest\r\n entrypoint: nginx\r\n command: -g \"daemon off;\" -c \/etc\/nginx\/nginx.conf\r\n ports:\r\n - 80:80\r\n labels:\r\n - \"interlock.ext.name=nginx\"\r\n environment:\r\n - \"constraint:node==public01<\/pre>\nAs you can see, we\u2019re starting an interlock container that can connect to the Swarm cluster and updates the
\/etc\/nginx\/nginx.conf<\/code> each time it\u2019s needed. Note that the NGINX container is bound to thepublic01<\/code> node, so all our HTTP services will be accessible trough that node.<\/p>\nTo start this on your cluster:<\/p>\n
eval $(docker-machine env --swarm manager)\r\nexport SWARM_HOST=tcp:\/\/$(docker-machine ip manager):2376\r\ndocker-compose up -d<\/pre>\n
Now that our cluster is ready, we can change our application\u2019s
docker-compose.yml<\/code> to reflect our thoughts:<\/p>\nversion: \"2\"\r\n\r\nservices:\r\n voting-app:\r\n hostname: voting.local\r\n image: docker\/example-voting-app-voting-app:latest\r\n ports:\r\n - \"80\"\r\n links:\r\n - redis\r\n networks:\r\n - front-tier\r\n - back-tier\r\n labels:\r\n - \"interlock.hostname=voting\"\r\n - \"interlock.domain=local\"\r\n\r\n result-app:\r\n hostname: result.local\r\n image: docker\/example-voting-app-result-app:latest\r\n ports:\r\n - \"80\"\r\n links:\r\n - db\r\n networks:\r\n - front-tier\r\n - back-tier\r\n labels:\r\n - \"interlock.hostname=result\"\r\n - \"interlock.domain=local\"\r\n\r\n worker:\r\n image: docker\/example-voting-app-worker:latest\r\n networks:\r\n - back-tier\r\n\r\n redis:\r\n image: redis:alpine\r\n ports: [\"6379\"]\r\n networks:\r\n - back-tier\r\n\r\n db:\r\n image: postgres:9.5\r\n volumes:\r\n - \"db-data:\/var\/lib\/postgresql\/data\"\r\n networks:\r\n - back-tier\r\n environment:\r\n - \"constraint:node==node01\"\r\n\r\nvolumes:\r\n db-data:\r\n\r\nnetworks:\r\n front-tier:\r\n back-tier:<\/pre>\n
A few things are different now:<\/p>\n
- \n
- The voting and result apps now have the
hostname<\/code> and the interlock\u2019s hostname and domain labels that are used by interlock to configure NGINX.<\/li>\n- The Postgres database now is bound to the
node01<\/code>.<\/li>\n<\/ul>\nThe last thing to do is to add the IP address of the
public01<\/code> node to your DNS records. For simplicity, you can add it to your local host\u2019s file. You can obtain the right host\u2019s line with this command:<\/p>\necho $(docker-machine ip public01) voting.local result.local<\/pre>\n
Now you can just run the applications with a
docker-compose up<\/code> and point your browser to http:\/\/voting.local<\/a> to choose your favorite pet! (Pro tip: Cats are the right choice.)<\/p>\nConclusion<\/h2>\n
In this post, we achieved a few things in a relatively simple way by using just the official tools provided by Docker. The coolest achievement was that our entire cluster is now exposed as a single Docker daemon by the Swarm manager which also matches the definition of cluster you can find on Wikipedia<\/a>:<\/p>\n
A computer cluster consists of a set of loosely or tightly connected computers that work together so that, in many respects, they can be viewed as a single system.<\/p><\/blockquote>\n
\n - The Postgres database now is bound to the
- The voting and result apps now have the
- At least one node to host the key\/value datastore. Here I used Consul. As for the Swarm manager, you might want to have at least three nodes for this in production. This kind of node is not in the Swarm cluster.<\/li>\n<\/ul>\n
- Docker Compose >= 1.7.1<\/strong>: to orchestrate the application\u2019s services.<\/li>\n