Description
Recreating a Docker swarm with docker swarm leave -f and docker swarm init a couple of times leads to the following error in docker.log: Err: listen tcp 0.0.0.0:7946: bind: address already in use
Joining workers to such a swarm manager then leads to Mar 16 11:16:52 secondblue dockerd[758]: time="2017-03-16T11:16:52.145876195Z" level=warning msg="memberlist: failed to receive: No installed keys could decrypt the message from=192.168.17.112:53924"
I came across this problem while writing a helper script to reset a (real hardwar) demo environment again and again. But it can be reproduced with Docker Machine.
Steps to reproduce the issue:
docker-machine create -d virtualbox tsteval $(docker-machine env tst)docker swarm init --advertise-addr 192.168.99.100docker swarm leave -fdocker swarm init --advertise-addr 192.168.99.100docker swarm leave -fdocker swarm init --advertise-addr 192.168.99.100docker swarm leave -fdocker-machine ssh tstcat /var/log/docker.log
In parallel looking on the ports listening in the machine, the port 7946 is still listening after leaving the swarm:
$ docker-machine ssh tst
docker@tst:~$ netstat -an | grep 7946
tcp 0 0 :::7946 :::* LISTEN
udp 0 0 :::7946 :::*
Describe the results you received:
time="2017-03-17T07:37:17.975482519Z" level=info msg="Gossip cluster hostname tst-00b488f70cfd"
time="2017-03-17T07:37:17.975500395Z" level=debug msg="Encryption key 1: 6fc7c"
time="2017-03-17T07:37:17.975528348Z" level=debug msg="Encryption key 2: d51b0"
time="2017-03-17T07:37:17.975533599Z" level=debug msg="Encryption key 3: 8681a"
time="2017-03-17T07:37:17.975608225Z" level=error msg="Error in agentInit : failed to create memberlist: Failed to start TCP listener. Err: listen tcp 0.0.0.0:7946: bind: address already in use"
time="2017-03-17T07:37:17.976687877Z" level=debug msg="Root CA updated successfully" cluster.id=i1q2pbhbd31n02a39oo1ovuj6 method="(*Server).updateCluster" module=ca
time="2017-03-17T07:37:17.977759203Z" level=debug msg="Calling GET /v1.26/swarm"
time="2017-03-17T07:37:17.978666940Z" level=debug msg="Calling GET /v1.26/nodes/jl5321ooxdhr5kths3f1ai904"
time="2017-03-17T07:37:21.224266845Z" level=error msg="failed to send node leave: timed out broadcasting node event"
time="2017-03-17T07:37:21.342478674Z" level=debug msg="checkEncryption(8jlu4ma, <nil>, 0, true)"
time="2017-03-17T07:37:21.372807365Z" level=debug msg="Revoking external connectivity on endpoint gateway_ingress-sbox (8d8be82e3a664e223b320217070052bf21cfc08e9997f5cb1e96671a16d926b3)"
time="2017-03-17T07:37:21.443056711Z" level=debug msg="Releasing addresses for endpoint gateway_ingress-sbox's interface on network docker_gwbridge"
time="2017-03-17T07:37:21.443085904Z" level=debug msg="ReleaseAddress(LocalDefault/172.18.0.0/16, 172.18.0.2)"
time="2017-03-17T07:37:21.472006338Z" level=debug msg="Releasing addresses for endpoint ingress-endpoint's interface on network ingress"
time="2017-03-17T07:37:21.472074337Z" level=debug msg="ReleaseAddress(LocalDefault/10.255.0.0/16, 10.255.0.3)"
time="2017-03-17T07:37:21.473235690Z" level=debug msg="releasing IPv4 pools from network ingress (8jlu4majgdvqiercuqqqhatgw)"
time="2017-03-17T07:37:21.473255783Z" level=debug msg="ReleaseAddress(LocalDefault/10.255.0.0/16, 10.255.0.1)"
time="2017-03-17T07:37:21.473269962Z" level=debug msg="ReleasePool(LocalDefault/10.255.0.0/16)"
Describe the results you expected:
Should work multiple times.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client:
Version: 17.03.0-ce
API version: 1.26
Go version: go1.7.5
Git commit: 60ccb22
Built: Thu Feb 23 10:40:59 2017
OS/Arch: darwin/amd64
Server:
Version: 17.03.0-ce
API version: 1.26 (minimum version 1.12)
Go version: go1.7.5
Git commit: 3a232c8
Built: Tue Feb 28 07:52:04 2017
OS/Arch: linux/amd64
Experimental: false
Output of docker info:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 17.03.0-ce
Storage Driver: aufs
Root Dir: /mnt/sda1/var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 0
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: jl5321ooxdhr5kths3f1ai904
Is Manager: true
ClusterID: i1q2pbhbd31n02a39oo1ovuj6
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 192.168.99.100
Manager Addresses:
192.168.99.100:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.4.52-boot2docker
Operating System: Boot2Docker 17.03.0-ce (TCL 7.2); HEAD : f11a204 - Thu Mar 2 00:14:47 UTC 2017
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.8 MiB
Name: tst
ID: DI4T:HKXV:ZQJE:UGL4:EWIC:LQE3:72NV:YSTO:T5DU:6JAC:H5R3:6JLF
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 25
Goroutines: 125
System Time: 2017-03-17T07:54:40.549954175Z
EventsListeners: 0
Username: stefanschererseal
Registry: https://index.docker.io/v1/
Labels:
provider=virtualbox
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
VirtualBox, Raspberry Pi
Description
Recreating a Docker swarm with
docker swarm leave -fanddocker swarm inita couple of times leads to the following error in docker.log:Err: listen tcp 0.0.0.0:7946: bind: address already in useJoining workers to such a swarm manager then leads to
Mar 16 11:16:52 secondblue dockerd[758]: time="2017-03-16T11:16:52.145876195Z" level=warning msg="memberlist: failed to receive: No installed keys could decrypt the message from=192.168.17.112:53924"I came across this problem while writing a helper script to reset a (real hardwar) demo environment again and again. But it can be reproduced with Docker Machine.
Steps to reproduce the issue:
docker-machine create -d virtualbox tsteval $(docker-machine env tst)docker swarm init --advertise-addr 192.168.99.100docker swarm leave -fdocker swarm init --advertise-addr 192.168.99.100docker swarm leave -fdocker swarm init --advertise-addr 192.168.99.100docker swarm leave -fdocker-machine ssh tstcat /var/log/docker.logIn parallel looking on the ports listening in the machine, the port 7946 is still listening after leaving the swarm:
Describe the results you received:
Describe the results you expected:
Should work multiple times.
Additional information you deem important (e.g. issue happens only occasionally):
Output of
docker version:Output of
docker info:Additional environment details (AWS, VirtualBox, physical, etc.):
VirtualBox, Raspberry Pi