Container hangs when using fluentd logger #40063
Description
Description
Docker container hangs when using fluentd logger with fluentd-async-connect=true and unreachable (turned off) fluentd server.
Steps to reproduce the issue:
- Create docker-compose.yml with any container and logger set to
logging:
driver: fluentd
options:
fluentd-address: 10.147.49.79:24224
fluentd-async-connect: 'true'
fluentd-retry-wait: '1s'
fluentd-max-retries: '30'
tag: your_app
docker-compose up --build -d
Describe the results you received:
Docker container hangs, does not respond to docker-compose down -f, docker stop, docker kill.
Describe the results you expected:
Container exits with error message.
Additional information you deem important (e.g. issue happens only occasionally):
Issue happens every time the remote fluentd server is down or unreachable.
Output of docker version:
Client: Docker Engine - Community
Version: 19.03.2
API version: 1.40
Go version: go1.12.8
Git commit: 6a30dfc
Built: Thu Aug 29 05:28:55 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.2
API version: 1.40 (minimum version 1.12)
Go version: go1.12.8
Git commit: 6a30dfc
Built: Thu Aug 29 05:27:34 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
Output of docker info:
Client:
Debug Mode: false
Server:
Containers: 30
Running: 29
Paused: 0
Stopped: 1
Images: 47
Server Version: 19.03.2
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-957.27.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 64
Total Memory: 125.3GiB
Name: server55
ID: SHMQ:YUKH:MZOS:EEYR:WZI6:JYVH:NPPJ:CEEQ:ILXV:6JNU:BVBG:2NTE
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled```
**Additional environment details (AWS, VirtualBox, physical, etc.):**
physical