Dockerd eats too much RAM #39916
Description
Description
Currently the dockerd process of a leader node is eating ~11GB of RAM.
On that node there are no scheduled containers, it participates to the cluster only as manager (currently as leader).
Output of docker version:
root@vim11:/home/goetas# docker version
Client:
Version: 18.09.8
API version: 1.39
Go version: go1.10.8
Git commit: 0dd43dd87f
Built: Wed Jul 17 17:41:13 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.12)
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:20:35 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
Output of docker info:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 46
Server Version: 19.03.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: 27ajn65e9o1zfla8j532wbt5b
Is Manager: true
ClusterID: tzm417keomjtxndqclyvsz6fd
Managers: 3
Nodes: 13
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Orchestration:
Task History Retention Limit: 3
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 10 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 172.16.21.231
Manager Addresses:
172.16.21.231:2377
172.16.21.233:2377
172.16.21.238:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.0-9-amd64
Operating System: Debian GNU/Linux 9 (stretch)
OSType: linux
Architecture: x86_64
CPUs: 3
Total Memory: 15.68GiB
Name: vim11
ID: CU7J:E35G:YUKM:76S7:LYCX:2XJD:Y46P:VBXT:HD5A:UYXT:44AR:UH4Q
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
root@vim11:/home/goetas# uname -a
Linux vim11 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u3 (2019-06-16) x86_64 GNU/Linux
I saw that there are #32728 #32711 , but they refer to older versions of docker. This one is pretty recent.
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 31186 76.5 72.1 24888568 11781872 ? Ssl Aug01 46401:24 /usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --ip-forward=true --iptables=true
How can I provide you more info to debug this?