Expected behavior
docker run container correctly
Actual behavior
when run: docker run, it prints 'Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/c022c470eecf0ae4f81c289505c6b45815a78caebe11f3cd0a201330b6ad4122-init/merged: no such file or directory.',
when run : docker inspect ,it prints 'Error response from daemon: readlink /var/lib/docker/overlay2/l/4GWADAHD5OC5SYHTIOEPIBN64E: no such file or directory'
Steps to reproduce the behavior
0,vdb lvcreate mount to /var/lib/docker
1, install docker(19.03.1) kubernetes(v1.15.3) ,three node cluster
2,build docker registry(2.7)
3,deploy 200 kubernetes pods and all pods are running
4,reboot vm and reboot all kubernetes pod
Output of docker version:
[root@centos66 l]# docker version
Client: Docker Engine - Community
Version: 19.03.1
API version: 1.40
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:21:07 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.12)
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:19:36 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
You have mail in /var/spool/mail/root
[root@centos66 l]# docker images|grep pause
h3crd-wlan1.chinacloudapp.cn:5000/private-cloud/pause-amd64 3.1 da86e6ba6ca1 21 months ago 742kB
You have mail in /var/spool/mail/root
[root@centos66 l]# docker run da86e6ba6ca1
docker: Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/52e57461c24b35217aff10255db24d456251e4f7a029f864053d1cdfe9acb621-init/merged: no such file or directory.
See 'docker run --help'.
See 'docker run --help'.
[root@centos66 l]# docker inspect bb4a6b774658
[]
Error response from daemon: readlink /var/lib/docker/overlay2/l/4GWADAHD5OC5SYHTIOEPIBN64E: no such file or directory
Output of docker info:
[root@centos66 l]# docker info
Client:
Debug Mode: false
Server:
Containers: 6
Running: 0
Paused: 0
Stopped: 6
Images: 155
Server Version: 19.03.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.19.12-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 57.59GiB
Name: centos66
ID: EKY6:L2LL:UC7M:62L7:RLFI:KE3L:LOFU:GQEM:X3ZK:KHSG:BX4M:ZGI3
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
h3crd-wlan1.chinacloudapp.cn:5000
127.0.0.0/8
Registry Mirrors:
https://registry.docker-cn.com/
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.)
[root@centos66 l]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
[root@centos66 l]
[root@centos66 l]# uname -a
Linux centos66 4.19.12-1.el7.elrepo.x86_64 #1 SMP Fri Dec 21 11:06:36 EST 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@centos66 l]#
[root@centos66 l]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
[root@centos66 l]#
Expected behavior
docker run container correctly
Actual behavior
when run: docker run, it prints 'Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/c022c470eecf0ae4f81c289505c6b45815a78caebe11f3cd0a201330b6ad4122-init/merged: no such file or directory.',
when run : docker inspect ,it prints 'Error response from daemon: readlink /var/lib/docker/overlay2/l/4GWADAHD5OC5SYHTIOEPIBN64E: no such file or directory'
Steps to reproduce the behavior
0,vdb lvcreate mount to /var/lib/docker
1, install docker(19.03.1) kubernetes(v1.15.3) ,three node cluster
2,build docker registry(2.7)
3,deploy 200 kubernetes pods and all pods are running
4,reboot vm and reboot all kubernetes pod
Output of
docker version:[root@centos66 l]# docker version
Client: Docker Engine - Community
Version: 19.03.1
API version: 1.40
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:21:07 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.12)
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:19:36 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
You have mail in /var/spool/mail/root
[root@centos66 l]# docker images|grep pause
h3crd-wlan1.chinacloudapp.cn:5000/private-cloud/pause-amd64 3.1 da86e6ba6ca1 21 months ago 742kB
You have mail in /var/spool/mail/root
[root@centos66 l]# docker run da86e6ba6ca1
docker: Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/52e57461c24b35217aff10255db24d456251e4f7a029f864053d1cdfe9acb621-init/merged: no such file or directory.
See 'docker run --help'.
See 'docker run --help'.
[root@centos66 l]# docker inspect bb4a6b774658
[]
Error response from daemon: readlink /var/lib/docker/overlay2/l/4GWADAHD5OC5SYHTIOEPIBN64E: no such file or directory
Output of
docker info:[root@centos66 l]# docker info
Client:
Debug Mode: false
Server:
Containers: 6
Running: 0
Paused: 0
Stopped: 6
Images: 155
Server Version: 19.03.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.19.12-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 57.59GiB
Name: centos66
ID: EKY6:L2LL:UC7M:62L7:RLFI:KE3L:LOFU:GQEM:X3ZK:KHSG:BX4M:ZGI3
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
h3crd-wlan1.chinacloudapp.cn:5000
127.0.0.0/8
Registry Mirrors:
https://registry.docker-cn.com/
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.)
[root@centos66 l]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
[root@centos66 l]
[root@centos66 l]# uname -a
Linux centos66 4.19.12-1.el7.elrepo.x86_64 #1 SMP Fri Dec 21 11:06:36 EST 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@centos66 l]#
[root@centos66 l]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
[root@centos66 l]#