Skip to content

docker: Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/c022c470eecf0ae4f81c289505c6b45815a78caebe11f3cd0a201330b6ad4122-init/merged: no such file or directory. #791

Description

@mengqiuyuwl
  • This is a bug report
  • This is a feature request
  • I searched existing issues before opening this one

Expected behavior

docker run container correctly

Actual behavior

when run: docker run, it prints 'Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/c022c470eecf0ae4f81c289505c6b45815a78caebe11f3cd0a201330b6ad4122-init/merged: no such file or directory.',
when run : docker inspect ,it prints 'Error response from daemon: readlink /var/lib/docker/overlay2/l/4GWADAHD5OC5SYHTIOEPIBN64E: no such file or directory'

Steps to reproduce the behavior

0,vdb lvcreate mount to /var/lib/docker
1, install docker(19.03.1) kubernetes(v1.15.3) ,three node cluster
2,build docker registry(2.7)
3,deploy 200 kubernetes pods and all pods are running
4,reboot vm and reboot all kubernetes pod

Output of docker version:
[root@centos66 l]# docker version
Client: Docker Engine - Community
Version: 19.03.1
API version: 1.40
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:21:07 2019
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.12)
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:19:36 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
You have mail in /var/spool/mail/root

(paste your output here)

[root@centos66 l]# docker images|grep pause
h3crd-wlan1.chinacloudapp.cn:5000/private-cloud/pause-amd64 3.1 da86e6ba6ca1 21 months ago 742kB
You have mail in /var/spool/mail/root
[root@centos66 l]# docker run da86e6ba6ca1
docker: Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/52e57461c24b35217aff10255db24d456251e4f7a029f864053d1cdfe9acb621-init/merged: no such file or directory.
See 'docker run --help'.

See 'docker run --help'.
[root@centos66 l]# docker inspect bb4a6b774658
[]
Error response from daemon: readlink /var/lib/docker/overlay2/l/4GWADAHD5OC5SYHTIOEPIBN64E: no such file or directory
Output of docker info:
[root@centos66 l]# docker info
Client:
Debug Mode: false

Server:
Containers: 6
Running: 0
Paused: 0
Stopped: 6
Images: 155
Server Version: 19.03.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.19.12-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 57.59GiB
Name: centos66
ID: EKY6:L2LL:UC7M:62L7:RLFI:KE3L:LOFU:GQEM:X3ZK:KHSG:BX4M:ZGI3
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
h3crd-wlan1.chinacloudapp.cn:5000
127.0.0.0/8
Registry Mirrors:
https://registry.docker-cn.com/
Live Restore Enabled: false

(paste your output here)

Additional environment details (AWS, VirtualBox, physical, etc.)
[root@centos66 l]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
[root@centos66 l]

[root@centos66 l]# uname -a
Linux centos66 4.19.12-1.el7.elrepo.x86_64 #1 SMP Fri Dec 21 11:06:36 EST 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@centos66 l]#

[root@centos66 l]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
[root@centos66 l]#

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions