Rootless Docker cannot pull certain images?!
No matter what I do, I can't quite get it to work, and the internet is full of reports where this is the case. I'm on Ubuntu 20.04 with a 5.4.0-58 kernel. systemctl --user status docker reports:
dockerd-rootless.sh[1932]: [...] level=warning msg=" Not using native diff for overlay2, this may cause degraded performance for building images: failed to set opaque flag on middle layer: operation not permitted" I tried:
Setting options overlay metacopy=off redirect_dir=off for the overlay kernel module
Installing packages like fuse-overlayfs and its libraries
https://mikeshade.com/posts/docker-native-overlay-diff/
Reports:
Failed to pull images in rootless mode with overlayfs: "createDirWithOverlayOpaque(...) ... input/output error" docker/for-linux#1055 https://stackoverflow.com/questions/65273160/how-to-enable-native-overlay-diff-on-debian-docker-rootless ubuntu:18.04 container commit fails with error processing tar file: operation not permitted podman-container-tools/buildah#2326 https://github.com/containers/buildah/blob/master/troubleshooting.md#6-rootless-buildah-bud-fails-when-using-overlayfs
Why is this important? Some images cannot be pulled, and fail due to a strange error where lchown does not find a file or directory:
~ $ docker pull docker.io/gitlab/gitlab-ee:13.6.3-ee.0
13.6.3-ee.0: Pulling from gitlab/gitlab-ee
be8ec4e48d7f: Pull complete
33b8b485aff0: Pull complete
d887158cc58c: Pull complete
05895bb28c18: Pull complete
5a5da1768523: Pull complete
fabe58ce18d2: Pull complete
7eb97e608ff0: Pull complete
1ab51a6460c9: Pull complete
b07e61468e0b: Pull complete
c5dd26abc4c2: Extracting [==================================================> ] 901.9MB/901.9MB
failed to register layer: Error processing tar file(exit status 1): lchown /opt/gitlab/sv/sshd/log/supervise/ok: no such file or directory Steps to reproduce the issue:
Install rootless Docker on Ubuntu 20.04.1
Describe the results you received:
I'm seeing
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false and I cannot pull (large?) images.
Describe the results you expected:
If would like to see
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true < -------- and I'd like to pull large images.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client: Docker Engine - Community
Version: 20.10.0
API version: 1.41
Go version: go1.13.15
Git commit: 7287ab3
Built: Tue Dec 8 18:54:00 2020
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.0
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: eeddea2
Built: Tue Dec 8 18:58:04 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 20.10.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
init version: de40ad0
Security Options:
seccomp
Profile: default
rootless
cgroupns
Kernel Version: 5.4.0-58-generic
Operating System: Ubuntu 20.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.775GiB
Name: itbsd
ID: WCKE:YRNK:PITP:G345:T3KJ:IMZH:2N7L:Y5YP:FFLO:MLZW:CFD7:ZMGU
Docker Root Dir: /home/notroot/.local/share/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support
WARNING: Support for cgroup v2 is experimental
Additional environment details (AWS, VirtualBox, physical, etc.):
Happens here: https://github.com/docker/docker-ce/blob/master/components/engine/daemon/graphdriver/overlay2/check.go#L55 ?
Rootless Docker cannot pull certain images?!
No matter what I do, I can't quite get it to work, and the internet is full of reports where this is the case. I'm on Ubuntu 20.04 with a
5.4.0-58kernel.systemctl --user status dockerreports:dockerd-rootless.sh[1932]: [...] level=warning msg="Not using native diff for overlay2, this may cause degraded performance for building images: failed to set opaque flag on middle layer: operation not permitted" storage-driver=overlay2I tried:
options overlay metacopy=off redirect_dir=offfor the overlay kernel modulefuse-overlayfsand its librariesReports:
Why is this important? Some images cannot be pulled, and fail due to a strange error where
lchowndoes not find a file or directory:Steps to reproduce the issue:
Describe the results you received:
I'm seeing
and I cannot pull (large?) images.
Describe the results you expected:
If would like to see
and I'd like to pull large images.
Additional information you deem important (e.g. issue happens only occasionally):
Output of
docker version:Output of
docker info:Additional environment details (AWS, VirtualBox, physical, etc.):
Happens here: https://github.com/docker/docker-ce/blob/master/components/engine/daemon/graphdriver/overlay2/check.go#L55 ?