Skip to content

[20.10 backport] rootless: overlay2: fix "createDirWithOverlayOpaque(...) ... input/output error"#42235

Merged
tiborvass merged 3 commits into
moby:20.10from
AkihiroSuda:fix-overlay2-nativediff-2010
Apr 1, 2021
Merged

[20.10 backport] rootless: overlay2: fix "createDirWithOverlayOpaque(...) ... input/output error"#42235
tiborvass merged 3 commits into
moby:20.10from
AkihiroSuda:fix-overlay2-nativediff-2010

Conversation

@AkihiroSuda

@AkihiroSuda AkihiroSuda commented Apr 1, 2021

Copy link
Copy Markdown
Member

Cherry-pick #42188


- What I did
Fix createDirWithOverlayOpaque(...) ... input/output error on rootless.

Fix docker/for-linux#1055 (intermittent error, on Debian 10, kernel < 5.11)
Fix #42177 (100% reproducible error, on kernel >= 5.11)
Fix #42206

- How I did it

[bug fix] Commit 1: overlay2: call d.naiveDiff.ApplyDiff when useNaiveDiff==true

Previously, d.naiveDiff.ApplyDiff was not used even when useNaiveDiff()==true

[optimization] Commit 2: overlay2: doesSupportNativeDiff: add fast path for userns

When running in userns, returns error (i.e. "use naive, not native") immediately.

No substantial change to the logic.

[clean up] Commit 3: archive: do not use overlayWhiteoutConverter for UserNS

overlay2 no longer sets archive.OverlayWhiteoutFormat when running in UserNS, so we can remove the complicated logic in the archive package.

- How to verify it

$ docker --context=rootless pull python:3.9

- Description for the changelog

rootless: overlay2: fix "createDirWithOverlayOpaque(...) ... input/output error"

- A picture of a cute animal (not mandatory but encouraged)

🐧

Previously, `d.naiveDiff.ApplyDiff` was not used even when
`useNaiveDiff()==true`

Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit dd97134)
Signed-off-by: Akihiro Suda <[email protected]>
When running in userns, returns error (i.e. "use naive, not native")
immediately.

No substantial change to the logic.

Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit 67aa418)
Signed-off-by: Akihiro Suda <[email protected]>
overlay2 no longer sets `archive.OverlayWhiteoutFormat` when
running in UserNS, so we can remove the complicated logic in the
archive package.

Signed-off-by: Akihiro Suda <[email protected]>
(cherry picked from commit 6322dfc)
Signed-off-by: Akihiro Suda <[email protected]>
@AkihiroSuda AkihiroSuda added this to the 20.10.6 milestone Apr 1, 2021
@thaJeztah
thaJeztah marked this pull request as ready for review April 1, 2021 12:06

@thaJeztah thaJeztah left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah

Copy link
Copy Markdown
Member

s390x and ppc64le failures are unrelated; we need to keep an eye on this one, which is flaky;

[2021-04-01T12:24:15.818Z] === Failed
[2021-04-01T12:24:15.818Z] === FAIL: daemon/logger TestCopierWithSized/With_RingLogger (0.00s)
[2021-04-01T12:24:15.818Z]     --- FAIL: TestCopierWithSized/With_RingLogger (0.00s)
[2021-04-01T12:24:15.818Z]         copier_test.go:265: invalid character '\x00' looking for beginning of value

@tiborvass
tiborvass merged commit b6f4e8b into moby:20.10 Apr 1, 2021
@chri2

chri2 commented Apr 23, 2021

Copy link
Copy Markdown

Q: Just installed docker-ce 20.10.6 in Debian Buster. Until now I used fuse-overlayfs, because overlayfs without fuse didn't work. If this is now fixed in 20.10.6, what are the arguments and drawbacks moving on to overlayfs without fuse?

@AkihiroSuda

Copy link
Copy Markdown
Member Author

kernel-mode overlayfs is faster and more robust than fuse-overlayfs.

But Debian version of kernel-mode overlayfs seems still broken: #42302 .
So, to use kernel-mode overlayfs, please use either Ubuntu kernel or kernel >= 5.11.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants