Skip to content

Conversation

@neersighted
Copy link
Member

@neersighted neersighted commented May 3, 2022

This is helpful for debugging issues with the overlay2 storage driver.

Metacopy may be enabled by CONFIG_OVERLAY_FS_METACOPY, the overlay.metacopy parameter, or the metacopy mount option. See Documentation/filesystems/overlayfs.txt in the kernel source tree for details.

Backported from containers/storage@05c69f1b2a.

@thaJeztah thaJeztah added area/storage/overlay status/2-code-review kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. impact/changelog labels May 4, 2022
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Had a first glance over the changes; out of interest; this currently only does the detection; should (if supported) this detection also be used by the actual driver?

Comment on lines 170 to 166
} else {
logger.Warnf("overlay test mount did not indicate whether or not metacopy is being used: %v", err)
return nil, err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this information currently appears to only be for "informational" purposes, we should not return an error here; do we know under what circumstances this may fail? Effectively, this is a "best effort" attempt to detect the feature (nothing should fail if we fail to detect), so I'm inclined to;

  • in case of failure, log the error, and consider it unsupported ("false")?
  • consider moving all logging into doesMetacopy() / supportsMetaCopy(), and make it just return a boolean.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds reasonable to me. There are situations in which the mount could fail, but we mostly should not reach those as the overlay2 driver wouldn't be supported/loaded.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I'm looking at this again and thinking that we should still bubble any errors up out of doesMetacopy() -- specifically, any failure to Mkdir/Chmod/manipulate the filesystem in any way should be bubbled up instead of swallowed by doesMetacopy().

While yes, it is optional/informational, I don't think it makes sense to silently ignore errors that may indicate an underlying system problem (like being unable to create a temporary directory, or being unable to mount a directory).

@neersighted neersighted force-pushed the overlay2-report-metacopy branch 2 times, most recently from a80ad77 to 9af74c6 Compare May 9, 2022 21:34
@neersighted
Copy link
Member Author

I've had a go at addressing review feedback -- please let me know if any of the points I pushed back on are problematic.

Regarding this:

out of interest; this currently only does the detection; should (if supported) this detection also be used by the actual driver?

I don't think there is anything else to be done in the daemon at this time. We don't currently request metacopy (on, or off) in our mounts, so this is merely meant to report the status of metacopy to aid in debugging issues with the daemon/driver in the field. Typically metacopy will be off, as that is the upstream default, but it could be turned on by a custom kernel, a kernel command line, or writing to /sys/module/overlay/parameters/metacopy.

In theory we could add configuration to force metacopy on/off and change this check to detect support -- however there is currently no drive to do so. However it does give a good jumping off point if such a feature is ever desirable for some reason.

@AkihiroSuda AkihiroSuda self-requested a review May 10, 2022 13:44
Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@neersighted neersighted force-pushed the overlay2-report-metacopy branch from 9af74c6 to 35f5577 Compare May 13, 2022 13:36
@neersighted neersighted requested a review from tianon as a code owner May 13, 2022 13:36
@neersighted neersighted changed the title overlay2: test for and report metacopy status [v2] overlay2: test for and report metacopy status May 13, 2022
This is a first, naive implementation, that does not account for
userxattr/UserNS.

Signed-off-by: Bjorn Neergaard <[email protected]>
@neersighted neersighted force-pushed the overlay2-report-metacopy branch from 35f5577 to da9ef2c Compare May 13, 2022 13:37
func usingMetacopy(d string) (bool, error) {
userxattr := false
if userns.RunningInUserNS() {
needed, err := overlayutils.NeedsUserXAttr(d)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for reviewers: this results in us doing a mount to check if we need userxattr twice -- however, I didn't want to attach a major refactor to this PR.

The cost is negligible as this is only done at driver Init. There will be a follow-up PR that restructures this to avoid the duplicate work.

@thaJeztah
Copy link
Member

@neersighted looks like a linter is failing;


daemon/graphdriver/overlayutils/overlayutils.go:82:1: exported function `GetOverlayXattr` should have comment or be unexported (golint)
 func GetOverlayXattr(name string) string {
 ^

@neersighted
Copy link
Member Author

@neersighted looks like a linter is failing;


daemon/graphdriver/overlayutils/overlayutils.go:82:1: exported function `GetOverlayXattr` should have comment or be unexported (golint)
 func GetOverlayXattr(name string) string {
 ^

Fixed, thanks!

@neersighted neersighted force-pushed the overlay2-report-metacopy branch from da9ef2c to ce3e2d1 Compare May 17, 2022 13:08
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

only wondering if the two commits should be squashed (if the first commit is broken, they probably should be squashed)

@thaJeztah thaJeztah added this to the 22.06.0 milestone May 19, 2022
Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah thaJeztah dismissed AkihiroSuda’s stale review May 19, 2022 19:16

Looks like this was addressed in the second commit

@thaJeztah thaJeztah merged commit 517afce into moby:master May 19, 2022
@neersighted neersighted deleted the overlay2-report-metacopy branch May 19, 2022 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/storage/overlay impact/changelog kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. status/2-code-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants