Skip to content

new storage driver: fuse-overlayfs#40483

Merged
cpuguy83 merged 1 commit intomoby:masterfrom
AkihiroSuda:fuse-overlayfs
Mar 11, 2020
Merged

new storage driver: fuse-overlayfs#40483
cpuguy83 merged 1 commit intomoby:masterfrom
AkihiroSuda:fuse-overlayfs

Conversation

@AkihiroSuda
Copy link
Member

@AkihiroSuda AkihiroSuda commented Feb 9, 2020

- What I did
Added a new storage driver: fuse-overlayfs.

fuse-overlayfs provides rootless overlayfs functionality without depending on any kernel patch.

Aside from rootless, fuse-overlayfs could be potentially used for eliminating chown() calls that happen in userns-remap mode, because fuse-overlayfs also provides shiftfs functionality.

System requirements:

  • fuse-overlayfs needs to be installed. Tested with 0.7.6.
  • kernel >= 4.18

Fix #40218

- How I did it

The implementation is based on Podman's overlay driver which supports both kernel-mode overlayfs and fuse-overlayfs in the single driver instance:
https://github.com/containers/storage/blob/39a8d5ed/drivers/overlay/overlay.go

However, Moby's implementation aims to decouple fuse-overlayfs driver from the kernel-mode driver (overlay2) for simplicity.

- How to verify it

$ go test -exec sudo -v ./daemon/graphdriver/fuse-overlayfs
=== RUN   TestFUSEOverlayFSSetup
--- PASS: TestFUSEOverlayFSSetup (0.00s)
=== RUN   TestFUSEOverlayFSCreateEmpty
--- PASS: TestFUSEOverlayFSCreateEmpty (0.00s)
=== RUN   TestFUSEOverlayFSCreateBase
--- PASS: TestFUSEOverlayFSCreateBase (0.00s)
=== RUN   TestFUSEOverlayFSCreateSnap
--- PASS: TestFUSEOverlayFSCreateSnap (0.02s)
=== RUN   TestFUSEOverlayFS128LayerRead
--- PASS: TestFUSEOverlayFS128LayerRead (4.32s)
=== RUN   TestFUSEOverlayFSTeardown
--- PASS: TestFUSEOverlayFSTeardown (0.09s)
PASS
ok      github.com/docker/docker/daemon/graphdriver/fuse-overlayfs      4.450s

- Description for the changelog

new storage driver: fuse-overlayfs

- A picture of a cute animal (not mandatory but encouraged)

🐧

@AkihiroSuda AkihiroSuda requested a review from dmcgowan as a code owner February 9, 2020 17:29
@AkihiroSuda AkihiroSuda mentioned this pull request Feb 9, 2020
Closed
@AkihiroSuda AkihiroSuda added area/rootless Rootless Mode area/storage/overlay kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny labels Feb 9, 2020
@AkihiroSuda AkihiroSuda mentioned this pull request Feb 9, 2020
Closed
6 tasks
@thaJeztah
Copy link
Member

thaJeztah commented Feb 10, 2020

@cpuguy83 @dmcgowan PTAL

@AkihiroSuda
new storage driver: fuse-overlayfs
7418745 
`fuse-overlayfs` provides rootless overlayfs functionality without depending
on any kernel patch.

Aside from rootless, `fuse-overlayfs` could be potentially used for eliminating
`chown()` calls that happen in userns-remap mode, because `fuse-overlayfs` also
provides shiftfs functionality.

System requirements:
* fuse-overlayfs needs to be installed. Tested with 0.7.6.
* kernel >= 4.18

Unit test: `go test -exec sudo -v ./daemon/graphdriver/fuse-overlayfs`

The implementation is based on Podman's `overlay` driver which supports
both kernel-mode overlayfs and fuse-overlayfs in the single driver instance:
https://github.com/containers/storage/blob/39a8d5ed/drivers/overlay/overlay.go

However, Moby's implementation aims to decouple `fuse-overlayfs` driver from the
kernel-mode driver (`overlay2`) for simplicity.

Fix moby#40218

Signed-off-by: Akihiro Suda <[email protected]>
@cpuguy83
Copy link
Member

cpuguy83 commented Feb 13, 2020

SGTM

Even though we are moving to containerd, I think this is good to bring in.
As @tonistiigi pointed out on the maintainers call, this doesn't pose any new problems for that migration.

@cpuguy83 cpuguy83 added this to the 20.03.0 milestone Feb 13, 2020
@thaJeztah
Copy link
Member

thaJeztah commented Feb 13, 2020

Agreed, I'm ok with moving this forward (did not review the code changes other than glancing over it)

@AkihiroSuda
Copy link
Member Author

AkihiroSuda commented Feb 26, 2020

@cpuguy83 @tonistiigi @tiborvass PTAL?

@thaJeztah
Copy link
Member

thaJeztah commented Feb 27, 2020

@dmcgowan @kolyshkin would you be able to have a look?

kolyshkin
kolyshkin approved these changes Feb 29, 2020
View reviewed changes
Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@AkihiroSuda
Copy link
Member Author

AkihiroSuda commented Mar 9, 2020

@cpuguy83 @kolyshkin LGTY?

@kolyshkin
Copy link
Contributor

kolyshkin commented Mar 11, 2020

So, I see that in https://github.com/containers/storage/ they use the same overlay driver for in-kernel mounting and fuse-overlayfs, by supplying one additional parameter to overlay driver (see for example https://github.com/containers/storage/blob/master/drivers/overlay/overlay.go#L952).

Is this something we'd want to do here as well, or is a separate driver a better option? @AkihiroSuda @thaJeztah @dmcgowan

@AkihiroSuda
Copy link
Member Author

AkihiroSuda commented Mar 11, 2020

Let's avoid mixing up different drivers

@kolyshkin
Copy link
Contributor

kolyshkin commented Mar 11, 2020

LGTM then

@kolyshkin kolyshkin requested a review from cpuguy83 March 11, 2020 17:10
@kolyshkin
Copy link
Contributor

kolyshkin commented Mar 11, 2020

We got SGTM from @cpuguy83 and LGTM from myself, so I think @thaJeztah you can merge it now

cpuguy83
cpuguy83 approved these changes Mar 11, 2020
View reviewed changes
Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cpuguy83 cpuguy83 merged commit 470d32d into moby:master Mar 11, 2020
@AkihiroSuda AkihiroSuda mentioned this pull request Mar 17, 2020
Closed
This was referenced Jun 24, 2020
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cpuguy83 cpuguy83 cpuguy83 approved these changes

@kolyshkin kolyshkin kolyshkin approved these changes

@dmcgowan dmcgowan Awaiting requested review from dmcgowan dmcgowan is a code owner

Assignees

No one assigned

Labels

area/rootless Rootless Mode area/storage/overlay impact/changelog kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny status/2-code-review

Projects

None yet

Milestone

20.10.0

Development

Successfully merging this pull request may close these issues.

new storage driver: fuse-overlayfs

4 participants

@AkihiroSuda @thaJeztah @cpuguy83 @kolyshkin