dockerd: fix rootless detection (alternative to #39024) by AkihiroSuda · Pull Request #39138 · moby/moby

AkihiroSuda · 2019-04-25T07:57:24Z

Signed-off-by: Akihiro Suda [email protected]

- What I did
Fix #38702 #39009

- How I did it

The --rootless flag had a couple of issues:

Rootful-Docker-in-Rootless-Docker doesn't work #38702: euid=0, $USER="root" but no access to cgroup ("rootful" Docker in rootless Docker)
19.03.0-beta1: could not get XDG_RUNTIME_DIR (dockerd fails to start) #39009: euid=0 but $USER="docker" (rootful boot2docker)

To fix #38702, XDG dirs are ignored as in rootful Docker, unless the
dockerd is directly running under RootlessKit namespaces.

RootlessKit detection is implemented by checking whether $ROOTLESSKIT_STATE_DIR is set.

To fix #39009, the non-robust $USER check is now completely removed.

The entire logic can be illustrated as follows:

withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR")
rootlessMode := withRootlessKit || cliFlag("--rootless")
honorXDG := withRootlessKit
useRootlessKitDockerProxy := withRootlessKit
removeCgroupSpec := rootlessMode
adjustOOMScoreAdj := rootlessMode

Close #39024
Fix #38702 #39009

- How to verify it

Make sure the regular rootless mode works:

$ dockerd-rootless.sh --experimental
WARN[2019-04-25T17:17:48.240093114+09:00] Running experimental build
WARN[2019-04-25T17:17:48.240656519+09:00] Running in rootless mode. Cgroups, AppArmor, and CRIU are disabled.
INFO[2019-04-25T17:17:48.241104912+09:00] Running with RootlessKit integration
...
INFO[2019-04-25T17:17:48.991132927+09:00] API listen on /run/user/1001/docker.sock

Make sure "rootful" docker in rootless docker works, with the socket listening on /var/run/docker.sock in the container, not on $XDG_RUNTIME_DIR/docker.sock in the container:

$ export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock
$ docker run -it --rm --name kinda-rootful-in-rootless --privileged -v /usr/local:/usr/local ubuntu
/ # apt update && apt install -y kmod iptables ca-certificates
/ # dockerd --experimental --rootless
WARN[2019-04-25T07:46:48.534752062Z] Running experimental build
WARN[2019-04-25T07:46:48.535769972Z] Running in rootless mode. Cgroups, AppArmor, and CRIU are disabled.
...
INFO[2019-04-25T07:46:48.909046101Z] API listen on /var/run/docker.sock

Make sure setting $USER to non-"root" does not break rootful Docker

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)
https://twitter.com/hashtag/WorldPenguinDay?f=image

The `--rootless` flag had a couple of issues: * moby#38702: euid=0, $USER="root" but no access to cgroup ("rootful" Docker in rootless Docker) * moby#39009: euid=0 but $USER="docker" (rootful boot2docker) To fix moby#38702, XDG dirs are ignored as in rootful Docker, unless the dockerd is directly running under RootlessKit namespaces. RootlessKit detection is implemented by checking whether `$ROOTLESSKIT_STATE_DIR` is set. To fix moby#39009, the non-robust `$USER` check is now completely removed. The entire logic can be illustrated as follows: ``` withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR") rootlessMode := withRootlessKit || cliFlag("--rootless") honorXDG := withRootlessKit useRootlessKitDockerProxy := withRootlessKit removeCgroupSpec := rootlessMode adjustOOMScoreAdj := rootlessMode ``` Close moby#39024 Fix moby#38702 moby#39009 Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda · 2019-04-25T07:58:28Z

@tonistiigi @tiborvass @cpuguy83 @tianon @thaJeztah

I'd like to get this merged before DockerCon 🙏

codecov · 2019-04-25T14:32:43Z

Codecov Report

Merging #39138 into master will decrease coverage by <.01%.
The diff coverage is 39.13%.

@@            Coverage Diff             @@
##           master   #39138      +/-   ##
==========================================
- Coverage   37.05%   37.05%   -0.01%     
==========================================
  Files         612      612              
  Lines       45400    45411      +11     
==========================================
+ Hits        16824    16826       +2     
- Misses      26294    26302       +8     
- Partials     2282     2283       +1

tonistiigi · 2019-04-25T18:34:05Z

Not a functional change and only slightly different but can

withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR")
rootlessMode := withRootlessKit || cliFlag("--rootless")
honorXDG := withRootlessKit
useRootlessKitDockerProxy := withRootlessKit
removeCgroupSpec := rootlessMode
adjustOOMScoreAdj := rootlessMode

be

withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR")
rootlessMode := cliFlag("--rootless")
honorXDG := withRootlessKit && rootlessMode
useRootlessKitDockerProxy := withRootlessKit && rootlessMode
removeCgroupSpec := rootlessMode
adjustOOMScoreAdj := rootlessMode

edit: Basically, the difference is that --rootless is always required and withRootlessKit is extended capabilities on top of it.

AkihiroSuda · 2019-04-26T06:01:16Z

That might be hard to implement, because honorXDG is used for setting up default config before parsing CLI flags such as --rootless.

tonistiigi · 2019-04-26T06:02:24Z

@AkihiroSuda Ok, not that important.

AkihiroSuda · 2019-04-26T12:05:55Z

@tianon @cpuguy83

PTAL?

rootless/rootless.go

cpuguy83

LGTM

dinvlad · 2019-05-10T02:55:19Z

Rootful ("child") docker info reports vfs as storage driver, even on Ubuntu hosts for which rootless docker info parent reports overlay2. Is overlay/overlay2 going to be supported for the child Docker? Thanks!

AkihiroSuda · 2019-05-10T03:19:05Z

Yes, but make sure to create volume for /home/foo/.local

dinvlad · 2019-05-10T17:06:32Z

Thanks @AkihiroSuda! We're now running the rootless container as follows:

docker run -it --rm --name kinda-rootful-in-rootless --privileged -v $HOME/bin:/opt/docker/bin:ro -v /lib/modules:/lib/modules:ro -v docker:/var/lib/docker ubuntu

Note that -v docker:/var/lib/docker creates rootless docker volume for use by rootful docker as overlay2.
Additionally, we mount host /lib/modules into the rootless container, so that we get rid of modprobe warnings (not sure if this is necessary though).

With these changes, the only "suspicious" warning we still get on startup of rootful dockerd is Not using native diff for overlay2, this may cause degraded performance for building images: failed to set opaque flag on middle layer: operation not permitted storage-driver=overlay2

GordonTheTurtle added the status/0-triage label Apr 25, 2019

AkihiroSuda mentioned this pull request Apr 25, 2019

decompose --rootless into $DOCKER_HONOR_XDG and --exec-opt native.* #39024

Closed

AkihiroSuda added area/rootless Rootless Mode status/1-design-review rebuild/experimental and removed status/0-triage labels Apr 25, 2019

GordonTheTurtle removed rebuild/experimental labels Apr 25, 2019

AkihiroSuda added the rebuild/janky label Apr 25, 2019

GordonTheTurtle removed the rebuild/janky label Apr 25, 2019

tonistiigi approved these changes Apr 26, 2019

View reviewed changes

cpuguy83 reviewed Apr 26, 2019

View reviewed changes

rootless/rootless.go Show resolved Hide resolved

cpuguy83 approved these changes Apr 26, 2019

View reviewed changes

cpuguy83 merged commit 51f27b1 into moby:master Apr 26, 2019

thaJeztah added the process/cherry-pick label May 10, 2019

thaJeztah mentioned this pull request May 13, 2019

[19.03 backport] backport rootless fixes docker-archive/engine#208

Merged

thaJeztah added process/cherry-picked and removed process/cherry-pick labels May 25, 2019

thaJeztah added this to the 20.03.0 milestone Apr 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dockerd: fix rootless detection (alternative to #39024)#39138

dockerd: fix rootless detection (alternative to #39024)#39138
cpuguy83 merged 1 commit intomoby:masterfrom
AkihiroSuda:fix-rootless-issues-alternative-to-39024

AkihiroSuda commented Apr 25, 2019 •

edited

Loading

Uh oh!

AkihiroSuda commented Apr 25, 2019

Uh oh!

codecov bot commented Apr 25, 2019

Uh oh!

tonistiigi commented Apr 25, 2019 •

edited

Loading

Uh oh!

AkihiroSuda commented Apr 26, 2019

Uh oh!

tonistiigi commented Apr 26, 2019

Uh oh!

AkihiroSuda commented Apr 26, 2019

Uh oh!

Uh oh!

cpuguy83 left a comment

Uh oh!

dinvlad commented May 10, 2019 •

edited

Loading

Uh oh!

AkihiroSuda commented May 10, 2019

Uh oh!

dinvlad commented May 10, 2019 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

AkihiroSuda commented Apr 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented Apr 25, 2019

Uh oh!

codecov bot commented Apr 25, 2019

Codecov Report

Uh oh!

tonistiigi commented Apr 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented Apr 26, 2019

Uh oh!

tonistiigi commented Apr 26, 2019

Uh oh!

AkihiroSuda commented Apr 26, 2019

Uh oh!

Uh oh!

cpuguy83 left a comment

Choose a reason for hiding this comment

Uh oh!

dinvlad commented May 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented May 10, 2019

Uh oh!

dinvlad commented May 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AkihiroSuda commented Apr 25, 2019 •

edited

Loading

tonistiigi commented Apr 25, 2019 •

edited

Loading

dinvlad commented May 10, 2019 •

edited

Loading

dinvlad commented May 10, 2019 •

edited

Loading