Skip to content

fix issue: make --device works at privileged mode #36258

Closed
wenlxie wants to merge 1 commit intomoby:masterfrom
wenlxie:origin.master.fixdevicemapissuewhenenableprivilege
Closed

fix issue: make --device works at privileged mode #36258
wenlxie wants to merge 1 commit intomoby:masterfrom
wenlxie:origin.master.fixdevicemapissuewhenenableprivilege

Conversation

@wenlxie
Copy link
Copy Markdown
Contributor

@wenlxie wenlxie commented Feb 9, 2018

Signed-off-by: wenlxie [email protected]

Please provide the following information:
-->

- What I did
fixs #35991
- How I verify it
Make the docker binary, test following cases

  • case 1
docker run -it --privileged=true --device /dev/vdc:/dev/vda busybox sh

Then I will get the error message

docker: Error response from daemon: linux runtime spec devices: In privileged mode,Path:/dev/vda In Container should be different with any path on host.
  • case 2
docker run -it --privileged=true --device /dev/vdc:/dev/vdx busybox sh

Then I enter the docker,
I can see the device /dev/vdx
I exit the docker, and there is no /dev/vdx at host

- Description for the changelog
Make --device options works at privileged mode

@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from 71bd87b to d1e7e67 Compare February 9, 2018 08:20
@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from 55bbdef to 651a5d1 Compare February 9, 2018 13:20
@GordonTheTurtle GordonTheTurtle added the dco/no Automatically set by a bot when one of the commits lacks proper signature label Feb 11, 2018
@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from dcc4917 to ba5a007 Compare February 11, 2018 11:43
@GordonTheTurtle GordonTheTurtle added dco/no Automatically set by a bot when one of the commits lacks proper signature and removed dco/no Automatically set by a bot when one of the commits lacks proper signature labels Feb 11, 2018
@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from 52c0543 to 8310364 Compare February 11, 2018 11:46
@GordonTheTurtle GordonTheTurtle removed the dco/no Automatically set by a bot when one of the commits lacks proper signature label Feb 11, 2018
@thaJeztah
Copy link
Copy Markdown
Member

ping @kolyshkin PTAL

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user specified --privileged --device xxx:xxx, then xxx will anyway be available, so specifying --device xxx:xxx is redundant, and it should not cause an error. One option is to just ignore it, the other is to have a warning, something like this maybe:

 log.Warnf("Specifying a device %s is redundant for privileged mode", deviceMapping.PathOnHost)

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah Feb 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was thinking about that; --device allows setting r (read), w (write), and m (mknod) options; what options are used if the device passed through --device overlaps with a device on the host?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a valid concern which I totally overlooked. I think in case restrictive permissions (anything other than the most permissive rwm) are given, the code should return an error as it can't enforce the required permissions (e.g. read-only).

What do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of a conflict, yes. Perhaps a warning is ok, after all, --privileged is expected to remove any restrictions, so perhaps lifting the restrictions that were specified to be ignored is ok.

@wenlxie what would your expectations be?

Copy link
Copy Markdown
Contributor Author

@wenlxie wenlxie Feb 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user specified --privileged --device xxx:xxx, then xxx will anyway be available, so specifying --device xxx:xxx is redundant, and it should not cause an error. One option is to just ignore it, the other is to have a warning, something like this maybe:

Thanks for the comments, will change the code.

Was thinking about that; --device allows setting r (read), w (write), and m (mknod) options; what options are used if the device passed through --device overlaps with a device on the host?

@thaJeztah @kolyshkin
For this case, I'd like to ignore the restrictions and respect the --privileged option.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One problem, though, is that the user (say, docker cli user) won't see it -- this is a warning in daemon logs. Or will they? Frankly I don't remember

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolyshkin It will be at the daemon logs and user will not see it.

@GordonTheTurtle GordonTheTurtle added the dco/no Automatically set by a bot when one of the commits lacks proper signature label Feb 21, 2018
@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from b8a060b to df9931e Compare February 21, 2018 13:42
@GordonTheTurtle GordonTheTurtle removed the dco/no Automatically set by a bot when one of the commits lacks proper signature label Feb 21, 2018
@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from 3be9d6d to ad10f6f Compare February 22, 2018 02:15
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 22, 2018

Codecov Report

❗ No coverage uploaded for pull request base (master@deac65c). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master   #36258   +/-   ##
=========================================
  Coverage          ?   36.08%           
=========================================
  Files             ?      610           
  Lines             ?    45151           
  Branches          ?        0           
=========================================
  Hits              ?    16293           
  Misses            ?    26617           
  Partials          ?     2241

@wenlxie
Copy link
Copy Markdown
Contributor Author

wenlxie commented Feb 26, 2018

@thaJeztah @kolyshkin
Now all of the tests had passed, so if you don't have more comments or concerns, can you please help to approve this PR? Thanks

@moby moby deleted a comment from GordonTheTurtle Feb 26, 2018
@vdemeester
Copy link
Copy Markdown
Member

@wenlxie
Copy link
Copy Markdown
Contributor Author

wenlxie commented May 12, 2018

@cpuguy83 #35991

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra space after Specify
Also should be Specifying.
These could all use some extra context on the log messages... e.g. logrus.WithField("container", c.ID)

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra space after Specify, and should be Specifying

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return errdefs.InvalidParameter(errors.New(errorMsg))

Additionally this should be validated on container create.

@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from ad10f6f to daae071 Compare May 15, 2018 07:21
@wenlxie
Copy link
Copy Markdown
Contributor Author

wenlxie commented May 15, 2018

@cpuguy83 code changes as your commnets , PTAL

Comment thread daemon/container.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clean up some of the grammar here?
Suggestion:

container device path must be different from any host device path for privileged mode containers

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above re: grammer

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread daemon/container.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be cleaner like:

if _, err := os.Stat(...); err != nil {
  if os.IsNotExist(err) {
    continue
  }
  return nil, errors.Wrap(err, "error stating device path in container")
}
return nil, errors.Errorf(...)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks better, tks

@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch 3 times, most recently from 0d0b7b1 to 40f8953 Compare May 15, 2018 15:02
Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer this warning to look more like a warning; it's not really "redundant", but will be ignored (so you'd end up with different permissions than you configured);

Something like:

running in privileged mode: custom %s permissions for device %s are ignored

or

custom %s permissions for device %s are ignored in privileged mode

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above: in what situation do we end here?

The error message seems to be if deviceMapping.PathOnHost == deviceMapping.PathInContainer (but that's already handled above, and prints a warning ? (perhaps I'm looking wrong, 😅)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the logic to make it more clear

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this again, I'm wondering if it's worth logging this, or if it's ok to just skip this warning: if there are no custom options (permissions) set, then the end result is exactly the same (with or without this device), so perhaps it's ok to just ignore this without logging a warning

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cpuguy83 wdyt? ^^

Comment thread daemon/container.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what situation do we end here? (looks like we only get here if deviceMapping.PathOnHost != deviceMapping.PathInContainer and if deviceMapping.PathInContainer exists.

But the error says (if I interpret correctly); it's not allowed to add a device if deviceMapping.PathOnHost == deviceMapping.PathInContainer ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah deviceMapping.PathOnHost == deviceMapping.PathInContainer has a continue

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to go through the logic:

  1. --device /dev/vdc:/dev/vdc, means: deviceMapping.PathOnHost == deviceMapping.PathInContainer
  • continue (no error)
  1. --device /dev/vdc:/some/path, and /some/path doesn't exist in the container:
  • continue (no error)
  1. --device /dev/vdc:/some/path, and /some/path cannot be read in the container:
  • error: error stating device path in container
  1. --device /dev/vdc:/some/path, and exists in the container:
  • error: container device path: /some/path must be different from any host device path for privileged mode containers

So for 4., the device target in the container is not allowed to overlap with an existing path in the container;

  • Is this different for non-privileged containers? (I guess not; but we don't seem to be validating that)
  • If deviceMapping.PathOnHost == deviceMapping.PathInContainer, should we still validate if PathInContainer is not an existing path in the container? (could there be situations where a device exists on the host at /foo/bar, but the container has a directory (or file) at that location?)
  • The error message confuses me, because (IIUC) it's actually about the container path (not the host path), correct?

(again, apologies if I'm misunderstanding 😄)

Copy link
Copy Markdown
Contributor Author

@wenlxie wenlxie May 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah
For step 4:
The device target in the container is not allowed to overlap with an existing path on the host but not in the container.
For privileged mode containers , some hostpath maybe mapped into the container, so I do this check here. I admit it is a bit excessive, maybe just check the /dev/ path is enough? What do you think?

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here re: error message wording

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from 40f8953 to 0d8733a Compare May 15, 2018 15:29
@runcom
Copy link
Copy Markdown
Member

runcom commented Aug 27, 2018

ping, any update on this? kube is affected by this

Comment thread daemon/oci_linux.go Outdated
Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One problem, though, is that the user (say, docker cli user) won't see it -- this is a warning in daemon logs. Or will they? Frankly I don't remember

Comment thread daemon/container.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/contianer/container/

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@kolyshkin
Copy link
Copy Markdown
Contributor

kolyshkin commented Aug 31, 2018

The only concern I have, will docker CLI user see those errors/warnings about device perms being ignored? If yes, this PR LGTM.

If not, that looks like a big issue -- a user set perms, they are ignored, and the user is not even aware of that.

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this needs a rebase 🤗

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this err is not used other than inside the if below, so to prevent confusion, it may be better to just;

if _, err := os.Stat(deviceMapping.PathInContainer); os.IsNotExist(err) {

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change error to something else (e.g., err2)? error is also the "type" for an error, so the name is confusing.

Comment thread daemon/oci_linux.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was looking if we could get rid old all these nested ifs (for example, but extracting the "non-privileged" and "privileged" branches to separate functions), but may be best to leave that refactoring for another PR

@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch 3 times, most recently from 05f5e2d to 85615cf Compare September 29, 2018 02:05
@wenlxie
Copy link
Copy Markdown
Contributor Author

wenlxie commented Sep 29, 2018

The only concern I have, will docker CLI user see those errors/warnings about device perms being ignored? If yes, this PR LGTM.
If not, that looks like a big issue -- a user set perms, they are ignored, and the user is not even aware of that.

It is better for user to know the info.
But for the current code behavior , it should return either error or nil. We can't push such warning info to the client.

@kolyshkin @thaJeztah Any Ideas?

@wenlxie wenlxie force-pushed the origin.master.fixdevicemapissuewhenenableprivilege branch from 85615cf to 4cdc729 Compare September 29, 2018 06:30
@thaJeztah
Copy link
Copy Markdown
Member

It is better for user to know the info.
But for the current code behavior , it should return either error or nil. We can't push such warning info to the client.

I think I'm ok with logging the error in that case. Showing the warning to the user would definitely be better, but if we don't have a mechanism to do so, I don't think it should be a blocker.

We should be clear in the documentation though; point out that limitation.

@kolyshkin WDYT?

@kolyshkin
Copy link
Copy Markdown
Contributor

This needs a documentation update (that goes to https://github.com/docker/cli repo I guess) noting that device permissions are ignored when --privileged is set. Otherwise LGTM.

@olljanat
Copy link
Copy Markdown
Contributor

@wenlxie this one needs to be rebased

@derek derek Bot added the status/failing-ci Indicates that the PR in its current state fails the test suite label Dec 22, 2018
@akhilerm
Copy link
Copy Markdown
Contributor

akhilerm commented Dec 5, 2019

@wenlxie If you are not working on this, can I take it forward.
cc : @thaJeztah @cpuguy83

@kolyshkin
Copy link
Copy Markdown
Contributor

Carried over in #40291

@kolyshkin kolyshkin closed this Dec 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status/failing-ci Indicates that the PR in its current state fails the test suite status/2-code-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.