Skip to content

remove auto-creating non-existant volume host path#19953

Closed
keloyang wants to merge 1 commit intomoby:masterfrom
keloyang:non-auto-create
Closed

remove auto-creating non-existant volume host path#19953
keloyang wants to merge 1 commit intomoby:masterfrom
keloyang:non-auto-create

Conversation

@keloyang
Copy link
Copy Markdown
Contributor

@keloyang keloyang commented Feb 3, 2016

move from #19537
currently, docker create non-existent volume host path automatically.

e.g.

docker run --rm -ti -v /hello:/hello ubuntu bash

if the /hello dir don't exist in host, docker will create it to avoid the fail of mount.

But it is not reasonable.in fact,we can skip non-existant volume host path and remove "MkdirAll" from Setup.
Signed-off-by: yangshukui [email protected]

@hqhq
Copy link
Copy Markdown
Contributor

hqhq commented Feb 3, 2016

Code looks good, but you should use push -f instead of create a new PR next time.

hqhq added a commit to hqhq/moby that referenced this pull request Feb 3, 2016
Follow up moby#19953

It'll break lots of applications if container won't start
with none exist source bind volume. It could be hard for
administrator to create every directories applications need.

So we can add an option for daemon, to assign a safe volume
which can create new directories by user's will, if none
existed directories are in these editable volumes, daemon can
create them when starting containers.

Signed-off-by: Qiang Huang <[email protected]>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) I think "has been" isn't needed here; simply "Removed in release: 1.11"

@thaJeztah thaJeztah added this to the 1.11.0 milestone Feb 3, 2016
@LK4D4
Copy link
Copy Markdown
Contributor

LK4D4 commented Feb 3, 2016

@keloyang need rebase and CI seems unhappy

@keloyang
Copy link
Copy Markdown
Contributor Author

keloyang commented Feb 4, 2016

integration-cli test pass

04:37:49 OK: 1195 passed, 13 skipped
04:37:49 PASS

but the docker-py failed,maybe we should modify the testcase of docker-py.

@keloyang keloyang closed this Feb 4, 2016
@keloyang keloyang reopened this Feb 4, 2016
@GordonTheTurtle GordonTheTurtle added dco/no Automatically set by a bot when one of the commits lacks proper signature and removed dco/no Automatically set by a bot when one of the commits lacks proper signature labels Feb 4, 2016
@keloyang
Copy link
Copy Markdown
Contributor Author

keloyang commented Feb 4, 2016

@LK4D4 rebased.

@lowenna
Copy link
Copy Markdown
Member

lowenna commented Feb 4, 2016

Not sure about other contexts, but the windowsTP4 failure is genuine.

@GordonTheTurtle GordonTheTurtle added dco/no Automatically set by a bot when one of the commits lacks proper signature and removed dco/no Automatically set by a bot when one of the commits lacks proper signature labels Feb 5, 2016
@calavera
Copy link
Copy Markdown
Contributor

calavera commented Feb 8, 2016

I think we'll need to fix docker-py to support this. I'm pretty sure the failing tests are because that library assumes that docker creates the mount points.

@thaJeztah
Copy link
Copy Markdown
Member

ping @hqhq @calavera what's the status here? I see a PR linked from docker-py that was merged?

@vdemeester
Copy link
Copy Markdown
Member

@keloyang I think you should update the DOCKER_PY_COMMIT in the Dockerfile(s) to fix the tests (https://github.com/docker/docker/blob/master/Dockerfile#L183).

PS: @keloyang if you're short on time, we can carry it for you 😉

@keloyang
Copy link
Copy Markdown
Contributor Author

@vdemeester I will change it as soon as possible. thank you.

@SvenDowideit
Copy link
Copy Markdown
Contributor

wow - the test failure at https://jenkins.dockerproject.org/job/docs-docker-pr/4223/ is due to alot of commits being merged between the creation of the PR and when the job actually ran - lets see what happens now.

retest this please

@SvenDowideit
Copy link
Copy Markdown
Contributor

the newly triggered checker job failed due to the docker/tutorials repo going private: https://jenkins.dockerproject.org/job/docs-docker-pr/4228/console

@thaJeztah
Copy link
Copy Markdown
Member

WindowsTP4 error looks like an issue with that node; https://jenkins.dockerproject.org/job/Docker-PRs-WoW-TP4/2323/console

08:47:11 
08:47:11 ----------------------------------------------------------------------
08:47:11 FAIL: docker_cli_build_test.go:4270: DockerSuite.TestBuildFromGIT
08:47:11 
08:47:11 docker_cli_build_test.go:4286:
08:47:11     c.Fatal(err)
08:47:11 ... Error: failed to build the image: Sending build context to Docker daemon 50.18 kB

08:47:11 Step 1 : FROM busybox
08:47:11  ---> 19cbf5ea5661
08:47:11 Step 2 : ADD first /first
08:47:11  ---> 2ff8e7df7b98
08:47:11 Removing intermediate container 6a63e7f058d1
08:47:11 Step 3 : RUN [ -f /first ]
08:47:11  ---> Running in 8b1a4f37905c
08:47:11 Cannot start container 8b1a4f37905c410d45d72cfbf143573f72343814243d35cb05bbe7ec428a51f3: HCSShim::StartComputeSystem failed in Win32: The object identifier does not represent a valid object. (0x10d8) id=8b1a4f37905c410d45d72cfbf143573f72343814243d35cb05bbe7ec428a51f3
08:47:11 
08:47:11 

@thaJeztah thaJeztah removed the status/failing-ci Indicates that the PR in its current state fails the test suite label Mar 9, 2016
@thaJeztah
Copy link
Copy Markdown
Member

Oh, weird stuff; experimental also actually failed on the docker-py tests, but was marked SUCCESS. Wondering if there's something related to this PR https://jenkins.dockerproject.org/job/Docker-PRs-experimental/16047/console

@thaJeztah thaJeztah added the status/failing-ci Indicates that the PR in its current state fails the test suite label Mar 9, 2016
@keloyang
Copy link
Copy Markdown
Contributor Author

@thaJeztah, thank you very much, I will check as soon as possible.

@icecrime icecrime removed the status/failing-ci Indicates that the PR in its current state fails the test suite label Mar 10, 2016
// Can't bind-mount volumes with separate host daemon.
return []string{"/vol1", "/vol2", "/vol3", "/vol_ro:/vol_ro:ro"}
for _, volume := range volumes {
dir := strings.Split(volume, ":")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work well for Windows paths. Can we instead create the paths, and then build the volumes slice?

@cpuguy83
Copy link
Copy Markdown
Member

Couple of nits.
We should get this in quickly or push off to 1.12 so that we can add explicit create flags before code freeze.

@thaJeztah
Copy link
Copy Markdown
Member

ping @keloyang could you have a look at the comments @cpuguy83 gave? Hopefully we can still get this into 1.11 if those are addressed

@tiborvass
Copy link
Copy Markdown
Contributor

Unfortunately, it seems like the cons (breaking change) outweigh the pros of this PR. As far as the create or nocreate options: they are not that useful since if by default we can't break the user, it would have to be a nocreate option, which the user can already do by simply creating the dirs or files beforehand.

Sorry @keloyang it's really not your fault there has been some misunderstanding among maintainers.

I suggest opening another PR to remove the deprecation in the docs and the warning as well.

@m59peacemaker
Copy link
Copy Markdown

I have a need in which I'm fairly certain there is no solution without a nocreate option. I have a script that run a docker-in-docker container, passing the script's arguments into a script in that container, and that container script starts other docker containers. I pass in the hosts cwd as an environment variable. The container script is smart enough to parse the arguments passed in, but the host script that starts all of this is not. The container script is going to start sibling containers since docker.sock is mounted and it mounts volumes with docker run based on the arguments passed in. However, the container script can't determine if volumes already exist (that I know of). That means that if a non-existent directory path is passed to the host script as an argument, my docker run command is going to use it whether it exists or not, and thus it will be created on the host. Lacking nocreate, my script has the unfortunate flaw of creating directories when a non-existent path is given.

@YarekTyshchenko
Copy link
Copy Markdown

YarekTyshchenko commented Aug 13, 2021

My usecase would benefit from the nocreate option. I have a container deployed on an embedded system, and if a directory doesn't exist, that means a prior mount has failed. Docker would happilly create the dir off the mount, and fill the root partition with data :(

@thaJeztah
Copy link
Copy Markdown
Member

thaJeztah commented Aug 13, 2021

@YarekTyshchenko you can use the "advanced" syntax --mount flag instead, which does not auto-create the path; see https://docs.docker.com/storage/bind-mounts/

$ docker run --rm --mount type=bind,src=/no/such/dir,target=/container/path alpine
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /no/such/dir.

@YarekTyshchenko
Copy link
Copy Markdown

@thaJeztah I unfortunately don't have access to the command directly. It runs as part of microsoft's Azure IoT Edge offering.
https://docs.microsoft.com/en-us/azure/iot-edge/how-to-access-host-storage-from-module?view=iotedge-2020-11
I will check the integration to see if there is a syntax that i've missed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.