docker cp to and from containers by jlhawn · Pull Request #13171 · moby/moby

jlhawn · 2015-05-13T01:25:08Z

Copy files/folders between containers and the local filesystem.

Usage:  docker cp [options] CONTAINER:PATH LOCALPATH|-
        docker cp [options] LOCALPATH|- CONTAINER:PATH

--help  Print usage statement

In the first synopsis form, the docker cp utility copies the contents of
PATH from the filesystem of CONTAINER to the LOCALPATH (or stream as
a Tar Archive to STDOUT if - is specified).

In the second synopsis form, the contents of LOCALPATH (or a Tar Archive
streamed from STDIN if - is specified) are copied from the local machine to
PATH in the filesystem of CONTAINER.

You can copy to or from either a running or stopped container. The PATH can
be a file or directory. The docker cp command assumes all CONTAINER:PATH
values are relative to the / (root) directory of the container. This means
supplying the initial forward slash is optional; The command sees
compassionate_darwin:/tmp/foo/myfile.txt and
compassionate_darwin:tmp/foo/myfile.txt as identical. If a LOCALPATH value
is not absolute, is it considered relative to the current working directory.

Behavior is similar to the common Unix utility cp -a in that directories are
copied recursively and file mode, permission, and ownership are preserved if
possible.

Assuming a path separator of /, a first argument of SRC_PATH and second
argument of DST_PATH, the behavior is as follows:

SRC_PATH specifies a file
- DST_PATH does not exist
  - the file is saved to a file created at DST_PATH
- DST_PATH does not exist and ends with /
  - Error condition: the destination directory must exist.
- DST_PATH exists and is a file
  - the destination is overwritten with the contents of the source file
- DST_PATH exists and is a directory
  - the file is copied into this directory using the basename from
    SRC_PATH
SRC_PATH specifies a directory
- DST_PATH does not exist
  - DST_PATH is created as a directory and the contents of the source
    directory are copied into this directory
- DST_PATH exists and is a file
  - Error condition: cannot copy a directory to a file
- DST_PATH exists and is a directory
  - SRC_PATH does not end with /.
    - the source directory is copied into this directory
  - SRC_PAPTH does end with /.
    - the content of the source directory is copied into this
      directory

The command requires SRC_PATH and DST_PATH to exist according to the above
rules. If SRC_PATH is a symbolic link, the symbolic link, not the target, is
copied. If a path separator immediately follows the symbolic link, it will be
resolved to its target and the target resource will be copied.

A colon (:) is used as a delimiter between CONTAINER and PATH, but :
could also be in a valid LOCALPATH, like file:name.txt. This ambiguity is
resolved by requiring a LOCALPATH with a : to be made explicit with a
relative or absolute path, for example:

`/path/to/file:name.txt` or `./file:name.txt`

It is not possible to copy certain system files such as resources under
/proc, /sys, /dev, and mounts created by the user in the container.

Using - as the first argument in place of a LOCALPATH will stream the
contents of STDIN as a Tar Archive which will be extracted to the PATH in
the filesystem of the destination container. In this case, PATH must specify
a directory.

Using - as the second argument in place of a LOCALPATH will stream the
contents of the resource from the source container as a Tar Archive to
STDOUT.

vbatts · 2015-05-13T22:31:33Z

Oh my goodness. I wanted this some much last year, but we couldn't agree on the syntax of addresssing src and dest.

duglin · 2015-05-14T03:46:02Z

Not sure if it impacts your help comment, but FYI: #11858

jlhawn · 2015-05-14T03:57:24Z

@duglin I would definitely have to rebase if that gets merged before this ;-)

jlhawn · 2015-07-22T00:04:58Z

🎉 Thanks everyone!

tobegit3hub · 2015-07-22T01:27:50Z

Great job @jlhawn 👍

sa2ajj · 2015-07-22T03:52:25Z

😄

thaJeztah · 2015-07-22T06:07:07Z

Wow, it's there!! looks like you need to update your story #13171 (comment) @jlhawn :-)

tonistiigi · 2015-07-22T17:47:05Z

Some follow-up...

Symlink sources don't seem to rebase properly:

docker run --name t0 busybox sh -c "mkdir /foo && touch /foo/bar && ln -s /foo /baz"
docker cp t0:/baz bax
ls -l # should have bax/ but has foo/

It's because client expects base directory baz in tar when it does the replacement but daemon returns foo.

If I run docker cp id:/foo bar I would expect that the client protects me against anything being put outside local file/dir bar. Currently client trusts API to provide valid data in this case. If there is anything weird inside the tar then it can leak out at least one level.

How can we protect this from running containers updating a filesystem/volumes when the request is taking place? I mean we check for breakouts in the beginning of GET/PUT but if a container is using the filesystem it could just flip a symlink in the right time and then we would have full read/write to host.

tonistiigi · 2015-07-22T17:50:09Z

daemon/archive.go

Is this correct? GetResourcePath() should never return a symlink so I don't think this has much effect. Similar logic in ExtractToDir and comment in ArchivePath.

When I request stat for a symlink (with or without slash) I always get a directory as a response. AFAIK there isn't actually any harm of runnning Lstat on a path that is only joined and symlinks aren't evaluated. Reading/writing is different of course.

You're right, if FollowSymlinkInScope resolves all symlinks then that part doesn't matter. But, (It doesn't mention it in the comment), a trailing separator is also important because it asserts that the resource is a directory. The Lstat a couple of lines below this should capture that error condition (not a directory).

When I request stat for a symlink (with or without slash) I always get a directory as a response.

Is that when you stat a symlink on your local system or using this API?

Stat-ing a symlink with a trailing separator has different behavior depending on the system you're running on. Apparently on darwin, if a symlink foo points to a file bar and you call stat foo/ it will return stat info for bar even though bar is not a directory. On linux though, it will say stat: cannot stat 'foo/': Not a directory which is the error I expect it to pick up here.

onesuper · 2015-11-22T14:56:24Z

Great job @jlhawn

I have marked this serious and constructive discussion : )

calebebrim · 2015-12-07T23:32:26Z

I have tried something like this:

$ docker cp 0converted sleepy_rosalind:/home/test/data/aero_spectrum

cannot copy directory

What is wrong?
I'm running version 1.9.

where 0converted is an directory and aero_spectrum is another directory inside my container.

stevvooe · 2015-12-08T19:50:26Z

@calebebrim Please avoid commenting on closed issues. There are many other avenues to get support on using docker cp, such as the mailing list or IRC. If you still can't find a solution to your problem, you may then want to open an issue at that time.

This change changes the default for noOverwriteDirNonDir to be true internally, with the intent to change the default at the API to follow accordingly. The `AllowOverwriteDirWithFile` option in the Client was added when reimplementing the CLI using the API Client lib in [moby/moby@1b2b91b]. Before that refactor, the `noOverwriteDirNonDir` query argument [would be set unconditionally][1] by the CLI, with no options to control the behavior. The `noOverwriteDirNonDir` query parameter was added in [moby/moby@db9cc91] to set the `NoOverwriteDirNonDir` option that was implemented in pkg/archive in [moby/moby@a74799b]. It was added in [PR13171-comment2], following a discussion on the risk of replacing a directory with a file and vice-versa in [PR13171-comment]. > In my latest changes from yesterday: > > - Removed the `GET stat-path` endpoint and added a `HEAD` handler to > the `archive-path` endpoint. Updated the api docs to reflect this. > Also moved api docs changes from `v1.19` to `v1.20`. > - Added a `NoOverwriteDirNonDir` flag to `archive.TarOptions` to indicate > that we do not want to overwrite a directory with a non-directory (and > vice versa) when unpacking an archive. > - Added a corresponding but optional `noOverwriteDirNonDir` parameter > to the `PUT extract-to-dir` endpoint to specify desired behavior. > > These changes combine to keep the behavior we want It's unclear why these were added as an *option* and why it was implemented as opt-in (not opt-out), as overwriting a file with a directory (or vice-versa) would generally be unexpected behavior. [1]: https://github.com/moby/moby/blob/8c9ad7b818c0a7b1e39f8df1fabba243a0961c2d/api/client/cp.go#L345-L346 [moby/moby@1b2b91b]: moby@1b2b91b [moby/moby@a74799b]: moby@a74799b [moby/moby@db9cc91]: moby@db9cc91 [PR13171-comment]: moby#13171 (comment) [PR13171-comment2]: moby#13171 (comment) Signed-off-by: Sebastiaan van Stijn <[email protected]>

GordonTheTurtle added the status/0-triage label May 13, 2015

jlhawn mentioned this pull request May 13, 2015

docker cp to, from, and between containers #10198

Closed

jlhawn self-assigned this May 13, 2015

jlhawn added this to the 1.7.0 milestone May 13, 2015

jlhawn added status/assigned and removed status/0-triage labels May 13, 2015

jlhawn force-pushed the archive_copy branch 7 times, most recently from 658bce0 to 450e672 Compare May 13, 2015 21:23

jlhawn force-pushed the archive_copy branch 4 times, most recently from 3d410b7 to 677dfc4 Compare May 14, 2015 03:05

jlhawn force-pushed the archive_copy branch 5 times, most recently from 16edb82 to 693fbdd Compare May 14, 2015 20:52

GordonTheTurtle added the platform/windows label May 14, 2015

jlhawn force-pushed the archive_copy branch 3 times, most recently from e34da4b to 505e798 Compare May 14, 2015 22:55

jlhawn mentioned this pull request May 14, 2015

Respect shell globs when running docker cp. #12858

Closed

icecrime added impact/changelog impact/cli and removed area/security labels Jul 21, 2015

unclejack mentioned this pull request Jul 22, 2015

pkg/archive needs a refactor, extra maintenance #10038

Closed

tonistiigi reviewed Jul 22, 2015
View reviewed changes

jlhawn mentioned this pull request Jul 23, 2015

Fix copying of symlinks in containers #14885

Merged

rhuss mentioned this pull request Jul 24, 2015

new goal: mvn docker:run-build to speed up trying out a new build inside docker on a developer's laptop fabric8io/docker-maven-plugin#119

Open

vdemeester mentioned this pull request Jul 26, 2015

FATA[0000] invalid symlink when copying a symlink with relative parent paths #11394

Closed

jlhawn deleted the archive_copy branch July 31, 2015 18:02

muayyad-alsadi mentioned this pull request Aug 5, 2015

Proposal: Nested builds #7115

Closed

thaJeztah mentioned this pull request Aug 9, 2015

Moving a file from the host system to a container #905

Closed

thaJeztah mentioned this pull request Jan 5, 2017

update events related doc and swagger.yml #29906

Merged

jlhawn mentioned this pull request Jul 6, 2017

Fix docker cp dir with hard link #33956

Merged

thaJeztah mentioned this pull request Feb 9, 2021

Feature Request: Allow copying between containers #42006

Open

thaJeztah mentioned this pull request Apr 17, 2022

client: CopyToContainer(), CopyFromContainer(): remove status-code handling #43494

Merged

thaJeztah mentioned this pull request Sep 27, 2022

daemon: replaced exported errors with errdefs #44208

Merged

thaJeztah mentioned this pull request Jan 22, 2024

api: remove deprecated API versions (API < v1.24) #47155

Merged

3 tasks

thaJeztah mentioned this pull request Jul 13, 2025

cli/command/container: don't set CopyToContainerOptions.AllowOverwriteDirWithFile docker/cli#6176

Merged

thaJeztah mentioned this pull request Jul 13, 2025

daemon: ContainerExtractToDir: make AllowOverwriteDirWithFile opt-in #50404

Merged

Comments

Conversation

jlhawn commented May 13, 2015 • edited by thaJeztah Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Copy files/folders between containers and the local filesystem.

Uh oh!

vbatts commented May 13, 2015

Uh oh!

duglin commented May 14, 2015

Uh oh!

jlhawn commented May 14, 2015

Uh oh!

jlhawn commented Jul 22, 2015

Uh oh!

tobegit3hub commented Jul 22, 2015

Uh oh!

sa2ajj commented Jul 22, 2015

Uh oh!

thaJeztah commented Jul 22, 2015

Uh oh!

tonistiigi commented Jul 22, 2015

Uh oh!

tonistiigi Jul 22, 2015

Choose a reason for hiding this comment

Uh oh!

jlhawn Jul 22, 2015

Choose a reason for hiding this comment

Uh oh!

onesuper commented Nov 22, 2015

Uh oh!

calebebrim commented Dec 7, 2015

Uh oh!

stevvooe commented Dec 8, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

jlhawn commented May 13, 2015 •

edited by thaJeztah

Loading