Skip to content

Reorganize Run Reference and CLI Run Reference for clarity#26791

Closed
mdlinville wants to merge 2 commits intomoby:masterfrom
mdlinville:14384_improve_run_docs
Closed

Reorganize Run Reference and CLI Run Reference for clarity#26791
mdlinville wants to merge 2 commits intomoby:masterfrom
mdlinville:14384_improve_run_docs

Conversation

@mdlinville
Copy link
Contributor

Fixes #14384

  • Expanded and reorganized the Run Reference
  • Moved some examples from CLI reference to Run Reference
  • Removed some less useful examples from CLI reference
  • Split out resource constraints content into a new topic and
    expanded the info

I apologize in advance because this is big and may be hard to review

Signed-off-by: Misty Stanley-Jones [email protected]

@mdlinville
Copy link
Contributor Author

@thaJeztah PTAL and deputize others who can help review this.

@thaJeztah
Copy link
Member

great work @mstanleyjones, I'll go through this, and review

ping @justincormack @cpuguy83 @aaronlehmann

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick read, no full review yet (left some comments)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"at image creation" may be confusing (could be confused with / interpreted as baking the constraints into the image). Given that docker build does not allow setting all these, I'm not sure it should be mentioned

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, we don't include possible future enhancements (also there's a markdown error in the link, one ] too many)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/when Docker detects/when the kernel detects/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing back-tic after "memory"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"a container's kernel" -> needs some change, as a container doesn't have a kernel (all containers share the host's kernel)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer a markdown bullet-list for this, but let me know if that complicated things (just indenting more should create a nested bullet-list in markdown)

@mdlinville
Copy link
Contributor Author

Addressed all of @thaJeztah feedback so far.

@mdlinville
Copy link
Contributor Author

This needs to be closed out by tomorrow EOD so we can migrate to the new repo. If it doesn't seem feasible to finish the review there, I will redo the PR against that repo after it is all sorted out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should use --privileged in any examples, can we just use --cap-add sys_admin instead?

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments / suggestions. Unfortunately, GitHub doesn't let me review the actual run.md - diff too large to be displayed 😞 (hmf)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/system/system's/ ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/control control/control/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there other resources we don't currently support? Tricky to mention just this one, if there's potentially other things as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/memory/Memory/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we explain 4m is 4 megabytes?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add "in the background" / "detached" here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, always tricky, as localhost may not be where it's accessible, not sure if we should explain that here though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mention that --cap-add is a better approach in most (if not all) cases?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a follow-up; looks like we only describe bind mounting host directories here, and not actual volumes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah
Copy link
Member

ping @mstanleyjones looks like part of this PR now needs to go into the other repository. May need a check though if there's no changes in there that are for 1.13 only

@mlaventure
Copy link
Contributor

ping @mstanleyjones, please TAL :)

@mdlinville
Copy link
Contributor Author

Rebased and part of this PR split out into docker/docs#334. @thaJeztah @dnephin PTAL

@LK4D4
Copy link
Contributor

LK4D4 commented Nov 11, 2016

@thaJeztah @dnephin @mstanleyjones what's up here?

@dnephin
Copy link
Member

dnephin commented Nov 14, 2016

@LK4D4 we're pulling in changes from the docs repo so that we can publish reference docs from docker/docker, instead of having a copy in both repos.

@mdlinville
Copy link
Contributor Author

@thaJeztah PTAL and let me know if this is ready. I had to rebase again to resolve more conflicts.

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mstanleyjones Not yet finished reviewing, I'll try to complete the rest this weekend 😄

Copy link
Member

@thaJeztah thaJeztah Nov 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be sys_admin. The daemon (or RunC, should check) automatically prepends CAP_;

docker: Error response from daemon: linux spec capabilities: Unknown capability to add: "CAP_cap_sys_admin".

In most examples we use uppercase, i.e. --cap-add SYS_ADMIN

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"and it persists even if no volume is" -> "and it persists even if no container is"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should replace the primary example here. publish to 127.0.0.1 is probably not the most used option,
-p 8080:80 would be a more common example (publishes port 80 of the container on port 8080 of the host's public interface).

For another rewrite; we should put more emphasis on when and when not to publish ports. I see a lot of confusion by people; for containers to communicate on the same network, no ports have to be published. So, for example if you have a MySQL container running and only the containers need to have access, do not publish its port.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"expose" may be for an "advanced" section as well. Expose does not expose, only "advertises" that the container may have something listening on port 80. It's only adding metadata, and still requires to actually have something running in the container that's listening on that port 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i.e. there's been a lot of confusion about this as well; even without "expose", every port of the container can be accessed on the container-container network, or (when published) externally)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub does not allow me to review the reference/run.md document, so I'll add comments here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker run command is a central Docker command. In short, it evaluates the Dockerfile for your container's image and downloads any layers your Docker client does not have locally

The run command does not evaluate the Dockerfile. The Dockerfile is only used to build an image. The run command is a combination of docker create (, implicitly: docker pull), and docker start.

docker run foobar:latest;

  • attempts to create a new container from the foobar:latest image
  • if the foobar:latest image is not available locally, it is pulled from Docker Hub
  • after the image is successfully pulled;
  • creates a new container from the foobar:latest image using the options specified on the run command;
    • mounts the image read-only to use as rootfs for the container
    • mounts a writable layer on top of the mounted image
    • etc..
  • once the container is created, starts the container, creates network endpoints and adds them to the container, mounts volumes etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicitly named volumes are not removed. by docker run or docker rm.

Stray full-stop after "removed"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many docker run flags have a long and short form. For instance, -i is equivalent to --interactive. When a short-form flag takes an ....

It's mentioned further down the document, but perhaps we should mention here that short flags can be concatenated; -a -b -c is equal to -abc.

Actually wondering if we describe that short flags have a single dash (-), and long flags have a double dash (--).

Should this information as it is general for our CLI, be moved to https://docs.docker.com/engine/reference/commandline/cli/ ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are running the container in privilged mode

typo privilged

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running a container as a different user

Most Docker containers run as the root user (UID=0). If an image is
created with additional users (such as by adding the useradd to a RUN
directive in the Dockerfile), you can specify that user's login or UID
using the -u or --user flag. The user must be valid and have permission
to run the command specified in the Dockerfile or at runtime.

Most Docker containers run as the root user (UID=0).

It's the default if no user is set (in the image). I'm not sure we should say the "most Docker containers..". It feels as if we agree with that (whenever possible, you should drop to a non-privileged user inside the container)

The user must be valid

If you use numeric uid:gid, there is no need to create a user, only
if a username:groupname is used. Just as chown 123:456 something will always work,
also if no user/group with those ID's exists in /etc/passwd

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overriding the image's working directory

The default working directory for most containers is the root directory (/).

There's many images that define a different working directory. I don't think we should use "most" here; perhaps describe that it can be set by the image author, and you can override at runtime.

The working directory must exist and be accessible by the container's user.

The working directory does not have to exist. If it does not exist, it's created automatically. However, (at least in docker 1.12) the workdir is created as "root";

docker run -it --rm -u 123:456 --workdir /foo/bar alpine sh
/foo/bar $ ls -lah
total 8
drwxr-xr-x    2 root     root        4.0K Nov 18 23:47 .
drwxr-xr-x    3 root     root        4.0K Nov 18 23:47 ..
/foo/bar $ echo foo -> bar
sh: can't create bar: Permission denied

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mounting extra volumes

I see the information about ro, rw, SELinux labeling (z / Z) and mount propagation is removed. Did this content move somewhere else?

Only anonymous volumes can be specified in a Dockerfile. See the Dockerfile reference for more information.

Please make Dockerfile reference a link

Duplicating another container's volumes

By default, volumes are mounted read-write, so containers that share a volume can share files using the volume.

That's correct, but when using volumes-from, the mount-options are copied (ro/rw, and if I'm correct, z/Z, mount propagation as well).

This was the description before this PR:

"By default, the volumes are mounted in the same mode (read write or read only) as the reference container."

Viewing mounts from within a container

I wonder if we need to mention this; is there a use case for inspecting mounts inside the container?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overriding or augmenting the image's environment variables

Quite some information got lost in the rewrite; I'll try to summarize them;

If no value, and no = is set for an environment variable passed by
-e / --env, the value from the current environment is used. For
example;

$ export FOOBAR=hello && MY_NAME=sebastiaan docker run --rm -e MY_NAME -e FOOBAR alpine env

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=e9af7a0f8fef
MY_NAME=sebastiaan
FOOBAR=hello
HOME=/root 

The order in which -e, --env, and --env-file is processed is important
information, and no longer mentioned. From the old version;

Regardless of the order of these three flags, the --env-file are
processed first, and then -e, --env flags. This way, the -e or --env
will override variables as needed.

It's no longer mentioned that env-files can have comments

Env-files also can take exported values from the environment;

$ cat my-env-file

# foobar does not have a value set, so takes its
# value from the environment
FOOBAR

# foobar2 _does_ have a value set,
# but its value is empty.
# consequently, it does _not_ take
# its value from the environment
FOOBAR2=
BLA=HAHA
DONT="dont use quotes"
BECAUSE=because the file format simply splits at the first `=` and takes everything after as value
MULTI_LINE=multiple \n lines \n are not possible
$ export FOOBAR=hello
$ export FOOBAR2=hello2

$ docker run --rm --env-file=./my-env-file alpine env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=ec63ff387cc4
FOOBAR=hello
FOOBAR2=
BLA=HAHA
DONT="dont use quotes"
BECAUSE=because the file format simply splits at the first `=` and takes everything after as value
MULTI_LINE=multiple \n lines \n are not possible
HOME=/root

The following is an example of a correctly-formatted environment variable file:

The format is "correct", but it won't give you what you expect; the
format does not use quotes, so the quotes are now part of the value
of the env-var.

Worth noting is that there is no official specification for "valid" and
"invalid" names for environment variables. For that reason, we don't
do any validation of the environment-variable's names, and leave it
to the container / shell's responsibility to either fail or succeed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exposing and publishing extra ports

We should provide more information about expose here. As mentioned in
another comment, expose basically does nothing, it only adds metadata
on the container that;

  • allows tooling to docker inspect the container and know what ports
    may be used (but no guarantees, because there may not be something
    listening)
  • it is used for -P / --publish-all to automatically publish all
    ports that are marked as "exposed".

The important thing here is that;

  • on the docker (container) network, all ports of a container are "exposed",
    there are no restrictions for other containers, or processes that
    have access to the network to connect to any port of the container.
  • if a container is using IPv6 and has a routable IP-address, all ports
    of the container are exposed, unless custom firewall/IPTables rules
    are present.

Publishing ports

In these examples;

  • I suggest to remove the --expose, as it may add to the assumption
    that --expose is needed in order to publish a port (which is not
    the case), simply; $ docker run -p 8080:80
  • Perhaps use a functional example image (nginx:alpine could be a
    good choice as it's small, and actually serves something)
  • All examples use -it --rm. This makes it difficult to inspect / try
    the example without opening a second shell. Consider using -d

Also, the -p / --publish flag has many options not mentioned here.
possibly some of them are mentioned in the networking section, but I
think we should have a central location to describe them. For example;

  • -p 80:80/tcp -p 80:80/udp to publish udp/tcp
  • mention that a single port can be mapped multiple times; -p 81:80 -p 82:80
  • or on multiple interfaces -p 192.168.10.20:80:80 -p 192.168.10.30:80:80
  • port ranges in which to pick a random target port -p 8000-9000:80
  • a range of ports to publish on a range of random ports -p 8000-8010
  • a range of ports to publish on a specified range of host ports: -p 8000-8010:80-90
  • parts can be omitted when using these options, e.g.: -p 192.168.10.20::80 to use a random host port, but a specific IP-address / interface

Changing or disabling your container's network

  • I miss the tables 😢. This may be highly personal, but having the
    tables allow me to quickly scan the page, and find which options are available.
    The table can be limited to a short description, and details following it.
    In the new format, there's a lot of text, and it's hard to find a specific
    option. Maybe that's meant to be in "reference", but I think we should
    have this somewhere
  • Using another container's network-space may deserve it's own section.
    we should add more information though (can be in a follow up), as it's
    an advanced feature, but very useful.
  • Do we still have the information that was removed? For example,
    https://docs.docker.com/engine/userguide/networking/#/default-networks
    only mentions "The host network adds a container on the hosts network
    stack. You’ll find the network configuration inside the container is
    identical to the host."
    The sections that were removed contained some
    details that should be available somewhere. No longer having a separate
    section for these network modes / networks makes it more difficult
    to refer users to the right information. For example, I can no longer
    link to https://docs.docker.com/engine/reference/run/#/network-host

@mdlinville
Copy link
Contributor Author

Addressed everything before #26791 (comment). The rest tomorrow.

@vdemeester
Copy link
Member

ping @mstanleyjones @thaJeztah 🙇

@mdlinville
Copy link
Contributor Author

Wow I can't believe this is still open! Let me rebase.

Fixes #14384

- Expanded and reorganized the Run Reference
- Moved some examples from CLI reference to Run Reference
- Removed some less useful examples from CLI reference
- Split out resource constraints content into a new topic and
  expanded the info

Signed-off-by: Misty Stanley-Jones <[email protected]>
@mdlinville
Copy link
Contributor Author

Rebased.

@thaJeztah
Copy link
Member

@mstanleyjones wasn't sure if you worked on it after #26791 (comment)

@LK4D4
Copy link
Contributor

LK4D4 commented Jan 27, 2017

ping @mstanleyjones @thaJeztah

@GordonTheTurtle GordonTheTurtle added the dco/no Automatically set by a bot when one of the commits lacks proper signature label Jan 27, 2017
@mdlinville
Copy link
Contributor Author

Resolved conflict.

@GordonTheTurtle GordonTheTurtle removed the dco/no Automatically set by a bot when one of the commits lacks proper signature label Jan 27, 2017
@AkihiroSuda
Copy link
Member

@AkihiroSuda AkihiroSuda closed this Jul 4, 2017
@mdlinville
Copy link
Contributor Author

Geez this was an old PR. Maybe it's not worth trying to get it in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants