What's the usecase for this?

@duglin it's described quite well in #26234
but you may, for example, wish to let your containers know a little about where they are running.
a concrete example would be that some big data applications like to know which rack, aisle etc.. that they are being run on...
you could also use this to propagate http proxy settings in to your containers if you are operating in an environment where you know upfront that any containers run will require a proxy to be able to access WWW.

A note to maintainers... I'm still working on this, but pushing very early to try and get the tests running on CI as building locally is proving difficult for some reason

Can we use daemon labels instead of daemon env vars?
(It will be exposed to containers via on-going introspection filesystem #24893 )

For proxy settings, I think perhaps it should be tied to a network rather than the daemon

Thinking about this... I'm wondering about people who are very sensitive to the env vars in their containers - they might be annoyed with new ones showing up that they have no control over. One of the benefits of containers/images is that I can know for sure exactly what the container will look like. With this I lose that control. I can see providers injecting all sorts of things into people containers with this feature and container owner can't stop it.

Today new env vars can be injected (via -e on docker run) but that's a opt-in approach - which is good. I wonder if this should be an opt-in thing as well via something like a --system-envs flag. I thought about a flag to disable it, but the default should be to NOT muck with people.

@duglin if you set things on the daemon, that is opt in too as far as I see it. Ok it is one step removed, but we allow a lot of defaults to be set there. I think the idea here was you would be able to unset them with -e var= at run time.

That's a different level of opt-in sure, but then you would require each person (on each host) to modify their run cmd based on that host's daemon env vars. Keep in mind there are envs where the person doing the 'docker run' isn't the same person who owns the machine so they have no idea what daemon-level env vars might be set until its too late. And in a swarm setup if there are variants between each host's env var list (which I hope wouldn't be true, but ya never know) then since the host my app is put is on is random, there may be no way for me to unset things in a consistent way.

Client-side "docker run" and config file options to specify this (either opt-in or out depending on which default is chosen at the host level) would really help and give the person managing the container the control they need.

We discussed this in a maintainers review session, and we're not sure about this; if the introspection API is implemented, this would probably allow the daemon to obtain this information as well, and may be a cleaner approach (#26331). Would that solve the use case?

Also, perhaps this is something for Swarm (dynamically set environment variables, based on where a container is scheduled?)

ping @dave-tucker ^^ wdyt?

@thaJeztah Introspection doesn't work for this use case... Unfortunately, only environment variables will cut it as this is what the applications are already expecting.

While it could theoretically be implemented in Swarm it wouldn't fix the use case for a single daemon... unless if were a Swarm of one of course.

Updated this PR. It now works! ... but I'm having some trouble with integration-cli.
While it looks like the daemon is started with the correct arguments, the environment variables aren't appearing in containers. When I run this manually it works 100% of the time.
Could be an environment issue on my end so will let CI have the final say ;)

@thaJeztah time to discuss again?
I'm quite reluctant to this because I think new features is extremely heavy on us and use-case in #26234 does not require daemon-wide option for sure.

@LK4D4 @dave-tucker yeah, I'm not convinced on the "any env var" use case. We were discussing a use case for allowing just HTTP_PROXY , HTTPS_PROXY etc, which may make more sense (my 0.02c).

For example, what happens if I set MYSQL_ROOT_PASSWORD as a daemon level env-var, and I run the official MySQL image? https://hub.docker.com/_/mysql/

We discussed this in the maintainers meeting, and we are ok with adding just the HTTP_PROXY (etc.) on the daemon; basically we want to limit to the default options that --build-arg supports.

We do have one concern; how can we unset an environment variable on a single container? We need to have a solution for that before merging / going ahead

@tianon just came up with another question; are these env-vars baked in into images I build on that daemon? For build, we have --build-args for that; how does this PR handle this?

@thaJeztah simple, order of overrides is as follows:

daemon global -> image -> run

So to override you may either bake an env variable in to an image or supply it at runtime with -e

I hadn't considered the build-args interaction.
In that case I would suggest that it were to support a similar override behavior:

daemon global -> build-args

IMO, the resulting image should not bake in the daemon global variables. Does this happen today with build-args?

simple, order of overrides is as follows:

We should look at unsetting though; -e FOO="" is not the same as FOO not being set. (And, no -e FOO doesn't work, because that copies the value of $FOO in the environment if it's exported/set)

IMO, the resulting image should not bake in the daemon global variables. Does this happen today with build-args?

No, build-args are specifically designed to not be persisted in the image; that way you can set a proxy during build, but don't have it persisted in the final image. build-args are visible in the image's history though (something to take into account), because they are prefixed for each run;

docker history --no-trunc foo

IMAGE                                                                     CREATED             CREATED BY                                                                                          SIZE                COMMENT
sha256:317b6d045ac269d65f9915a02cbef61802957ab79a96012325926dd45c87197d   17 seconds ago      |1 HTTP_PROXY=user:[email protected] /bin/sh -c echo "hello"                                         0 B
sha256:e02e811dd08fd49e7f6032625495118e63f597eb150403d02e3238af1df240ba   5 weeks ago         /bin/sh -c #(nop)  CMD ["sh"]                                                                       0 B
<missing>                                                                 5 weeks ago         /bin/sh -c #(nop) ADD file:ced3aa7577c8f970403004e45dd91e9240b1e3ee8bd109178822310bb5c4a4f7 in /    1.093 MB

They don't persist in the image layers, unless I make them (e.g. ENV FOO=$HTTP_PROXY)

@thaJeztah thanks.
You are right re: unsetting, I'd forgotten that -e FOO="" is not equivalent.
Perhaps -e !FOO might be a good way of un-setting it. WDYT?

In relation to --build-args I'm not sure how to proceed.

We can either:
a) Disable this feature for build.
b) Allow this feature to replace build-args for proxy variables if they are configured on the daemon. This is backwards compatible as build-args will override the daemon arguments anyway, but IMO would be a major UX benefit - especially if you don't know that your daemon is behind a proxy.

My original intention was a), but I'd be happy to look at implementing b).

ping @dave-tucker @thaJeztah @tianon what is the status here ?

@vdemeester well, the code has bit-rotted quite substantially since I opened the PR 😨
I'm going to try to update this next week and incorporate all feedback so far so we can look at the design again...

Addressed feedback from @AkihiroSuda and @thaJeztah.
My tests are failing now with docker run exiting 125... still debugging my end but hopeful that a solution is in sight

Running tests locally...

PASS: docker_cli_run_test.go:4419: DockerDaemonSuite.TestRunWithProxyEnv	1.835s
PASS: docker_cli_run_test.go:4427: DockerDaemonSuite.TestRunWithProxyEnvOverride	1.855s
OK: 2 passed
PASS

hmmmm.....

Remind me of the Docker make command to run the tests for a specific package?

for integration tests : TESTFLAGS="-check.f DockerDaemonSuite.TestRun" make test-integration-cli will run all tests in the DockerDaemonSuite starting by TestRun.
@alexellis ^^

Excuse my clumsy rebasing

@vdemeester the TESTFLAGS=.. should go after make (before usually works, but I think I've seen cases where it made a difference)

Windows CI failures appear unrelated

Left a comment in #30588 (comment)

Closing this one as well for now (#30588 (comment)), but we'll definitely want some options for this, so ping me as well to re-open 🤗

I'm sure @alexellis will remind us as well 😄