Repository navigation
Closed as not planned
Closed as not planned
Copy link
Description
Description
Docker secrets specified using the long syntax for the docker-compose.yml file fail to set the specified uid, gid and mode.
Also, from the docs, the default value of the uid and gid fields should be the user that runs the container however the value remains whatever was set on the host machine.
Steps to reproduce the issue:
- Create a
Dockerfile
❯ cat Dockerfile
FROM ubuntu:20.04
RUN adduser tester -u 1005 --disabled-password
CMD ["/bin/bash", "-c", "ls -al /run/secrets/target_secret_file.txt"]
- Create a
docker-compose.yml
❯ cat docker-compose.yml
services:
secrets-tester:
build: .
secrets:
- source: some_secret_file
target: target_secret_file.txt
uid: "1005"
gid: "1005"
mode: 0440
secrets:
some_secret_file:
file: somefile.txt
- Create a text file for secrets (
somefile.txt)
❯ cat somefile.txt
Text from a secret file
- Run the service
docker compose run secrets-tester
Describe the results you received:
Received Output:
❯ docker compose run secrets-tester
-rw-r--r-- 1 1000 1000 24 Jul 13 16:47 /run/secrets/target_secret_file.txt
Describe the results you expected:
Expected Output:
❯ docker compose run secrets-tester
-r--r----- 1 1005 1005 24 Jul 13 16:47 /run/secrets/target_secret_file.txt
Additional information you deem important (e.g. issue happens only occasionally):
Same behavior is observed in these cases:
- Without creating the
testeruser - With same
sourceandtargetnames for the file indocker-compose.yml - Using the
uidandgidforroot - Using random values for
uidandgid - Different values for
mode
Output of docker compose version:
❯ docker --version
Docker version 20.10.17, build 100c701
❯ docker compose version
Docker Compose version v2.6.0
Output of docker info:
❯ docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
compose: Docker Compose (Docker Inc., v2.6.0)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 1
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc version: v1.1.2-0-ga916309
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.102.1-microsoft-standard-WSL2
Operating System: Ubuntu 20.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 7.763GiB
Name: DESKTOP-IGS6AOM
ID: MBVL:76QX:UWDQ:AK7Z:BNGW:PYGL:EXFG:ZHGD:JM53:RBFZ:4SQV:5I7Z
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
Additional environment details:
Reacted by Nikolai Gorodnov, Marcos de Oliveira, Sindre Nistad, surya-kasturi-lb, Rijnhard Hessel, Sam Herrmann, Peter Gerhards, Paulo Romeira, simon-liebehenschel, Ilya Vassyutovich and 20 more
Metadata
Metadata
Assignees
Labels
No labels
Activity
ev-adias commentedon Sep 5, 2022
Same thing happens to me. I'm trying to use secrets to workaround having to set the permissions on the host file, by setting the mode in the secret, but it end up with the exact same permissions of the host.
This issues comes up for me when using a keyfile for MongoDB replica authentication. The keyfile must not be world-readable, and in *nix this is easy to fix, in Windows (docker-desktop) it's not, so I was hoping that using secrets might fix the problem. It seems not
markkrj commentedon Oct 25, 2022
As bind mounts do not support different permissions (uid, gid and mode), I suspect that in order to support this, the content of such secrets must be copied into the container (as in #9553), or, use POSIX ACLs (more complex). Am I right @ndeloof?
But why does the compose documentation mention such options if they are not supported in the first place?
rijnhard commentedon Nov 15, 2022
well secrets and configs arent writeable (as per specification) so no reason why the implementation cant actually be a copy.
unless the idea is that they can change during runtime...
mario-gravel commentedon Dec 15, 2022
I'm using a secret for a SSH Private Key that must have a 0400 mode. Setting the uid, gid and mode is a requirement.
ndeloof commentedon Dec 15, 2022
Same issue applies both to Compose v1 and v2
the root cause is that both use a bind mount to "inject" secret inside container, but bind mount doesn't allow a distinct UID/GID/mode to be exposed inside container.
A possible alternative I can imagine is to replace this approach with a
copy(just like we do with support for secret set byenvironment). The sole drawback is that, doing so, changes to secret on host won't apply to secret inside container. Doesn't seem to be a major issue to me, but let me know if this would be a blocker for your usage.mario-gravel commentedon Dec 15, 2022
In my own opinion, having to rebuild the image to change a secret isn't an option. Major issue? Not now because we haven't reach the production, we're still developing. But once in production, it will be a security concern.
The access to the container repository isn't as secure as the access to the production host. Somebody pulling the image will have access to the private keys. Not a good thing at all !
There must be control over the uid, gid and mode for a secret file.
ndeloof commentedon Dec 15, 2022
This is not my proposal: compose would copy the secret into the container (not image) after creation, and before start. This is how secret based on environment is implemented already.
mario-gravel commentedon Dec 15, 2022
Oh, my bad ! Sorry 'bout that !
mario-gravel commentedon Dec 15, 2022
So if a secret change, we need to restart the container. Am I right ?
ndeloof commentedon Dec 15, 2022
yes indeed
mario-gravel commentedon Dec 15, 2022
Does it implies a redeploy of the container or just the starting phase will be enough ?
It would be great if a refresh can be triggered manually (ex: docker compose refresh-secrets) to redo the copy in the container. Having a watchdog to do this automatically may consume resources unnecessary. Secrets don't change often but being able to update them without a restart can be a good solution.
ndeloof commentedon Dec 15, 2022
we could copy secret again at any time, or have a dedicated command for this purpose, but the more obvious one would be to trigger a container
restartmario-gravel commentedon Dec 15, 2022
As long as the secret file can be updated on the fly and still have the good
uid,gidandmode, I'm good !56 remaining items
polarathene commentedon Mar 17, 2025
@ndeloof no worries! I appreciate all the time and effort put into figuring out a solution 😎
Slightly-offtopic, but is there any blockers for ID mapped support?
I did try to get it working with compose and volume mount options a while back but ran into some issue that presumably is a limitation with Docker Engine, it worked fine to create the equivalent mount on the host without docker involved though.
schaubl commentedon Mar 17, 2025
Hi all,
Happy to see discussions ongoing on this topic.
@polarathene: Yes it does resolve it for
filebased configs and secrets.The ones based on
environmentandcontentalready support setting theuid,gidandmode.My PR resolve it by using the same implementation as for
environmentandcontent(copying the file in the container).It doesn't support directories, only a single file (as for
environment,contentand swarm/stack).I don't have a very strong opinion on that but I don't think directories should be supported because I think it doesn't match the original concept behind configs and secrets and it also have some implementation issues like using
modewith sub-directories (should they have the same permissions as the files ?) and there are probably a lot of cases where not all the files should have the same permissions. Also the doc always mentions "file" and not "files"/"directories" (starting with the param name itself).However I do agree that in some cases being able to inject files and directories could be useful.
Maybe the solution for this point would be to have an extra parameter named like
path/files/directorieswhich would be specific to Docker Compose (on top offilewhich will keep similar behaviour as in swarm/stack).@ndeloof: Would it be enough to add a big warning of the behaviour change (and the way to update compose files) in the release notes as you initially mentioned here #11867 (comment) ?
I don't know how many users rely on the bind mount behaviour but I find strange to use configs/secrets for this when bind mounts are already available. The only use case I can see is if a user uses a secret to share sensitive informations to the container but this is even worse because the user think there are extra security measures which is not the case.
In case what the user wants is a bind mount, there is a drop-in replacement (bind mounts) but unfortunately if we have to inject a file into a container when
docker composeis executed from a remote machine, there is no alternative.polarathene commentedon Mar 18, 2025
That happened for me when I initially came across it and thought from the docs (at least back then, haven't checked the current docs for it) that it was meant to be more secure, along with the
uid/gid/modefeatures to ensure it is when providing a file.It's only because I noticed in the container none of those settings actually applied outside of swarm that I realized it was effectively a misleading bind mount with different syntax added as a fallback convenience for swarm users (actual justification for it IIRC).
As for avoiding regression reports from existing users... I suppose until you're ready to make it a breaking change, one option is to go opt-in as suggested with something like
secure: trueas the opt-in attribute.Then clearly document that the
secretsfeature on non-swarm is not secure by default for legacy reasons, and requires additional explicit opt-in to toggle on the secure feature, with the caveat you lose the ability to sync updates from the host? 🤷♂Technically since
uid/gid/modearen't supported in the existing support and this is specific to behaviour on non-swarm usage...there's no reason those users should be configuring those given they don't do anything thus you might as well make any of those the opt-in trigger instead, nothing new to add then. Documentation just needs to be clear that those options will break the current functionality of being treated as a typical bind mount? 🤔 (EDIT: Oh I just realized that the referenced PR is doing exactly that for opt-in viauid/gid/modeusage)ndeloof commentedon Mar 18, 2025
A common pattern is to declare a whole
config.dfolder as a composeconfigand use it inside container. Many tools support this modularity pattern so that user don't hav to maintain a huge config file but can provide fragmentsThis was introduced to offer some parity with docker swarm, then users started relying on it in development to get immediate update on config changes (which could be implemented with
watchnowadays). Then, with thousands example and stackoverflow suggestions you'll see tons of users to select this option without any specific reason. Still we have to manage our legacy to prevent compatibility breaks.schaubl commentedon Mar 19, 2025
@ndeloof thanks for your answers.
To manage the legacy, I think it could be possible to create an environment variable like
USE_OLD_CONFIGS_SECRETSwhich will default toTrue/1(similarly to theCOMPOSE_BAKEintroduced recently in v2.33.0).At the same time, a warning can be added in the release notes to mention that the default value of this variable will, in the future, be changed to
False/0.With this, the current implementation will still be applied by default (for now) while allowing users to use the new implementation (by setting
USE_OLD_CONFIGS_SECRETStoFalse) and for users relying on the current behaviour, they will have time to either migrate to bind mounts or to setUSE_OLD_CONFIGS_SECRETStoTrue.@ndeloof What do you think about this strategy ?
Regarding the PR #12625, if I understood properly, bind mounts will still be used for
fileexcept if one of theuid,gidormodeparams are set.I have the impression that this could adds more confusion because the behaviour on handling the
filewill be completely different based on the usage or not of at least one of the 3 parameters mentioned above.This is especially true when the
docker composecommand is executed from a remote machine. In this case, for example, if I want to inject my/etc/hostsfile into the container, it will work if I explicitly set itsmodebut if I don't, it will take the/etc/hostsof the machine where the docker daemon is.Also the behaviour can be different inside the same compose file, because the
fileparam is in the top-levelconfigsbutmodeis in the service-levelconfigsso if I have a config used by two services where I need to change the permissions for only one service, the two different behaviour will be used and with my example with the/etc/hostsfile, I will end up with two different file contents in the two containers even that they both use the same config.ndeloof commentedon Mar 19, 2025
@schaubl fully agree. #12625 is just an experiment to prevent breaking changes, still unpleasant - also comes with multiple drawbacks I can't detail here. As you noticed,
secretssupport in compose was designed with local engine in mind (or Docker Desktop) but won't support an actual remote engine. On the other hand, if we usecopyto inject files inside container, we then miss a mechanism to enforce consistency when local config/secret is updated.Please note this is exactly what some expect !
The best option I have in mind is for docker engine to enable secrets/config support in standalone (non-swarm) mode. I can't tell about how complex this could be, but for sure the code is already there :P cc @colinhemmings
@thaJeztah I haven't found an open issue requesting such a feature on moby repo, does this ring a bell ?
docker-compose.yml: remove secrets mode attribute
docker-compose.yml: remove secrets mode attribute (#2068)
docker-compose.yml: remove secrets mode attribute
GammaGames commentedon Jun 12, 2025
I found this thread after coming across this in the docs:
Am I reading this wrong, or does the documentation have and invalid example?
Edit: Maybe because it's part of the spec and other programs support it
ianhinder commentedon Jun 12, 2025
Yes, I think it's because it is part of the spec, but docker compose doesn't implement that part of the spec. And as far as I know, docker swarm mode doesn't implement ComposeV2 either, so I don't know that there is any implementation of this.
ref
vanyauhalin commentedon Mar 21, 2026
Please, add a note to this section that docker compose does not support this feature. Thank you.
https://docs.docker.com/reference/compose-file/services/#configs
stefanfed commentedon May 24, 2026
What is the correct, secure way to have the contents of a local file readable only by root passed as a secret to an unprivileged container?
stalb commentedon May 24, 2026
Either change the uid/gid/mode of the file which contains the secret or use an inline secret (cf https://docs.docker.com/reference/compose-file/secrets/#example-2).
When using inline secrets (as in this example: https://docs.docker.com/reference/compose-file/services/#long-syntax-5), specifying uid/gid/mode works.
Fix the host-uid secret class for pgBouncer and pgAdmin