Skip to content

Secrets fail to set the uid, gid and mode specified in docker-compose.yml #9648

Closed as not planned

Description

@sanchitbapat

Description

Docker secrets specified using the long syntax for the docker-compose.yml file fail to set the specified uid, gid and mode.

Also, from the docs, the default value of the uid and gid fields should be the user that runs the container however the value remains whatever was set on the host machine.

Steps to reproduce the issue:

  1. Create a Dockerfile
❯ cat Dockerfile
FROM ubuntu:20.04

RUN adduser tester -u 1005 --disabled-password
CMD ["/bin/bash", "-c", "ls -al /run/secrets/target_secret_file.txt"]
  1. Create a docker-compose.yml
❯ cat docker-compose.yml
services:
  secrets-tester:
    build: .
    secrets:
      - source: some_secret_file
        target: target_secret_file.txt
        uid: "1005"
        gid: "1005"
        mode: 0440

secrets:
  some_secret_file:
    file: somefile.txt
  1. Create a text file for secrets (somefile.txt)
❯ cat somefile.txt
Text from a secret file
  1. Run the service docker compose run secrets-tester

Describe the results you received:
Received Output:

❯ docker compose run secrets-tester
-rw-r--r-- 1 1000 1000 24 Jul 13 16:47 /run/secrets/target_secret_file.txt

Describe the results you expected:
Expected Output:

❯ docker compose run secrets-tester
-r--r----- 1 1005 1005 24 Jul 13 16:47 /run/secrets/target_secret_file.txt

Additional information you deem important (e.g. issue happens only occasionally):
Same behavior is observed in these cases:

  1. Without creating the tester user
  2. With same source and target names for the file in docker-compose.yml
  3. Using the uid and gid for root
  4. Using random values for uid and gid
  5. Different values for mode

Output of docker compose version:

❯ docker --version
Docker version 20.10.17, build 100c701

❯ docker compose version
Docker Compose version v2.6.0

Output of docker info:

❯ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 1
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Ubuntu 20.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.763GiB
 Name: DESKTOP-IGS6AOM
 ID: MBVL:76QX:UWDQ:AK7Z:BNGW:PYGL:EXFG:ZHGD:JM53:RBFZ:4SQV:5I7Z
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional environment details:

Activity

  1. ev-adias commented on Sep 5, 2022

    @ev-adias

    Same thing happens to me. I'm trying to use secrets to workaround having to set the permissions on the host file, by setting the mode in the secret, but it end up with the exact same permissions of the host.

    This issues comes up for me when using a keyfile for MongoDB replica authentication. The keyfile must not be world-readable, and in *nix this is easy to fix, in Windows (docker-desktop) it's not, so I was hoping that using secrets might fix the problem. It seems not

  2. markkrj commented on Oct 25, 2022

    @markkrj

    As bind mounts do not support different permissions (uid, gid and mode), I suspect that in order to support this, the content of such secrets must be copied into the container (as in #9553), or, use POSIX ACLs (more complex). Am I right @ndeloof?
    But why does the compose documentation mention such options if they are not supported in the first place?

  3. rijnhard commented on Nov 15, 2022

    @rijnhard

    well secrets and configs arent writeable (as per specification) so no reason why the implementation cant actually be a copy.
    unless the idea is that they can change during runtime...

  4. mario-gravel commented on Dec 15, 2022

    @mario-gravel

    I'm using a secret for a SSH Private Key that must have a 0400 mode. Setting the uid, gid and mode is a requirement.

        secrets:
          - source: ssh_host_ecdsa_key
            target: /etc/ssh/ssh_host_ecdsa_key
            mode: 0400
    
    2022-12-14 20:41:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    2022-12-14 20:41:23 @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
    2022-12-14 20:41:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    2022-12-14 20:41:23 Permissions 0777 for '/etc/ssh/ssh_host_ed25519_key' are too open.
    2022-12-14 20:41:23 It is required that your private key files are NOT accessible by others.
    2022-12-14 20:41:23 This private key will be ignored.
    2022-12-14 20:41:23 sshd: no hostkeys available -- exiting.
    
  5. self-assigned this
    on Dec 15, 2022
  6. ndeloof commented on Dec 15, 2022

    @ndeloof
    Contributor

    Same issue applies both to Compose v1 and v2
    the root cause is that both use a bind mount to "inject" secret inside container, but bind mount doesn't allow a distinct UID/GID/mode to be exposed inside container.

    A possible alternative I can imagine is to replace this approach with a copy (just like we do with support for secret set by environment). The sole drawback is that, doing so, changes to secret on host won't apply to secret inside container. Doesn't seem to be a major issue to me, but let me know if this would be a blocker for your usage.

  7. mario-gravel commented on Dec 15, 2022

    @mario-gravel

    In my own opinion, having to rebuild the image to change a secret isn't an option. Major issue? Not now because we haven't reach the production, we're still developing. But once in production, it will be a security concern.

    The access to the container repository isn't as secure as the access to the production host. Somebody pulling the image will have access to the private keys. Not a good thing at all !

    There must be control over the uid, gid and mode for a secret file.

  8. ndeloof commented on Dec 15, 2022

    @ndeloof
    Contributor

    having to rebuild the image to change a secret isn't an option

    This is not my proposal: compose would copy the secret into the container (not image) after creation, and before start. This is how secret based on environment is implemented already.

  9. mario-gravel commented on Dec 15, 2022

    @mario-gravel

    Oh, my bad ! Sorry 'bout that !

  10. mario-gravel commented on Dec 15, 2022

    @mario-gravel

    So if a secret change, we need to restart the container. Am I right ?

  11. ndeloof commented on Dec 15, 2022

    @ndeloof
    Contributor

    yes indeed

  12. mario-gravel commented on Dec 15, 2022

    @mario-gravel

    Does it implies a redeploy of the container or just the starting phase will be enough ?

    It would be great if a refresh can be triggered manually (ex: docker compose refresh-secrets) to redo the copy in the container. Having a watchdog to do this automatically may consume resources unnecessary. Secrets don't change often but being able to update them without a restart can be a good solution.

  13. ndeloof commented on Dec 15, 2022

    @ndeloof
    Contributor

    we could copy secret again at any time, or have a dedicated command for this purpose, but the more obvious one would be to trigger a container restart

  14. mario-gravel commented on Dec 15, 2022

    @mario-gravel

    As long as the secret file can be updated on the fly and still have the good uid, gid and mode, I'm good !

  15. 56 remaining items

  16. polarathene commented on Mar 17, 2025

    @polarathene

    @ndeloof no worries! I appreciate all the time and effort put into figuring out a solution 😎

    Slightly-offtopic, but is there any blockers for ID mapped support?

    • I think that may be limited to rootful containers, and may be dependent upon Docker Engine moving to containerd 2.0 (I thought that was the case with v28 due to some PRs bumping containerd deps to 2.0, but it still seems to ship with 1.7.x).
    • It'd allow for bind mounts with UID/GID mapping AFAIK, not sure about mode changes though.

    I did try to get it working with compose and volume mount options a while back but ran into some issue that presumably is a limitation with Docker Engine, it worked fine to create the equivalent mount on the host without docker involved though.

  17. schaubl commented on Mar 17, 2025

    @schaubl

    Hi all,
    Happy to see discussions ongoing on this topic.


    #12448 (comment)
    This would resolve the lack of support apparently.

    @polarathene: Yes it does resolve it for file based configs and secrets.
    The ones based on environment and content already support setting the uid, gid and mode.
    My PR resolve it by using the same implementation as for environment and content (copying the file in the container).
    It doesn't support directories, only a single file (as for environment, content and swarm/stack).


    I don't have a very strong opinion on that but I don't think directories should be supported because I think it doesn't match the original concept behind configs and secrets and it also have some implementation issues like using mode with sub-directories (should they have the same permissions as the files ?) and there are probably a lot of cases where not all the files should have the same permissions. Also the doc always mentions "file" and not "files"/"directories" (starting with the param name itself).
    However I do agree that in some cases being able to inject files and directories could be useful.
    Maybe the solution for this point would be to have an extra parameter named like path/files/directories which would be specific to Docker Compose (on top of file which will keep similar behaviour as in swarm/stack).


    @polarathene #12448 coms with some limitations and breaking changes (which I tried to address in #12625). Those are significant enough so that we don't want to silently switch from bind mount to such a solution without an explicit opt-in by compose file author, otherwise we will have to deal with regression reports (many users rely on the bind mount behavior to get config/secret dynamically updated in container)

    @ndeloof: Would it be enough to add a big warning of the behaviour change (and the way to update compose files) in the release notes as you initially mentioned here #11867 (comment) ?
    I don't know how many users rely on the bind mount behaviour but I find strange to use configs/secrets for this when bind mounts are already available. The only use case I can see is if a user uses a secret to share sensitive informations to the container but this is even worse because the user think there are extra security measures which is not the case.
    In case what the user wants is a bind mount, there is a drop-in replacement (bind mounts) but unfortunately if we have to inject a file into a container when docker compose is executed from a remote machine, there is no alternative.

  18. polarathene commented on Mar 18, 2025

    @polarathene

    I don't know how many users rely on the bind mount behaviour but I find strange to use configs/secrets for this when bind mounts are already available.
    The only use case I can see is if a user uses a secret to share sensitive informations to the container but this is even worse because the user think there are extra security measures which is not the case.

    That happened for me when I initially came across it and thought from the docs (at least back then, haven't checked the current docs for it) that it was meant to be more secure, along with the uid/gid/mode features to ensure it is when providing a file.

    It's only because I noticed in the container none of those settings actually applied outside of swarm that I realized it was effectively a misleading bind mount with different syntax added as a fallback convenience for swarm users (actual justification for it IIRC).


    As for avoiding regression reports from existing users... I suppose until you're ready to make it a breaking change, one option is to go opt-in as suggested with something like secure: true as the opt-in attribute.

    Then clearly document that the secrets feature on non-swarm is not secure by default for legacy reasons, and requires additional explicit opt-in to toggle on the secure feature, with the caveat you lose the ability to sync updates from the host? 🤷‍♂

    Technically since uid/gid/mode aren't supported in the existing support and this is specific to behaviour on non-swarm usage...there's no reason those users should be configuring those given they don't do anything thus you might as well make any of those the opt-in trigger instead, nothing new to add then. Documentation just needs to be clear that those options will break the current functionality of being treated as a typical bind mount? 🤔 (EDIT: Oh I just realized that the referenced PR is doing exactly that for opt-in via uid/gid/mode usage)

  19. ndeloof commented on Mar 18, 2025

    @ndeloof
    Contributor

    I don't think directories should be supported because I think it doesn't match the original concept behind configs and secret

    A common pattern is to declare a whole config.d folder as a compose config and use it inside container. Many tools support this modularity pattern so that user don't hav to maintain a huge config file but can provide fragments

    I don't know how many users rely on the bind mount behaviour but I find strange to use configs/secrets for this when bind mounts are already available

    This was introduced to offer some parity with docker swarm, then users started relying on it in development to get immediate update on config changes (which could be implemented with watch nowadays). Then, with thousands example and stackoverflow suggestions you'll see tons of users to select this option without any specific reason. Still we have to manage our legacy to prevent compatibility breaks.

  20. schaubl commented on Mar 19, 2025

    @schaubl

    @ndeloof thanks for your answers.


    To manage the legacy, I think it could be possible to create an environment variable like USE_OLD_CONFIGS_SECRETS which will default to True/1 (similarly to the COMPOSE_BAKE introduced recently in v2.33.0).
    At the same time, a warning can be added in the release notes to mention that the default value of this variable will, in the future, be changed to False/0.
    With this, the current implementation will still be applied by default (for now) while allowing users to use the new implementation (by setting USE_OLD_CONFIGS_SECRETS to False) and for users relying on the current behaviour, they will have time to either migrate to bind mounts or to set USE_OLD_CONFIGS_SECRETS to True.
    @ndeloof What do you think about this strategy ?


    Regarding the PR #12625, if I understood properly, bind mounts will still be used for file except if one of the uid, gid or mode params are set.
    I have the impression that this could adds more confusion because the behaviour on handling the file will be completely different based on the usage or not of at least one of the 3 parameters mentioned above.
    This is especially true when the docker compose command is executed from a remote machine. In this case, for example, if I want to inject my /etc/hosts file into the container, it will work if I explicitly set its mode but if I don't, it will take the /etc/hosts of the machine where the docker daemon is.
    Also the behaviour can be different inside the same compose file, because the file param is in the top-level configs but mode is in the service-level configs so if I have a config used by two services where I need to change the permissions for only one service, the two different behaviour will be used and with my example with the /etc/hosts file, I will end up with two different file contents in the two containers even that they both use the same config.

    services:
      A:
        image: alpine
        configs:
          - source: etc_hosts
            target: /etc/hosts
        command: cat /etc/hosts
      B:
        image: alpine
        configs:
          - source: etc_hosts
            target: /etc/hosts
            mode: "0777"
        command: cat /etc/hosts
    configs:
      etc_hosts:
        file: /etc/hosts
  21. ndeloof commented on Mar 19, 2025

    @ndeloof
    Contributor

    @schaubl fully agree. #12625 is just an experiment to prevent breaking changes, still unpleasant - also comes with multiple drawbacks I can't detail here. As you noticed, secrets support in compose was designed with local engine in mind (or Docker Desktop) but won't support an actual remote engine. On the other hand, if we use copy to inject files inside container, we then miss a mechanism to enforce consistency when local config/secret is updated.

    it will take the /etc/hosts of the machine where the docker daemon is

    Please note this is exactly what some expect !

    The best option I have in mind is for docker engine to enable secrets/config support in standalone (non-swarm) mode. I can't tell about how complex this could be, but for sure the code is already there :P cc @colinhemmings
    @thaJeztah I haven't found an open issue requesting such a feature on moby repo, does this ring a bell ?

  22. GammaGames commented on Jun 12, 2025

    @GammaGames

    I found this thread after coming across this in the docs:

    Note that support for uid, gid, and mode attributes are not implemented in Docker Compose when the source of the secret is a file. This is because bind-mounts used under the hood don't allow uid remapping.

    The following example sets the name of the server-certificate secret file to server.cert within the container, sets the mode to 0440 (group-readable), and sets the user and group to 103. The value of server-certificate is set to the contents of the file ./server.cert.

    services:
      frontend:
        image: example/webapp
        secrets:
          - source: server-certificate
            target: server.cert
            uid: "103"
            gid: "103"
            mode: 0o440
    secrets:
      server-certificate:
        file: ./server.cert

    Am I reading this wrong, or does the documentation have and invalid example?

    Edit: Maybe because it's part of the spec and other programs support it

  23. ianhinder commented on Jun 12, 2025

    @ianhinder

    Yes, I think it's because it is part of the spec, but docker compose doesn't implement that part of the spec. And as far as I know, docker swarm mode doesn't implement ComposeV2 either, so I don't know that there is any implementation of this.

    The docker stack deploy command uses the legacy Compose file version 3 format, used by Compose V1. The latest format, defined by the Compose specification isn't compatible with the docker stack deploy command.

    ref

  24. vanyauhalin commented on Mar 21, 2026

    @vanyauhalin

    Please, add a note to this section that docker compose does not support this feature. Thank you.

    https://docs.docker.com/reference/compose-file/services/#configs

  25. stefanfed commented on May 24, 2026

    @stefanfed

    What is the correct, secure way to have the contents of a local file readable only by root passed as a secret to an unprivileged container?

  26. stalb commented on May 24, 2026

    @stalb

    What is the correct, secure way to have the contents of a local file readable only by root passed as a secret to an unprivileged container?

    Either change the uid/gid/mode of the file which contains the secret or use an inline secret (cf https://docs.docker.com/reference/compose-file/secrets/#example-2).

    When using inline secrets (as in this example: https://docs.docker.com/reference/compose-file/services/#long-syntax-5), specifying uid/gid/mode works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions