Skip to content

fix: Dockerfile - VOLUME directive is an anti-pattern#2948

Closed
polarathene wants to merge 2 commits intokanidm:masterfrom
polarathene:patch-1
Closed

fix: Dockerfile - VOLUME directive is an anti-pattern#2948
polarathene wants to merge 2 commits intokanidm:masterfrom
polarathene:patch-1

Conversation

@polarathene
Copy link
Copy Markdown

@polarathene polarathene commented Aug 2, 2024

The VOLUME directive provides no value to an image. It only causes problems.

For detailed justification, please see below context or my past writeup on this subject: ory/hydra#3683

Checklist

  • This PR contains no AI generated code
  • cargo fmt has been run
  • cargo clippy has been run
  • cargo test has been run and passes
  • book chapter included (if relevant)
  • design document included (if relevant)

Context

Reproduction Example

services:
  kanidm:
    image: docker.io/kanidm/server:latest
    ports:
      - 443:8443
    # This feature is like `volumes`,
    # except you can define the file content inline with the `compose.yaml` file.
    # I use this for simple copy/paste examples I share online.
    configs:
      - source: kanidm-config
        target: /data/server.toml
      # I've provisioned certs via Smallstep CA (content excluded due to added verbosity):
      - source: tls-leaf-cert-ecdsa-localhost
        target: /data/chain.pem
      - source: tls-leaf-key-ecdsa-localhost
        target: /data/key.pem

# `VOLUME /data` paired with `db_path = "/data/kanidm.db"` was problematic
# when I changed the `domain` + `origin` and certs.
configs:
  kanidm-config:
    content: |
      #domain = "auth.example.test"
      #origin = "https://auth.example.test:8443"
      domain = "auth.localhost"
      origin = "https://auth.localhost:8443"
      tls_chain = "/data/chain.pem"
      tls_key = "/data/key.pem"
      db_path = "/data/kanidm.db"
      bindaddress = "[::]:8443"

After running the above compose.yaml with the config update to domain + origin, the container fails unexpectedly:

$ docker compose up --force-recreate

[+] Running 1/1
 ✔ Container kanidm-1  Recreated
Attaching to kanidm-1
📜 Using config file: "/data/server.toml"
Log filter: Info
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: This is running as uid == 0 (root) which may be a security risk.
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: permissions on /data/server.toml may not be secure. Should be readonly to running uid. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: /data/server.toml has 'everyone' permission bits in the mode. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: /data/server.toml owned by the current uid, which may allow file permission changes. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: DB folder /data has 'everyone' permission bits in the mode. This could be a security risk ...
00000000-0000-0000-0000-000000000000 INFO     i [info]: Running in server mode ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: permissions on /data/chain.pem may not be secure. Should be readonly to running uid. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: permissions on /data/key.pem may not be secure. Should be readonly to running uid. This could be a security risk ...
00000000-0000-0000-0000-000000000000 WARN     🚧 [warn]: WARNING: /data/key.pem has 'everyone' permission bits in the mode. This could be a security risk ...
00000000-0000-0000-0000-000000000000 INFO     i [info]: Starting kanidm with configuration: address: [::]:8443, domain: auth.localhost, ldap address: disabled, origin: https://auth.localhost:8443 admin bind path: /data/kanidmd.sock, thread count: 16, dbpath: /data/kanidm.db, arcsize: AUTO, max request size: 262144b, trust X-Forwarded-For: false, with TLS: true, online_backup: disabled, integration mode: false, console output format: Text log_level: inforole: write replica, replication: disabled, otel_grpc_url: None
3773f806-c462-49d6-912f-fd5b6eae3710 INFO     system_initialisation [ 34.7ms | 28.69% / 100.00% ]
3773f806-c462-49d6-912f-fd5b6eae3710 INFO     ┝━ initialise_schema_core [ 18.4ms | 53.01% ]
3773f806-c462-49d6-912f-fd5b6eae3710 INFO     ┝━ initialise_domain_info [ 2.17ms | 6.27% ]
3773f806-c462-49d6-912f-fd5b6eae3710 WARN     ┝━ 🚧 [warn]: Using domain name from the database auth.example.com - was auth.localhost in memory | event_tag_id: 2
3773f806-c462-49d6-912f-fd5b6eae3710 WARN     ┝━ 🚧 [warn]: If you think this is an error, see https://kanidm.github.io/kanidm/stable/administrivia.html#rename-the-domain | event_tag_id: 2
3773f806-c462-49d6-912f-fd5b6eae3710 INFO     ┕━ qswt_commit [ 4.17ms | 12.03% ]
00000000-0000-0000-0000-000000000000 ERROR    🚨 [error]: Effective domain is not a descendent of server domain name (rp_id). | event_tag_id: 1
00000000-0000-0000-0000-000000000000 ERROR    🚨 [error]: You must change origin or domain name to be consistent. ed: "https://auth.localhost:8443" - rp_id: "auth.example.com" | event_tag_id: 1
00000000-0000-0000-0000-000000000000 ERROR    🚨 [error]: To change the origin or domain name see: https://kanidm.github.io/kanidm/server_configuration.html | event_tag_id: 1
00000000-0000-0000-0000-000000000000 ERROR    🚨 [error]: Unable to setup query server or idm server -> InvalidState
00000000-0000-0000-0000-000000000000 ERROR    🚨 [error]: Failed to start server core!
Logging pipeline completed shutdown
exited with code 1

Specifically the problem is identified at this log line:

🚧 [warn]: Using domain name from the database auth.example.com - was auth.localhost in memory | event_tag_id: 2

Problem triggered in relation to VOLUME directive

I am aware of the requirements for changing the configuration, but this failure was not expected as I was using docker compose up --force-recreate.

Typically that should be equivalent to docker run --rm ..., ensuring a fresh container instance is created, discarding any internal state. Nor did docker compose down help.

  • I provided no volume mount for /data since I was only in an evaluation stage and wanting to quickly get the bare minimum up and running.
  • Any additional runtime data that kanidm may write to /data in the container was expected to have been discarded for a clean slate.
  • Anonymous volumes are created when an image has a VOLUME directive. They will initialize by copying any of the existing content at the mount point, unlike a bind mount volume which replaces.
    • With docker run --rm ..., each container instance started will create a new anonymous volume. Without the --rm, these will accumulate pointlessly, the user may not be aware of it.
    • Docker Compose behaves differently, with additional logic to preserve the same anonymous volume across instances of the container for a given compose project.

Thus I had to remove the actual anonymous volume manually to get a fresh container like the first time I started kanidm, which would then not attempt to re-use the unwanted prior database in /data.

Conclusion

There is no reason to use VOLUME in Dockerfile. It only causes issues.

Your database in /data will persist with the internal container state for the lifecycle of that container. If a user wants to persist it externally, then they can do so with an explicit volume (be that anonymous/named/bind/etc).

Please avoid this anti-pattern, thanks!

This directive provides no value to an image. It only causes problems.
@polarathene
Copy link
Copy Markdown
Author

Side-note, the link that follows the mentioned warning is invalid. You should probably update that 👍

Copy link
Copy Markdown
Member

@yaleman yaleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you noted it and while you're there, could you please update the broken link to https://kanidm.github.io/kanidm/master/server_configuration.html 😄

RUN chmod +x /sbin/kanidmd

EXPOSE 8443 3636
VOLUME /data
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting it will break the container's default config as the folder doesn't exist, please change it to :

RUN mkdir /data

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please update the broken link

Link to source file and line and sure 👍

as the folder doesn't exist

Would it make sense to default to the location instead of / when running the ENTRYPOINT?

EDIT: Oh you only have CMD set (/sbin/kanidmd server -c /data/server.toml), that is still absolute path so a WORKDIR directive would create the location (not sure if that avoids an extra layer or not, I think it might).

Alternatively, I could add && mkdir /data in an earlier RUN?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The WORKDIR doesn't apply to the final container, note the FROM repos on line #65.

Alternatively, I could add && mkdir /data in an earlier RUN?

as long as it's after line #65

could you please update the broken link
Link to source file and line and sure 👍

https://github.com/search?q=repo%3Akanidm%2Fkanidm+%2FTo+change+the+origin+or+domain+name+see%2F+language%3ARust&type=code

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The WORKDIR doesn't apply to the final container, note the FROM repos on line 65.

Sorry I don't follow? That is the last FROM directive in this file, and given the remainder of the file it looks like the final stage? Are you doing something else to the image?

Separate Dockerfile:

FROM alpine
WORKDIR /data

compose.yaml that extends that image with a custom Dockerfile:

services:
  example:
    # The `image` setting now represents the tag for the local build configured below:
    image: local/workdir-extended:latest
    # Local build (no need to try pull `image` remotely):
    pull_policy: build
    # The original image was extended, the directory from WORKDIR still exists.
    # NOTE: If this were `FROM scratch`, obviously you'd need to COPY `/data` over
    #       but that's no different to how you approach creating `/data` prior.
    build:
      dockerfile_inline: |
        FROM local/workdir:latest
        RUN touch hello.txt
        CMD pwd && ls
# Build the separate image:
$ docker build --tag local/workdir .

# Build and run the variant image that extends it:
$ docker compose run --rm example
/data
hello.txt

# Also verify image metadata still has the workdir set:
$ docker image inspect local/workdir-extended:latest | jq -r '.[].Config.WorkingDir'
/data

Seems to work perfectly fine no?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please update the broken link
Link to source file and line and sure 👍

https://github.com/search?q=repo%3Akanidm%2Fkanidm+%2FTo+change+the+origin+or+domain+name+see%2F+language%3ARust&type=code

That is not the line you wanted changed? 🤷‍♂️

I've used that to identify the intended line and correct the link with the one you've provided, so thanks 👍

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought you were talking about the existing workdir command, not adding one.

Copy link
Copy Markdown
Author

@polarathene polarathene Aug 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the existing workdir command, not adding one.

That's in the builder stage and wouldn't make sense to replace an existing one with a different path.

  • If it makes sense to run the container from /data I can add a WORKDIR directive. I assumed /data was arbitrary for documentation, and not hard-coded/expected anywhere.
  • If that assumption is correct, you could just use an existing directory for documentation and anyone who wants to use a custom location like /data can do so via config + volume?

How would you like to proceed?

  • WORKDIR or RUN?
  • /data or an existing location like /tmp?

EDIT: Or sure... ignore responding and do the work again via a separate PR without any appreciation beyond a link ref to this PR to automate closing it. Way to encourage contributions 🙄

@yaleman yaleman mentioned this pull request Aug 4, 2024
6 tasks
@yaleman yaleman closed this in #2954 Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

2 participants