Repository navigation
Description
Would be nice to have caddy run as a non-root user inside the container. This is especially useful for rootfull containers (e.g. Docker), where the uid=0 inside the container is the same as uid=0 outside. Additionally, since the /data dir and the /config have been specified as a VOLUME, others are unable to extend the container and change the ownership of those directories. I'm currently using the below Dockerfile to run caddy as a non-root user. I've changed it slightly to suit my needs but much of it should be re-usable for this repo.
FROM caddy:2-builder AS builder
RUN caddy-builder github.com/caddy-dns/cloudflare
FROM caddy:2-alpine AS deps
# We cannot use FROM scratch because, despite adding cap_net_bind_service to the binary
# it still won't run. Presuming because libcap isn't available? Not sure.
FROM alpine:3.12
COPY --from=deps /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=deps /etc/mime.types /etc/nsswitch.conf /etc/
COPY --from=builder /usr/bin/caddy /caddy
RUN set -eux; \
apk add --no-cache libcap; \
setcap cap_net_bind_service=ep /caddy; \
mkdir -p /config/caddy /data/caddy; \
addgroup -g 101 -S www-data; \
adduser -u 101 -D -S -G www-data www-data; \
chown -R www-data:www-data /config /data
USER www-data
ENV XDG_CONFIG_HOME=/config XDG_DATA_HOME=/data
VOLUME /config /data
EXPOSE 80
EXPOSE 443
ENTRYPOINT ["/caddy"]
CMD ["run", "--config", "/Caddyfile", "--adapter", "caddyfile"]
One thing worth considering is that this might be not be an easy upgrade for many, indeed, it may be that we'd need a temporary stop gap that runs the container as root, changes ownership of files/folders, then drops privileges. Then after perhaps a couple of versions, this stop gap can be replaced fully with a non-root user without going through the trouble of dropping privileges.
Activity
hairyhenderson commentedon Aug 17, 2020
Hi @hoshsadiq - my apologies for replying so late (been vacationing and otherwise distracted)!
We actually originally did run as non-root by default, but simplicity we decided to drop that (see #24, and also #103 for some other related discussion).
If your
Dockerfileworks for you, that's great. In most cases where users want to run as non-root, they also don't need to listen to:80/:443in the container, so thesetpcapmagic isn't necessary at all.It's also worth noting that
caddyis an official image, and as such needs to be similarly-shaped to other official images of the same type. At a quick glance, none ofnginx,traefik,httpd, orhaproxysupport running as non-root out of the box either.Finally, it's worth considering why you want to run as non-root. What attack vectors are you trying to avoid? Container escape vulnerabilities are pretty much the only real risk, but anyone running a modern Docker version is immune to many of them. It's also worth considering user namespace remapping as a mitigation. In my experience the main reason for running as non-root is to pass compliance checks - not a bad reason, but it's also worth recognizing that non-compliance does not automatically equal decreased security (and vice-versa).
I will leave this open to discuss possibilities, but I think the starting point is probably to just provide some better documentation around running as non-root.
mirekum commentedon Sep 10, 2020
Hi, In the company I work for, I am not allowed to run any docker container as root. It would be nice to use the Caddy in the same way:
es:
container_name: es
image: elasticsearch:6.8.8
read_only: true
user: "elasticsearch:elasticsearch"
hoshsadiq commentedon Sep 10, 2020
@hairyhenderson sorry, I had forgotten about this issue.
I'm aware of this. I didn't see an option to run caddy on a non-standard port, hence why I used setpcap. Maybe I was blind and/or I misunderstood. If you could let me know what that option is I'll gladly update my image.
While this is true, it's worth noting that
nginxandhttpdboth have the option to delegate the actual workers (I don't know if it's the same withtraefikandhaproxy). One can indeed run as root, though it is discouraged in multiple places. In addition, fornginx, the configuration ignores the user/group options if the main process runs a non-root user.Caddy does not have an option to delegate workers (I'm presuming requests run in go routines so that wouldn't even make sense). It is also impossible to run it as a non-root user using docker's (and docker-compose's) user arguments because the volumes
/dataand/configare owned by root, so when caddy runs as non-root, it does not have the correct permissions to those directories to function correctly.Definitely agree with this. I'm using caddy for a personal project, but I think running as non-root is a no-brainer for me considering it's such a trivial task that gives a lot with little effort. Though this is a personal preference, and isn't entirely based on any specific threat model, rather it's more of a precautionary issue.
I think even if Caddy doesn't change the default, that would totally be okay, but making it easier for individual to run caddy in non-root mode, in my opinion is a must. As it stands, that is as simple as
chowning the/dataand/configdirectory as a non-root user. This was what initially prompted me to raise this as an issue. Due to docker's design, it is not possible to change the ownership (or the mode) of aVOLUMEin a running container. It is only possible by explicitly doing so in anotherDockerfile, changing the ownership/mode, and then re-assigning it as aVOLUME(as I've done in the my image).hairyhenderson commentedon Sep 12, 2020
@hoshsadiq thanks for following up!
That's really just a matter of configuration. In the
Caddyfilesyntax, there are two global optionshttp_portandhttps_portto tell Caddy to listen to nonstandard ports. You'll still need to be listening at 80/443 outside the container in order to get automatic TLS from Let's Encrypt though.So a config like this should work for you (warning, I didn't have time to test this):
Then when you run the container, just map the ports appropriately:
$ docker run -d -p 80:8080 -p 443:8443 ...Sure, but that's if you want to use
/dataand/config- you don't have to. You can set theXDG_DATA_HOMEandXDG_CONFIG_HOMEenvironment variables to different locations. (See https://caddyserver.com/docs/conventions#file-locations for details on what these are)The reason I used
VOLUMEs is to prevent these files from being written to the container, and hence lost during a container restart. It's important for the the data directory in particular to persist, because it contains TLS certs from Let's Encrypt, and if those are lost you can very quickly get rate-limited by LE.So, if you choose to configure Caddy to use different data/config directories (which you'll have to if you're running as non-root), you'll need to make sure you use persisted volumes.
Hope this helps! I'm definitely going to add some of this to the docs for the image on DockerHub, since most of this is definitely non-intuitive for people new to Caddy v2 😅
westurner commentedon Sep 13, 2020
The process should not have write access to the config.
More about not running as root here: #24 (comment)
francislavoie commentedon Sep 13, 2020
/configis actually a config cache directory, for when users are making use of Caddy's config API. If Caddy didn't write its current config to file, then the running config would be lost. In other words, any changes pushed to the config API would be lost. It also allows for the--resumeoption to work, which will make Caddy start up with the previously run configuration instead on start.https://caddyserver.com/docs/conventions#configuration-directory
Don't worry, Caddy never writes to your Caddyfile (unless you use the
caddy fmtcommand with the overwrite option, to have it re-indent your config).mholt commentedon Sep 13, 2020
To clarify, the running config is stored in memory, so it would only be lost lost if the process or container was terminated. (Which, of course, happens all the time.) You're right, but I just wanted to make that distinction.
hoshsadiq commentedon Dec 20, 2020
Sorry for the late reply - again.
Aah, that is fair. I did not think of this, I'll use that. Also thanks for the config options for running on non-standard port, I don't know why, but I just did not find the options prior to you mentioning it.
I'm more than happy for this issue to be closed as my issue is resolved, however, I think it's still worth changing the default container to being back run as a non-root user. Considering root inside a container is root outside a container (for docker anyway), this can be destructive if Caddy is compromised. Security should not be compromised on such a level for a little convenience. This should be okay once rootless becomes the norm, but currently the majority users, and production workloads are still running Docker, where all containers are root-full.
Anyway, if there is no wiggle room, feel free to close this issue. I suppose if that's the case, documentation is key.
Install libcap and run `setcap cap_net_bind_service=+eip` on caddy bi…
abjugard commentedon Jan 17, 2023
Would it be an acceptable compromise for the official image to additionally install
libcapand runsetcap cap_net_bind_service=+eip /usr/bin/caddyso that users that want to use a more secure configuration can do so without having to build their own image? As it stands, the image isn't compatible with K8s securityContext runAsNonRoot + runAsUser options, because if I do that then it won't be able to bind port 80, which I personally think it should be able to do, in order to be compatible with e.g. host networking.I've provided a pull-request which mitigates this issue, by making it possible to at least use the image with a different PID, given the user configures it in a compatible manner.
francislavoie commentedon Jan 17, 2023
Ah, yeah good idea @abjugard, that seems like a good approach.
Install libcap and run `setcap cap_net_bind_service=+eip` on caddy bi…
Install libcap and run `setcap cap_net_bind_service=+eip` on caddy bi…
Install `libcap` in builder image to make it easier for users buildin…
Install libcap and run `setcap cap_net_bind_service=+eip` on caddy bi…
1 remaining item
Install libcap and run `setcap cap_net_bind_service=+eip` on caddy bi…
Install `libcap` in builder image to make it easier for users buildin…
Install libcap and run `setcap cap_net_bind_service=+ep` on caddy binary
Install `libcap` in builder image to make it easier for users buildin…
Install libcap and run `setcap cap_net_bind_service=+ep` on caddy binary
Install `libcap` in builder image to make it easier for users buildin…
Install libcap and run `setcap cap_net_bind_service=+ep` on caddy binary
Install `libcap` in builder image to make it easier for users buildin…
Install libcap and run `setcap cap_net_bind_service=+ep` on caddy binary
Install `libcap` in builder image to make it easier for users buildin…
hairyhenderson commentedon Feb 8, 2023
Fixed in #274 - this'll be available in the next Caddy release
julsemaan commentedon Dec 18, 2023
Sorry for reviving a closed issue here but it seems there are capabilities to run this as non-root but although docs were called out in the issue description, I can't seem to find any docs related to how we can run this image as non-root
Could you point me at the right location for the docs and I think it will help to have them linked here in the OG issue
Appreciate the help!
jjlin commentedon Dec 18, 2023
@julsemaan In general, to run a container as non-root, you just pass
-u/--usertodocker run; see https://docs.docker.com/engine/reference/commandline/run/ for details.Some images (e.g., LinuxServer.io images) use environment variables like
PUID/PGIDto switch to another user/group during container startup, but there aren't any official standards behind that kind of behavior.polarathene commentedon Sep 7, 2024
FWIW, once containerd 2.0 is out k8s and others that leverage that dependency are likely to adopt the same default that Docker has had for a while where the need for the setcap will be unnecessary: kubernetes/kubernetes#102612 (comment)
That said, I do not like the
setcapapproach as it enforces the capability even if you explicitly do not want it. I go into more detail about that issue here: #299 (comment)Just to clear up these two statements. Root in the container is not equivalent to root outside of the container. Yes they both share the same UID/GID, but the root user in the container is run in a namespace, there is isolation going on there, reduced capabilities is a good example of that.
Regarding the statement about compromising security for convenience, that's technically what you get with
setcapapproach vs having the application request the capability at runtime if it's available.As the linked comment of mine notes, the
setcapapproach was being planned at one point to grant much more concerning capability to the binary which would not have been possible to opt-out of as asetcapworkaround instead of at runtime.I ended up here looking for why/when the
setcapgot introduced. Only after I found this issue did I realize it wouldn't have been as tricky to figure out if I had noticed the mainDockerfile.tmplfile 😑For reference it was part of the 2.6.3 release onwards (would be nicer if the repo actually used git tags for releases since the commit would also be associated to the eventual tagged release).