Repository navigation
Allowlist capabilities before 1.0 #3579
Description
Substrate's default container capability set is strictly AUDIT_WRITE, KILL, NET_BIND_SERVICE in cmd/atelet/oci.go and docs/reference/api-guide.md.
That's good, but it doesn't make sense to allow workloads to add super privileged capabilities using capabilities.add. We should allowlist capabilities now, and provide configurable means to relax them post 1.0 if there is demand.
Proposed Allowlist (13 Capabilities)
The following capabilities are safe to allow in capabilities.add because their effects are confined to the container's local namespaces and filesystems:
| Capability | Rationale |
|---|---|
AUDIT_WRITE |
Standard audit record generation (already in Substrate default set). |
CHOWN |
File ownership modification within container filesystems; required for container root entrypoint setup. |
DAC_OVERRIDE |
Bypass file permission checks on container files. |
DAC_READ_SEARCH |
Bypass directory search and file read checks. Documented for read-only access to files owned by other image users. |
FOWNER |
Bypass permission checks on file owner operations (extended attributes, timestamps). |
FSETID |
Retain setuid/setgid bits upon file modification. |
KILL |
Process signaling within the container PID namespace (already in Substrate default set). |
NET_BIND_SERVICE |
Bind to system ports below 1024 (already in Substrate default set). |
SETFCAP |
Set file capabilities on files within the container filesystem. |
SETGID |
Process GID and supplementary group manipulation; required for su/gosu/setpriv user switching. |
SETPCAP |
Capability bounding set manipulation within existing process capabilities. |
SETUID |
Process UID manipulation; required for user-switching container entrypoints. |
SYS_CHROOT |
chroot() within the container mount namespace; required by build tooling and SSH separation. |
Denied Capabilities & Security Rationale
All capabilities not explicitly listed above are rejected at admission (ValidateCapabilities). Specifically:
MKNOD: Denied. Eliminates raw device node creation, blocking direct device attacks against persistent memory rootfs.SYS_ADMIN: Denied. Prevents mount namespace escapes, arbitrary filesystem mounting, device node manipulation, and cgroup reconfiguration.SYS_RAWIO,SYS_MODULE,SYS_BOOT,SYS_TIME: Denied. Prevents raw I/O port operations, kernel module injection, guest reboots, and system clock tampering.SYS_PTRACE: Denied. Prevents cross-process memory inspection and syscall tracing outside the container's process tree.NET_ADMIN: Denied. Blocks manipulation of guest network interfaces, routing tables, and firewall rules.NET_RAW: Denied. Prevents raw socket creation. Standard diagnostic tools (such asping) are supported via the unprivileged ICMP sysctl (net.ipv4.ping_group_range = "0 2147483647"), matching gVisor parity.BPF,PERFMON,SYSLOG: Denied. Prevents eBPF program injection and hardware performance/kernel log inspection.
Accompanying Hardening
- Guest Mounts: Apply
nodevandnosuidto all volume bind mounts ininternal/ocispec/microvm.go(ShapeMicroVM). - Ping Sysctl: Configure
net.ipv4.ping_group_range = "0 2147483647"inShapeMicroVMsopingworks withoutNET_RAW. - Runtime Filter: Strip unallowed capabilities during
resolveCapabilitiesincmd/atelet/oci.goas defense-in-depth. - Documentation: Update
docs/reference/api-guide.mdand mark this change as breaking in the release notes.
Thanks to Fabiano Fidêncio (@fidencio) for the report on MKNOD that led us to make this change.
Reacted by Benjamin Elder and Fabiano Fidêncio
Activity
destijl commentedon Nov 3, 2026
Update from implementing this:
capabilities.addis now an allowlist:AUDIT_WRITE,CHOWN,DAC_OVERRIDE,DAC_READ_SEARCH,FOWNER,FSETID,KILL,NET_BIND_SERVICE,SETFCAP,SETGID,SETPCAP,SETUID,SYS_CHROOT. ateapi and atelet reject anything else.nosuid,nodev, and setnet.ipv4.ping_group_rangesopingworks withoutNET_RAW.I also tried
nosuid,nodevon gVisor volume mounts and left it out. On restore,ateworker-gvisorrebuilds the OCI spec from current code, and runsc requires the mounts to match the checkpointed spec:On GKE, after an upgrade with the change, actors suspended before the upgrade and new actors from older templates (whose golden snapshots are older) all went to
ACTOR_STATE_CRASHED. MicroVM doesn't have this problem, because a memory restore resumes the VM without rebuilding the guest spec.Question: do we care about
nosuid,nodevon gVisor? Fabricio Voznika (@fvoznika)nodev:MKNODcan no longer be added. It's also unclear that gVisor enforcesnodevon gofer mounts: with it in the spec, the sandbox's/proc/self/mountinfoshowed onlyrw,nosuidfor the volume.nosuid: runsc runs with--allow-suid, so a setuid binary on a volume can raise privilege inside the sandbox. That mostly matters for shared CSI volumes others can write to. The effect stays inside the sandbox, and the allowlist bounds it.If we don't care, nothing more is needed. If we do, or once some other mount change needs it, we'd want restore to work across mount changes first.
dims commentedon Nov 3, 2026
Fabiano Fidêncio (@fidencio) thanks for the pointers that uncovered this issue!
fvoznika commentedon Nov 3, 2026
Greg Castle (@destijl) gVisor should also have the same
nosuid,nodevsettings as a defense in depth mechanism. I'll take a look at thenodevissue you mentioned, looking at the code it should be working. Can you make the changes?fvoznika commentedon Nov 3, 2026
nodevdoesn't show inside the sandbox, because it's not yet implemented in the Sentry. But the host bind mount does get the option.Restrict actor capabilities.add and make mounts nodev (agent-substrat…
Restrict actor capabilities.add and make mounts nodev (#3580)
Restrict actor capabilities.add and make mounts nodev (agent-substrat…