You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit fd576d3
Browse filesBrowse the repository at this point in the historyBrowse files
ocispec: mount tmpfs at /dev/shm, and /tmp on both runtimes (#3533)
Part of #3226.
`gvisor` and `microvm` previously mounted different filesystems at
`/dev/shm`, and `/tmp`:
- `/dev/shm` on gVisor was a `0755` directory on `runsc`'s internal
`dev` filesystem, blocking non-root shared memory writes.
- `/tmp` was part of the host-backed rootfs upper layer on microVM and a
conditional `tmpfs` on gVisor when the image's `/tmp` was empty.
- Dropped the `/run` tmpfs from both runtimes, so `/run` comes from the
image's rootfs, as on Kubernetes.
Move the standard OCI system mounts (`/proc`, `/dev`, `/dev/pts`,
`/dev/shm`, `/dev/mqueue`, `/sys`) into `ocispec.Build` and add an
explicit `mode=1777` `tmpfs` mount at `/tmp`. If a container mounts a
volume at a system mount destination such as `/tmp`, `ocispec.Build`
replaces the default `tmpfs` entry there. `ShapeMicroVM` now rewrites
`bind` mounts in place.
AI use:
AI-assisted, I reviewed and tested it.
Breaking change:
- Old gVisor MEMORY snapshots, including goldens, don't restore. Recut
golden snapshots after upgrading.
- MicroVM MEMORY snapshots and goldens still resume, but keep the old
mounts until they are recut.
- MicroVM `/tmp`: now a tmpfs in guest memory instead of the host-backed
rootfs.
- Its files count against the actor's memory limit.
- It is capped at half of the guest's memory.
- ROOTFS snapshots no longer keep it, and files older ROOTFS snapshots
saved there are hidden.
- Files the image ships under `/tmp` are hidden.
- gVisor `/dev/shm`: now a 64 MiB tmpfs (`mode=1777`) instead of a
root-only directory. Non-root processes can now use it, but it is capped
at 64 MiB.
- gVisor `/tmp`: now always a tmpfs. runsc used to mount one only when
the image's `/tmp` was empty, so files the image ships under `/tmp` are
now hidden.
If an actor needs more space in `/dev/shm` or `/tmp`, or needs files
there kept in ROOTFS snapshots, mount a `durableDir` volume at that
path. It replaces the default tmpfs. Images that ship files under `/tmp`
should create them at startup instead.
- [x] Tests pass
- [x] Docs updated
---------
Signed-off-by: Nayana Bidari <[email protected]>
| Per-container limits (`containers[].resources.limits`) | Accepted in the API, ignored at runtime | Enforced via guest cgroups (`cpuMillicores` floored at 10) |
25
25
|`NET_RAW` capability | Stripped (`runsc` default `--net-raw=false`) | Granted when listed in `securityContext.capabilities.add`|
26
-
|`/dev/shm`|`0755` directory on the `runsc``dev` filesystem |`64 MiB``tmpfs` (`mode=1777`) |
27
-
|`/run`| Part of the container rootfs overlay |`64 MiB``tmpfs` (`mode=0755`) |
28
-
|`/tmp`|`tmpfs` when the image's `/tmp` is missing or empty, otherwise rootfs overlay | Part of the container rootfs overlay |
29
26
|`VOLUMES` and `ROOTFS` snapshot format |`runsc` filesystem checkpoint bundles (`fs/`, `rootfs/`) | Tar archives of the host directories |
30
27
31
28
## CPU Count and Throttling
@@ -62,10 +59,13 @@ They differ on `NET_RAW`:
62
59
63
60
See [Container Capabilities](api-guide.md#container-capabilities-securitycontextcapabilities) in the API Guide.
64
61
65
-
## System Mounts (`/dev/shm`, `/run`,`/tmp`)
62
+
## System Mounts (`/dev/shm` and`/tmp`)
66
63
67
-
-**gVisor (`gvisor`):**`runsc` mounts its internal `dev` filesystem at `/dev`, where `/dev/shm` is a `0755` directory, so only root can create files there. `/run` lives on the container's rootfs overlay. `runsc` mounts a `tmpfs` (`mode=01777`) at `/tmp` when the image's `/tmp` is missing or empty and no volume is mounted there. Files in `/dev/shm` and in that `/tmp``tmpfs` live outside the rootfs overlay, so only `MEMORY` snapshots keep them.
68
-
-**MicroVM (`microvm`):**`ateworker-microvm` mounts a `64 MiB``tmpfs` at `/dev/shm` (`mode=1777`, `nosuid`, `nodev`, `noexec`) and at `/run` (`mode=0755`, `nosuid`, `strictatime`). `/tmp` lives on the container's host-backed rootfs overlay, so `ROOTFS` snapshots keep its files. Files in `/dev/shm` and `/run` live in guest memory, so only `MEMORY` snapshots keep them.
64
+
Each container gets its own `tmpfs` at `/dev/shm` (`64 MiB`, mode `1777`, `noexec`) and `/tmp` (mode `1777`). `/tmp` holds at most half of `limits.memoryBytes` on `gvisor`, and half of usable guest RAM (`MemTotal`) on `microvm`. The `/tmp` mount hides any files the image has in `/tmp`. A volume at either path replaces the `tmpfs`. Files in these `tmpfs` mounts count against the actor's memory limit. They live outside the container's rootfs overlay, so `MEMORY` snapshots keep them and `ROOTFS` snapshots start with them empty.
65
+
66
+
Neither sandbox class mounts anything at `/run`. As with containerd, `/run` is part of the container's rootfs overlay, so the image's files there stay visible and `ROOTFS` snapshots keep them.
67
+
68
+
Volumes cannot mount at `/dev`, or at or under `/dev/pts`, `/proc`, or `/sys`.
0 commit comments