Repository navigation
snapshot rootfs uppers as one tar per container - #2110
Merged
Haven Xia (HavenXia) merged 4 commits intoOct 6, 2026
Merged
Haven Xia (HavenXia) merged 4 commits into
Haven Xia (HavenXia) merged 4 commits into
Conversation
Benjamin Elder (BenTheElder)
added
area/gvisor
area/microVM
breaking-change
The rootfs-upper tar used to hold the whole upper base, so the snapshot decided the <cid>/fs and <cid>/work layout that StageMergedRootfs then wiped and mounted by path. A crafted tar with symlinks there could point the overlay or the workdir wipe anywhere on the worker pod. Archive each container's upperdir on its own instead, and on restore create the layout in ateom before extracting into it. tarutil gains CreateWithRoot so the upperdir's own metadata, which overlayfs shows as the container's /, still round-trips. Old snapshots no longer restore.
The durable-dir tar held the whole mounts dir, so a crafted snapshot could replace a <volume> directory with a symlink. runsc bind-mounts that path into the sandbox, exposing whatever the link points at. Archive each volume's directory on its own and extract it into the directory atelet created, so the snapshot never decides the mount source. A volume with no tar, such as one added to the template since the snapshot, restores empty. Old snapshots no longer restore their durable data.
writeTree walked srcDir by path and opened each file by path after checking its type. An entry swapped for a symlink in between, such as a directory replaced by a link out of the tree, made the archive capture files from outside srcDir, which a restore would then hand to the workload. Walk, stat, readlink, open, and read xattrs through an os.Root instead, and refuse a file that is no longer regular when opened. The gVisor .gvisor.* cleanup after a durable restore uses the same handle.
Collaborator
Author
|
This is ready. |
Benjamin Elder (BenTheElder)
force-pushed
the
gvisor-durable-per-volume
branch
from
October 5, 2026 23:50
242421e to
a31d7a4
Compare
Wiz Scan Summary
To detect these findings earlier in the dev lifecycle, try the Wiz Code extension for VS Code, JetBrains, or Visual Studio. |
The upper base, each <cid> dir, and the overlay workdir were created 0755, but only root uses them. Create them 0700. The upperdir stays 0755 because overlayfs presents its mode as the container's /. Restore and cold boot now share one helper for this layout.
Haven Xia (HavenXia)
approved these changes
Oct 6, 2026
Haven Xia (HavenXia)
enabled auto-merge
October 6, 2026 00:06
Luomu (cloudonly)
pushed a commit
to cloudonly/substrate
that referenced
this pull request
Oct 7, 2026
…ent-substrate#2283) Archive and restore each durable-dir volume in its own durable-dir-<volume>.tar file instead of a single durable-dir.tar covering the parent directory, matching ateom-gvisor. Because kata-agent bind-mounts /run/kata-containers/shared/containers/durable/<volume> by path inside the guest VM on cold boot (including DATA-scope restores), extracting a single tar into the parent directory allowed a snapshot entry to replace <volume> with a symlink that kata-agent would resolve in the guest root filesystem. Creating each volume directory first and extracting its archive inside it keeps the volume layout out of the snapshot's control. > The PR title becomes the release note. Write it for users: what changed for them, not how the code changed. #### Breaking change See: agent-substrate#2110, applies the same change to microvm. Unlike gvisor this class was contained within the VM, but we'll do the same inside the VM for defense in depth.
This was referenced Oct 10, 2026
Timo Derstappen (teemow)
pushed a commit
to giantswarm/substrate
that referenced
this pull request
Oct 10, 2026
Snapshot tars no longer decide which host paths to mount or delete. Fixes agent-substrate#2083 We now take one rootfs upper per container in both gvisor and microvm runtimes. We take one tar per durable volume when durabledirs are used. We create the layout before extracting into it, so the snapshot has no control. #### Breaking change Old snapshots will not resume properly after this change. Because we are pre-1.0 and this is a security fix, we are moving forward with the straightforward fix. Fixes agent-substrate#2083 > It's a good idea to open an issue first for discussion. - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR (cherry picked from commit 893b1ed)
This was referenced Oct 10, 2026
Closed
1 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Snapshot tars no longer decide which host paths to mount or delete.
Fixes #2083
We now take one rootfs upper per container in both gvisor and microvm runtimes.
We take one tar per durable volume when durabledirs are used.
We create the layout before extracting into it, so the snapshot has no control.
Breaking change
Old snapshots will not resume properly after this change.
Because we are pre-1.0 and this is a security fix, we are moving forward with the straightforward fix.
Fixes #2083