Skip to content

Commit 8a14b3a

Browse files
fix(sandbox): skip read-only mounts during recursive chown of /sandbox (NVIDIA#2341)
* fix(sandbox): skip read-only mounts during recursive chown of /sandbox The sandbox supervisor crashed with EROFS when the recursive chown of /sandbox encountered read-only submounts. This is common in gVisor-based Kubernetes deployments where read-only volume mounts are the only way to enforce per-directory immutability (Landlock is unavailable under gVisor). The fix adds two guards to the ownership walk: 1. Mount-boundary detection via st_dev comparison — paths on a different filesystem than /sandbox are skipped entirely, avoiding the chown call on nested read-only mounts. 2. EROFS tolerance — if chown still returns EROFS (e.g. the root mount itself is read-only), the error is logged at debug level and startup continues. Symlink skipping (already present) is preserved and extracted into the recursive walker for consistency. Manually verified on a kind cluster with a read-only PVC mounted at /sandbox/readonly-data: the pod starts successfully, writable paths are owned by the sandbox user, and the read-only mount retains root ownership. Kubernetes e2e test coverage is a separate follow-up. Closes NVIDIA#2294 Signed-off-by: Varsha Prasad Narsing <[email protected]> * refactor(sandbox): consolidate symlink check into chown_recursive Move the per-child symlink guard from the directory iteration loop into the top of chown_recursive, reusing the symlink_metadata call that is already performed there. This centralizes all three skip guards (symlink, mount boundary, EROFS) in one place and eliminates a redundant symlink_metadata call per child entry. Suggested-by: elezar Signed-off-by: Varsha Prasad Narsing <[email protected]> * fix(sandbox): recurse into children after EROFS and fix doc comment When chown returns EROFS on a directory, the code previously returned early without recursing into children. This skipped writable submounts nested under a read-only directory on the same device (e.g., a writable emptyDir inside a read-only ConfigMap mount sharing the same st_dev). Now the EROFS case falls through to the directory recursion so that writable children are still chowned. Also fixes the doc comment which said EROFS errors are "logged as warnings" when they are actually logged at debug level. Signed-off-by: Varsha Prasad Narsing <[email protected]> * fix(sandbox): allow similar_names clippy lint in chown test Signed-off-by: Varsha Prasad Narsing <[email protected]> * fix(sandbox): remove cross-device skip and inject chown for EROFS tests Remove the unconditional st_dev boundary check that prevented chown on writable PVC mounts with a different device ID. Rely solely on EROFS errors to skip read-only paths while continuing traversal into children and siblings. Inject the chown operation into chown_recursive to enable unit testing the EROFS recovery path without requiring root or mount privileges. Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> * style(sandbox): apply rustfmt to chown test closures Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> * fix(sandbox): stop descending into EROFS subtrees to prevent TOCTOU race After chown returns EROFS, return immediately instead of recursing into the read-only directory's children. Descending into a mount we cannot own creates a TOCTOU window where an attacker could swap a child for a symlink between symlink_metadata and chown (CWE-367/CWE-59). Siblings of the EROFS path are still visited by the caller's loop. Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> * fix(sandbox): hard-error on root EROFS, log read_dir failures, eliminate double stat Chown root directly in chown_sandbox_home so EROFS on /sandbox itself is a hard startup error rather than being silently swallowed. Extract chown_children to log read_dir failures instead of silently skipping them. This also eliminates the redundant symlink_metadata syscall on the root path. Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> --------- Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]>
1 parent 0674a00 commit 8a14b3a

1 file changed

Lines changed: 143 additions & 16 deletions

File tree

‎crates/openshell-supervisor-process/src/process.rs‎

Lines changed: 143 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1168,10 +1168,14 @@ fn rewrite_group_at(path: &Path, gid: &str) -> Result<()> {
11681168
/// Symlinks are skipped (not followed) to prevent privilege escalation via
11691169
/// malicious container images. The TOCTOU window is not exploitable because
11701170
/// no untrusted process is running yet.
1171+
///
1172+
/// The root path is chowned unconditionally — EROFS there is a hard error
1173+
/// (a read-only `/sandbox` is a misconfiguration). For children, `EROFS`
1174+
/// causes the walker to skip that path and its entire subtree — descending
1175+
/// into a read-only mount we do not control would be a TOCTOU risk
1176+
/// (CWE-367/CWE-59). Siblings of the read-only path are still visited.
11711177
#[cfg(unix)]
11721178
fn chown_sandbox_home(root: &Path, uid: Option<Uid>, gid: Option<Gid>) -> Result<()> {
1173-
use nix::unistd::chown;
1174-
11751179
let meta = std::fs::symlink_metadata(root).into_diagnostic()?;
11761180
if meta.file_type().is_symlink() {
11771181
return Err(miette::miette!(
@@ -1180,23 +1184,63 @@ fn chown_sandbox_home(root: &Path, uid: Option<Uid>, gid: Option<Gid>) -> Result
11801184
));
11811185
}
11821186

1183-
chown(root, uid, gid).into_diagnostic()?;
1187+
nix::unistd::chown(root, uid, gid).into_diagnostic()?;
11841188

1185-
if meta.is_dir()
1186-
&& let Ok(entries) = std::fs::read_dir(root)
1187-
{
1188-
for entry in entries {
1189-
let entry = entry.into_diagnostic()?;
1190-
let path = entry.path();
1191-
if path
1192-
.symlink_metadata()
1193-
.is_ok_and(|m| m.file_type().is_symlink())
1194-
{
1195-
debug!(path = %path.display(), "Skipping symlink during sandbox home chown");
1196-
continue;
1189+
if meta.is_dir() {
1190+
chown_children(root, uid, gid, &nix::unistd::chown)?;
1191+
}
1192+
1193+
Ok(())
1194+
}
1195+
1196+
/// Walk directory children and chown each entry, skipping symlinks and
1197+
/// EROFS subtrees. Called after the parent has already been chowned.
1198+
#[cfg(unix)]
1199+
fn chown_children(
1200+
dir: &Path,
1201+
uid: Option<Uid>,
1202+
gid: Option<Gid>,
1203+
do_chown: &impl Fn(&Path, Option<Uid>, Option<Gid>) -> nix::Result<()>,
1204+
) -> Result<()> {
1205+
match std::fs::read_dir(dir) {
1206+
Ok(entries) => {
1207+
for entry in entries {
1208+
let entry = entry.into_diagnostic()?;
1209+
let child = entry.path();
1210+
chown_recursive(&child, uid, gid, do_chown)?;
11971211
}
1198-
chown_sandbox_home(&path, uid, gid)?;
11991212
}
1213+
Err(e) => {
1214+
debug!(path = %dir.display(), error = %e, "Cannot list directory during sandbox home chown");
1215+
}
1216+
}
1217+
Ok(())
1218+
}
1219+
1220+
#[cfg(unix)]
1221+
fn chown_recursive(
1222+
path: &Path,
1223+
uid: Option<Uid>,
1224+
gid: Option<Gid>,
1225+
do_chown: &impl Fn(&Path, Option<Uid>, Option<Gid>) -> nix::Result<()>,
1226+
) -> Result<()> {
1227+
let meta = std::fs::symlink_metadata(path).into_diagnostic()?;
1228+
1229+
if meta.file_type().is_symlink() {
1230+
debug!(path = %path.display(), "Skipping symlink during sandbox home chown");
1231+
return Ok(());
1232+
}
1233+
1234+
if let Err(e) = do_chown(path, uid, gid) {
1235+
if e == nix::errno::Errno::EROFS {
1236+
debug!(path = %path.display(), "Skipping read-only path during sandbox home chown");
1237+
return Ok(());
1238+
}
1239+
return Err(e).into_diagnostic();
1240+
}
1241+
1242+
if meta.is_dir() {
1243+
chown_children(path, uid, gid, do_chown)?;
12001244
}
12011245

12021246
Ok(())
@@ -2115,6 +2159,89 @@ mod tests {
21152159
.expect("should skip symlink children without error");
21162160
}
21172161

2162+
#[cfg(unix)]
2163+
#[test]
2164+
fn chown_recursive_skips_erofs_subtree_but_continues_siblings() {
2165+
use std::sync::{Arc, Mutex};
2166+
2167+
let dir = tempfile::tempdir().unwrap();
2168+
let root = dir.path().join("sandbox");
2169+
std::fs::create_dir(&root).unwrap();
2170+
2171+
let readonly_dir = root.join("ro-mount");
2172+
std::fs::create_dir(&readonly_dir).unwrap();
2173+
std::fs::write(readonly_dir.join("child-under-ro.txt"), "data").unwrap();
2174+
2175+
std::fs::write(root.join("writable-sibling.txt"), "data").unwrap();
2176+
2177+
let uid = Some(nix::unistd::geteuid());
2178+
let gid = Some(nix::unistd::getegid());
2179+
2180+
let chowned: Arc<Mutex<Vec<PathBuf>>> = Arc::new(Mutex::new(Vec::new()));
2181+
let chowned_ref = Arc::clone(&chowned);
2182+
2183+
let readonly_dir_clone = readonly_dir.clone();
2184+
let fake_chown =
2185+
move |path: &Path, _uid: Option<Uid>, _gid: Option<Gid>| -> nix::Result<()> {
2186+
if path == readonly_dir_clone {
2187+
return Err(nix::errno::Errno::EROFS);
2188+
}
2189+
chowned_ref.lock().unwrap().push(path.to_path_buf());
2190+
Ok(())
2191+
};
2192+
2193+
chown_children(&root, uid, gid, &fake_chown).expect("EROFS should be handled gracefully");
2194+
2195+
let chowned = chowned.lock().unwrap();
2196+
assert!(
2197+
!chowned.contains(&readonly_dir.join("child-under-ro.txt")),
2198+
"children under EROFS directory must NOT be descended into"
2199+
);
2200+
assert!(
2201+
chowned.contains(&root.join("writable-sibling.txt")),
2202+
"writable sibling should still be chowned"
2203+
);
2204+
}
2205+
2206+
#[cfg(unix)]
2207+
#[test]
2208+
fn chown_recursive_propagates_non_erofs_errors() {
2209+
let dir = tempfile::tempdir().unwrap();
2210+
let root = dir.path().join("sandbox");
2211+
std::fs::create_dir(&root).unwrap();
2212+
2213+
let uid = Some(nix::unistd::geteuid());
2214+
let gid = Some(nix::unistd::getegid());
2215+
2216+
let fake_chown = |_path: &Path, _uid: Option<Uid>, _gid: Option<Gid>| -> nix::Result<()> {
2217+
Err(nix::errno::Errno::EPERM)
2218+
};
2219+
2220+
let result = chown_recursive(&root, uid, gid, &fake_chown);
2221+
assert!(result.is_err(), "non-EROFS errors should propagate");
2222+
}
2223+
2224+
#[cfg(unix)]
2225+
#[test]
2226+
fn chown_children_skips_all_erofs_children_gracefully() {
2227+
let dir = tempfile::tempdir().unwrap();
2228+
let root = dir.path().join("sandbox");
2229+
std::fs::create_dir(&root).unwrap();
2230+
std::fs::create_dir(root.join("a")).unwrap();
2231+
std::fs::write(root.join("b.txt"), "data").unwrap();
2232+
2233+
let uid = Some(nix::unistd::geteuid());
2234+
let gid = Some(nix::unistd::getegid());
2235+
2236+
let always_erofs = |_path: &Path,
2237+
_uid: Option<Uid>,
2238+
_gid: Option<Gid>|
2239+
-> nix::Result<()> { Err(nix::errno::Errno::EROFS) };
2240+
2241+
chown_children(&root, uid, gid, &always_erofs)
2242+
.expect("EROFS on all children should be skipped gracefully");
2243+
}
2244+
21182245
#[cfg(unix)]
21192246
#[test]
21202247
fn rewrite_passwd_modifies_existing_sandbox_entry() {

0 commit comments

Comments
 (0)