Repository navigation
Commit 622a157
authored
fix(sandbox): keep idle execs and disconnect freezes off the shared blocking pool (#5217)
* fix(sandbox): read exec pipes and PTY masters without blocking pool threads
Exec stdin/stdout/stderr and the PTY master were wrapped in
tokio::fs::File, so every pending read on an idle pipe parked one tokio
blocking-pool thread until data arrived or the peer closed. That pool is
shared with the control plane (gRPC exchange, exec spawn/reap,
AcceptNetwork), and enough idle exec sessions exhausted it, blocking new
control connections.
Adopt the descriptors with a readiness-based AsyncFd adapter instead:
reads and writes only run when the descriptor is ready, and idle execs
park no pool threads. Flush and shutdown keep the tokio::fs::File
semantics the consumers were written against, and close-on-drop remains
the EOF mechanism, so no consumer changes.
An isolated boundary test caps the blocking pool at 16 threads, starts
12 idle execs, and asserts a fresh exec and a control request still
complete; without the fix it fails when the sixth idle exec stalls.
Signed-off-by: Emilien Macchi <[email protected]>
* fix(sandbox): defer freeze descendant scans off the supervisor lock
transport_disconnected froze the workload while holding the
supervisor_connection mutex, and the freeze signaled every descendant
through up to four synchronous /proc scans, so losing the supervisor
stalled a runtime worker and the lock for as long as the scans took,
which grows with the number of processes in the sandbox.
Freeze and resume now stop and continue the registered root groups
synchronously, so a confirmed recovery still implies the roots were
continued, and defer the descendant sweeps to a dedicated thread: the
shared blocking pool can be fully occupied, and a sweep queued behind
it would leave detached children running while the boundary reports
itself frozen. If thread creation itself fails, the sweep runs
synchronously rather than leave the freeze unenforced. Each sweep
collects the owned set without holding any lock and applies the
signal under a signal mutex only while the transition generation it
captured is still current, so a stale deferred SIGSTOP can never land
after a resume SIGCONT. A sweep that fails to deliver reports the gap
as an OCSF process activity naming the recovery.
The scans now run under neither supervisor_connection nor the signal
mutex; a transition that immediately follows another waits at most one
signal application, never a scan.
Closes #5184
Signed-off-by: Emilien Macchi <[email protected]>
---------
Signed-off-by: Emilien Macchi <[email protected]>1 parent f37687a commit 622a157
4 files changed
Lines changed: 2086 additions & 84 deletions
File tree
- crates/openshell-sandbox/src
0 commit comments