Skip to content

Return system.run when the shell exits, not when its pipes close - #26

Merged
shaeberling merged 1 commit into
facebookincubator:mainfrom
kartsan03:fix/system-run-wait-exit
Oct 5, 2026
Merged

shaeberling merged 1 commit into
facebookincubator:mainfrom
kartsan03:fix/system-run-wait-exit

Conversation

@kartsan03

Copy link
Copy Markdown
Contributor

Hey @anantn ,

Problem

PR #3 was a partial fix: it bounded only the after-kill grace wait. The first communicate() in Executor.system_run still waits for stdout and stderr EOF, not for the shell to exit.

A command that daemonizes and exits 0 while a child holds the inherited pipes keeps those pipes open. communicate() burns the full timeout, killpg then raises ProcessLookupError because bash is already gone, and the grace wait also times out. The gadget reports failure for a command that succeeded and blocks the command loop for the whole timeout (default 120s, max 600s).

Reproduction on main:

ex.system_run({"command": "setsid sleep 30 &", "timeout_ms": 2000}, None)

returns after about 4 seconds with timed_out: true and exit_code: 0.

Fix

Wait for the shell process itself, and read only bytes already queued on the pipes. If the shell exits in time, return timed_out: false and its real exit code, without calling communicate() in a way that waits for EOF. If it does not exit, the kill-and-grace path from PR #3 is unchanged.

Test

test_system_run_returns_when_the_shell_exits_but_a_child_holds_the_pipes runs setsid sleep 30 & with a 2000 ms timeout and asserts it returns in under a second with timed_out false and exit_code 0. It also checks that echo hi still returns stdout and exit 0, and that sleep 30 with a 300 ms timeout still reports timed_out. On main that test fails in 4.04s (exit_code 0 and timed_out true). With this change, pytest tests/test_executor.py is 13 passed and the linux suite is 138 passed.

Not tested on a real device, over BLE or against the Muse service. Only the unit suite.

The first communicate() in system_run waited for stdout and stderr EOF,
not for the shell process. PR facebookincubator#3 only bounded the grace wait after a
timeout kill. A command that daemonizes and exits 0 while a child still
holds the inherited pipes (setsid sleep 30 &) therefore burned the full
timeout, killpg raised ProcessLookupError because bash was already gone,
and the grace communicate timed out too. The call returned timed_out
with exit code 0 and blocked the gadget command loop for the whole
timeout.

Wait for the shell itself. If it exits in time, drain only the bytes
already queued on the pipes and return that exit code with timed_out
false. If it does not, keep the existing kill-and-grace path.
@meta-cla meta-cla Bot added the cla signed label Oct 3, 2026
@shaeberling
shaeberling self-requested a review October 5, 2026 21:41
@shaeberling
shaeberling merged commit 6c33c12 into facebookincubator:main Oct 5, 2026
23 checks passed
shaeberling added a commit that referenced this pull request Oct 5, 2026
_read_available, added in #26, reads until a pipe is empty or closed and
never checks the time. A writer that outpaces the reads could keep it
looping past the command's timeout, and the read after the shell exits
had no limit at all. Stop at the command's deadline while it runs, and
after DRAIN_S (0.5 s) once the shell has exited.
blacksamuraiiii pushed a commit to blacksamuraiiii/muse-esp32-s3-touch-lcd-1.85b that referenced this pull request Nov 3, 2026
_read_available, added in facebookincubator#26, reads until a pipe is empty or closed and
never checks the time. A writer that outpaces the reads could keep it
looping past the command's timeout, and the read after the shell exits
had no limit at all. Stop at the command's deadline while it runs, and
after DRAIN_S (0.5 s) once the shell has exited.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants