fix(diagnostics): bound the CEC subprocess reap and report its states distinctly - #3272

Issues Fixed

Fixes #3264. Addresses the user-visible half of #3267 (the passthrough half is a separate PR — see below).

Description

The bounded reap (#3264)

subprocess.run(..., capture_output=True, timeout=N) burns its entire timeout when a descendant still holds the inherited stdout pipe, because communicate() keeps draining it — even when the direct child has already exited.

Measured on three architectures, fast-exiting child with a grandchild holding stdout:

On a task that runs every 5 minutes against a 30s soft limit, repeatedly spending the whole budget for nothing is the real cost.

_run_bounded() differs from subprocess.run in three deliberate ways: start_new_session=True + killpg(SIGKILL) so a grandchild cannot survive the kill or stall the reap; output to temp files rather than pipes, so there is nothing to drain; and a timeout on every wait, walking away rather than blocking if the group somehow persists. It is deliberately not used as a context manager, because Popen.__exit__'s unbounded wait() is one of the very hazards it exists to avoid.

Three distinct CEC states (#3267, partial)

'CEC error' covered three different situations and reported the most common one — a plain monitor with no CEC support — to the operator as a fault. Now:

• 'No CEC adapter' — libcec raised; it found nothing usable.
• 'No CEC display detected' — the adapter works and nothing answered. Expected for a monitor without CEC, which is a large share of signage installs. Not an error.
• 'CEC adapter unresponsive' — libcec neither answered nor raised inside the timeout, so it was killed. This is what actually happens on vchiq-only boards.
• 'CEC error' — genuinely unexpected.

True/False and the str | bool return are deliberately unchanged: they are the data the v2 System Info API surfaces for display_power, so renaming them would be a field-semantics change for external clients. Verified no in-tree consumer branches on the string — api/views/v2.py, api/views/mixins.py, app/page_context.py and system_info.html all pass it straight through.

A bonus fix in the same function

The no-CEC branch's r.set('display_power', 'Not available', ...) sat outside the SoftTimeLimitExceeded handler, making it the one unprotected blocking call in the task — on exactly the boards that take that branch (x86, Pi 5). It is not unbounded (redis-py 8.0.1 defaults socket_timeout=5), but the default Retry(retries=10) costs 58.83s measured, which blows the 30s soft limit and clears the 60s hard limit by only ~1.2s. Outside the try, that soft limit escaped uncaught, failed the task, and filed a Sentry event — the very noise this work removes. Now inside the handler.

Hardware validation

What this does NOT fix, stated plainly

The root cause of ANTHIAS-A/9/B/31 is still open. My original theory — that SoftTimeLimitExceeded re-enters an unbounded wait() so the task sails past the hard limit — does not reproduce on CPython 3.13, which calls process.kill() before every wait(). Overshoot measured 0.00s for both implementations, agreed independently by three boards. And a genuine uninterruptible child still got reaped in 31.5ms.

So this is a robustness and latency fix, not a demonstrated cure for the hard-limit SIGKILL. A reap that actually overruns the grace period needs a driver that never returns, which was not safely inducible. #3264 should stay open after this merges.

Two documented limits of a process-group kill, neither reachable from the CEC scripts (which spawn no child processes — libcec uses threads), are recorded in the docstring: a grandchild calling setpgid(0,0) escapes the killpg, and a fast double-forking child leaves an orphan because no timeout fires at all. Also corrected there: container PID 1 is the celery worker, not an init that reaps orphans — so a killed grandchild would persist. Three boards caught that claim.

Related

The other half of #3267 — passing /dev/cec0 through on pi2/pi3/pi3-64/pi4-64, which is the root cause of display power never working there — is a separate PR, because it changes device passthrough during real OTA upgrades and deserves its own risk review.

Testing

1552 passed, 3 skipped (-m "not integration"), mypy clean across 169 files, ruff check + format clean.

New tests verify the reap behaviourally rather than by inspection: a hanging child dies inside timeout + grace; a grandchild does not survive the process-group kill; an unspawnable argv returns None rather than raising (this runs inside both an HTTP request and a celery beat); stdout and stderr stay separate so libcec chatter cannot corrupt the single-token contract. The 16 pre-existing CEC tests were re-pointed from the subprocess.run seam to _run_bounded, preserving every behavioural assertion — message formatting, length capping, stderr fallback, returncode fallback.

Checklist

• I have performed a self-review of my own code.
• New and existing unit tests pass locally and on CI with my changes.
• I have done an end-to-end test for Raspberry Pi devices.
• I have tested my changes for x86 devices.
• I added a documentation for the changes I have made (when necessary).