fix(viewer): bound AnthiasViewer output instead of buffering it in RAM - #3147
Conversation
The viewer spawns AnthiasViewer via sh with `_bg=True, _err_to_out=True` and no output sink, so sh aggregates the process's entire merged stdout+stderr in the viewer's RAM for the whole session and nothing drains it after the D-Bus handshake. When a file's audio makes ffmpeg's AAC decoder spam `channel element 0.0 duplicate` on every frame, that buffer grows without bound. On a 2 GB Pi4 it exhausts RAM and drives the box into swap, so asset transitions blank the screen for minutes and the device can wedge until the next asset is forced. Pass a bounded `_BoundedWebviewOutput` as `_out`: sh then streams each chunk to it instead of retaining the stream (and stops aggregating `.process.stdout`), and we keep only the last 64 KiB — enough for the startup-handshake scan and a crash tail. The handshake scan and the launch-failure message now read the sink; WEBVIEW_DEBUG reads it too. Verified on hardware with the shipped sh 2.3.0: the old spawn flags grew a Python process from 23 MB to 670 MB RSS in 24 s of decoder-style spam, while an `_out` sink held RSS flat at 23 MB. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
This PR addresses viewer memory exhaustion on low-memory devices by preventing sh from buffering AnthiasViewer’s merged stdout/stderr indefinitely in RAM (issue #3138). It introduces a bounded output sink that retains only the most recent output needed for handshake detection and crash diagnostics.
Changes:
- Add
_BoundedWebviewOutputand pass it assh’s_outsink to avoid unbounded stdout aggregation. - Switch handshake detection and launch-failure diagnostics from
candidate.process.stdoutto the bounded sink. - Update and extend unit tests to drive/capture the
_outsink and add bounded-buffer regression coverage.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/anthias_viewer/__init__.py |
Adds bounded output sink and routes handshake/debug output through it instead of sh’s accumulated stdout buffer. |
tests/test_viewer.py |
Refactors spawn/handshake tests to capture the _out sink and adds tests for bounded-buffer behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address review feedback on the issue #3138 fix. - _BoundedWebviewOutput.__call__ now coerces bytes to str. sh hands the _out callback str under the default encoding (verified with the shipped sh 2.3.0 on a real Pi4), so this is defensive: an _encoding=None / raw-bytes configuration would otherwise turn `str + bytes` into a TypeError that silently breaks the handshake scan. - Publish `_webview_output = output` at the start of _spawn_webview_once instead of only on handshake success, so the module global truly mirrors `browser` for the live attempt and a WEBVIEW_DEBUG read during launch/retry can't surface a previous process's stale tail. - Add a regression test for the bytes coercion. Validated on the 1 GB Pi4 (.143) viewer container with the shipped sh 2.3.0: the current no-_out flags grow the process RSS +76 MB for 12 MB of decoder-style spam (sh retains all 12 MB), while the bounded sink holds RSS flat (+0.5 MB) with 0 bytes aggregated and still detects the handshake. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…hunk Address further review feedback on the issue #3138 fix. _BoundedWebviewOutput.__call__ appended the whole incoming chunk before trimming, so a single chunk larger than the window (a big unbuffered write from the child) would force a `buf + chunk` allocation proportional to the chunk — spiking past maxlen, the very thing the sink exists to prevent. Slice an oversized chunk to its tail first (identical tail-retention result); the retained buffer and the transient now both stay within ~2x maxlen regardless of chunk size. Add a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…iced string Address further review feedback on the issue #3138 fix. `self._buf += chunk` followed by slicing to `[-maxlen:]` re-copied ~maxlen characters on every call once the window was full. A chatty decoder calls the sink per audio frame for the life of the process, so that is steady CPU burn on a weak Pi — against the spirit of a fix meant to be gentle on the box. Retain a deque of raw chunks instead: append and drop whole oldest chunks (amortized O(1), no recopy of kept data), and join to a single string only in text(), which is read rarely (the startup handshake poll, where the buffer is still small, and WEBVIEW_DEBUG). Retained size stays within [maxlen, maxlen + newest-dropped-chunk) < 2x maxlen. Add a test asserting the write path keeps a bounded, small chunk count under 10k writes. Re-validated on the 1 GB Pi4 (.143) viewer container through the shipped sh 2.3.0: handshake still detected, RSS flat (+0.4 MB) under 400k lines of decoder spam, retained tail exactly 64 KiB, sh aggregates 0 bytes. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…DISPLAY Address review feedback on the issue #3138 fix. _spawn_webview_once calls _wait_for_wayland_socket before spawning the command. On a Wayland desktop (WAYLAND_DISPLAY set to a socket that isn't present) that wait calls the mocked sleep, firing feed_handshake before the _out sink is captured — a KeyError on holder['sink'], or a spinning socket-wait loop. CI/Docker leaves WAYLAND_DISPLAY unset so it passed there, but the test was host-dependent. Clear WAYLAND_DISPLAY so _wayland_socket_path() returns None and the pre-spawn wait is a guaranteed no-op (sleep is then exercised only by the handshake poll loop), and guard feed_handshake so a stray pre-spawn sleep can never KeyError on an uncaptured sink. Verified the test now passes with WAYLAND_DISPLAY pointed at a nonexistent socket. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
tests/test_viewer.py:252
- These tests can become environment-dependent: if
WAYLAND_DISPLAY/XDG_RUNTIME_DIRare set but the Wayland socket path is absent (common in some CI/dev containers),_spawn_webview_once()will spend the entire startup budget in_wait_for_wayland_socket(). That changes the control flow (timeout path) and can make this test hang or fail (e.g., callingterminate()unexpectedly). ClearWAYLAND_DISPLAYhere (as done intest_load_browser) to keep the test hermetic.
def test_spawn_webview_once_raises_on_early_exit(
viewer_fixtures: _ViewerFixtures,
) -> None:
Address review feedback on the issue #3138 fix. The hermeticity fix from the previous commit covered only test_load_browser, but every test that drives _spawn_webview_once / load_browser has the same exposure: with sleep mocked, a set-but-dangling WAYLAND_DISPLAY makes the pre-spawn _wait_for_wayland_socket busy-loop for the whole startup budget (test_spawn_webview_once_raises_on_early_exit, _terminates_on_timeout, load_browser retry tests). Clear WAYLAND_DISPLAY once in the viewer_fixtures fixture so _wayland_socket_path() returns None and the wait is a no-op for the whole module; drop the now-redundant per-test delenv. The dedicated _wait_for_wayland_socket tests set their own env and don't use this fixture, so they're unaffected. Verified the full viewer suite passes with WAYLAND_DISPLAY pointed at a nonexistent socket (previously several tests would spin for the budget). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…stdout-accumulation # Conflicts: # src/anthias_viewer/__init__.py
|



Problem
Fixes #3138.
On a 2 GB Raspberry Pi 4 with large MP4s, moving to the next asset blanks the screen for minutes and the viewer can wedge until the next asset is forced in the UI.
Root cause: the viewer spawns
AnthiasViewervia theshlibrary with_bg=True, _err_to_out=Trueand no output sink.shtherefore aggregates the process's entire merged stdout+stderr in the viewer process's RAM for the whole session, and nothing ever drainsbrowser.process.stdoutafter the D-Bus handshake. When a file's audio makes ffmpeg's AAC decoder spamchannel element 0.0 duplicateon every frame (245k lines / 203 MB in the reporter's attached debug log), that in-RAM buffer grows without bound. On a 2 GB board it exhausts memory and drives the box into swap — hence the multi-minute blank transitions and the "device unresponsive for ~5 min" reports. Withdebug_loggingon,sh's own per-chunk DEBUG logging multiplies it into the 203 MB log.Fix
Pass a bounded
_BoundedWebviewOutputsink as_out.shthen streams each chunk to it instead of retaining the stream — and, importantly, stops aggregating.process.stdout— so we keep only the last 64 KiB, which is enough for the startup-handshake scan and any crash tail. The handshake scan and the launch-failure message now read the bounded sink;WEBVIEW_DEBUGreads it too.Verification
On the real Pi 4 (arm64), inside the viewer container, exact shipped
sh2.3.0 — feeding a fixed 28.6 MB of decoder-style spam through the two spawn variants:_outsink)_outbounded sink)Same result reproduced on x86 (23 MB → 670 MB RSS in 24 s of continuous spam under the old flags; flat 23 MB with the sink). The bounded probe left the board healthy (no wedge).
Unit tests: added a direct regression test that the sink discards old data (bounded) and preserves the handshake in its window; reworked the spawn/handshake tests to drive the new sink. Full non-integration suite: 1245 passed.
ruff check+ruff format --checkclean.🤖 Generated with Claude Code