Repository navigation
Description
Describe the bug
The process-local ACP observer retains raw payloads in two count-limited stores before the relay publisher trims or byte-bounds them. OBSERVER_BUFFER_CAP = 1_000 bounds event count, but not retained payload bytes. Large, valid ACP messages can consequently retain gigabytes of JSON while the downstream relay queue remains within its 4 MiB budget.
This is a source-confirmed retention gap, not a measured production RSS result. The code remains present on upstream main at d7a35afa9859a0c666de93ab1cd4f8a433a1cfe8.
Steps to reproduce / proposed regression
- Create an in-process
ObserverHandleand retain a subscriber that does not drain immediately. - Emit 1,000 observer events containing 1 MiB ASCII string payloads.
- Inspect the replay snapshot's serialized byte total. The current buffer keeps all 1,000 events because it evicts only when the event count reaches 1,000; the retained string data alone totals 1,000 MiB.
- A lagging subscriber's count-limited broadcast ring also retains raw event payloads.
This reproduction has not been run here; the observations above follow from the current retention paths. A regression should assert both a byte bound and bounded handling of a single event larger than that bound.
Expected behavior
Observer replay and broadcast retention should have a defined byte budget in addition to the count limit. Oversized events should become a bounded representation before retention/broadcast, or be explicitly dropped with loss accounting. Eviction should preserve ordering and recent events. The regression should exercise ObserverHandle::emit and snapshot, including large events and a stalled subscriber.
Version and platform
- Installed environment: Buzz Desktop 0.5.25, macOS.
- Source inspection: upstream
mainatd7a35afa9859a0c666de93ab1cd4f8a433a1cfe8. - The retention implementation is platform independent; other platforms have not been exercised.
Logs / additional context
Relevant production paths:
- Observer buffer count and stores: the replay
VecDequeand broadcast channel are count limited. - Raw observer retention:
emitclones the complete event into the replay buffer, then sends it to broadcast without a byte budget. - ACP input size limit: accepted NDJSON lines can be up to 10,000,000 bytes.
- Raw ACP read observation: parsed messages are observed before the downstream publisher fitting step.
Suggested contribution: add byte accounting to replay retention and bound the representation placed in the broadcast ring, with deterministic tests for many large events, a single oversized event, eviction order, and subscriber lag. Bounding only the replay VecDeque would leave the broadcast ring retaining the same large payloads.
Duplicate searches covered observer replay/buffer/memory/byte retention and the OBSERVER_BUFFER_CAP symbol. Closest related work: #6830 (publish cadence), #5718 (Desktop transcript rebuilding), #5220 (Unix event sink), and #5578/#7086 (NIP-44 plaintext limit). None found addressing this process-local raw retention gap.
Activity
perf(acp): reduce observer CPU and lock overhead (#9381)
fix(acp): bound observer retention by bytes (block#7987) (block#9436)
Merge origin/main into bostonaholic--agent-reply-count
Merge bostonaholic--agent-reply-count into bostonaholic--agent-reply-…
perf(acp): avoid repeated observer batch serialization (#9483)