fix(acp): bound observer retention by bytes (#7987) - #9366

## Summary

The Agent Client Protocol (ACP) observer records agent activity for the
owner's viewer. Sizing currently allocates JSON strings and rescans
whole frames for each trimmed field.

Count JSON bytes directly, reuse fitted sizes, and count changed fields
when cheaper. Skip raw-message copies for disabled observers and copy
replay payloads outside the mutex. Keep sequence numbers, timestamps,
replay insertion, and publication under one lock to preserve replay/live
ordering.

This adds one `Arc` allocation per replay entry, and snapshots pin
entries until copying completes. Replay retains the existing 1,000-event
limit.

### Related issue

Related to #7987. This PR targets `main` independently of #9366, which
owns the byte-budget fix. The changes overlap in observer plumbing, so
the second PR to land will need a rebase.

### Testing

The new ignored probe exercises the production publish queue. Timing
includes template clones and queue ingestion, before encryption. Five
alternating fresh release processes per arm/case compared `75d38b6c2`
with this commit on a busy host. Every pair retained identical queue
bytes and event counts.

```sh
cargo test --release -p buzz-acp --lib observer::perf_tests::publish_queue_probe -- --exact --ignored --nocapture
```

Set `BUZZ_OBSERVER_PROBE_EVENTS`, `BUZZ_OBSERVER_PROBE_BYTES`, and
`BUZZ_OBSERVER_PROBE_FIELDS` for each row. Medians:

| Events / string bytes per event / fields | Before | After |
| --- | ---: | ---: |
| 200 / 1 MiB / 64 | 5605 ms | 228 ms |
| 1,000 / 1 MiB / 1 | 714 ms | 352 ms |
| 10,000 / 1 KiB / 1 | 13.01 ms | 6.90 ms |

A separate 20-trial integration run used the same production Chromium
app build (`block/buzz-app@128cd4c2`) and four real ACP subprocesses.
Three peers emitted 225 synthetic messages over five seconds, each with
1 MiB across 64 strings. A fourth emitted 50 markers, 100 ms apart. Each
arm/mode used five fresh processes and browser contexts, with
alternating arm order.

The real observer, publisher, encryption/signing, browser broker, and
Activity UI ran through a modeled relay transport. Marker timing used
one controller clock, from release through transcript DOM commit plus
two animation frames. Observer timestamps were not the timing origin.
Observation-enabled medians:

| Metric | Before | After |
| --- | ---: | ---: |
| Rust CPU time | 4.49 s | 0.45 s |
| Marker-to-render p95 | 4471 ms | 1018 ms |
| Sampled peak RSS | 342.3 MiB | 296.9 MiB |

All 500 enabled-mode markers rendered exactly once; all 2,250 large
events were delivered with the existing payload elision.
Observation-disabled CPU stayed roughly flat (0.13 s to 0.12 s). The
controlled publication-order regression also fails original `main` and
passes this commit.

These are stress-workload measurements, not a general app speedup.
Native hosting and deployed-relay performance remain unmeasured. The
publish cadence remains one frame per second. No CI timing threshold was
added.

Generated with Codex

Signed-off-by: Sanket Kanjalkar <[email protected]>
Co-authored-by: Codex <[email protected]>

Replaces #9366 on a branch in this repository and resolves its conflicts
with main.

## Summary

The ACP observer kept up to 1,000 raw events in both replay and live
buffers. Large valid messages could retain gigabytes before downstream
trimming ran.

Fit telemetry before either buffer retains it. Cap replay at 4 MiB of
serialized JSON and keep the existing live window. Its per-event limit
bounds live JSON below 64 MiB.

Preserve the CPU and locking improvements from #9381: prepare large
payloads and copy snapshots outside the replay lock. Finalize sequence
and timestamp with publication. Account for that final metadata,
including sequence digit-width changes, in the byte limit.

The tradeoffs are extra work per emit, shorter replay history for large
events, and elided oversized telemetry. Agent protocol messages remain
unchanged.

### Related issue

Fixes #7987. Existing publisher byte limits act downstream of these
buffers.

### Testing

The opt-in probe exercises the real in-process emit, snapshot and
stalled-subscriber paths:

```sh
CARGO_INCREMENTAL=0 CARGO_PROFILE_RELEASE_LTO=thin CARGO_PROFILE_RELEASE_CODEGEN_UNITS=4 \
  cargo test --release -p buzz-acp --lib observer::tests::retention_probe -- --exact --ignored --nocapture
```

Compared baseline `a314975ce` with candidate `c2df3cb56` on macOS arm64,
Rust 1.98.1. The baseline observer, fitter, workspace manifest and
lockfile match the candidate's parent `8f3f4e266`. Both arms use
`opt-level=3`, thin LTO and four codegen units, with incremental
compilation and compiler-cache wrappers off. This profile is fixed for
the comparison; this PR does not change build settings.

Values are medians of three fresh-process runs per version and workload,
in alternating order. Peak RSS comes from `/usr/bin/time -l` around each
test executable; compilation is excluded. Each version was rebuilt
separately and its executable and workload identity verified.

| Workload | Peak RSS before → after | Emit time per event before →
after |
| --- | --- | --- |
| 10,000 × 1 KiB | 14.0 → 14.1 MiB | 0.47 → 0.89 µs |
| 1,000 × 60,000 bytes | 199.4 → 80.8 MiB | 8.54 → 22.53 µs |
| 1,000 × 1 MiB | 3,025.8 → 26.4 MiB | 123.50 → 307.89 µs |

For the 1 MiB workload, replay JSON fell from 1,000.2 MiB to 3.997 MiB.
Live JSON fell from 1,000.2 MiB to 5.91 MiB. Replay retained 676 fitted
events; the live subscriber received all 1,000.

Emit time includes payload construction. Peak RSS includes snapshot and
receive copies made by the probe. JSON budgets are not exact heap
limits. These local measurements show the memory/CPU tradeoff;
concurrent producer throughput and native app latency were not
benchmarked. See [the probe
instructions](https://github.com/block/buzz/blob/c2df3cb567a3766aa796c50369903b09a8f04036/crates/buzz-acp/TESTING.md#observer-retention-measurement)
for workloads, controls and limits.

Human verification remains pending.

Generated with AI assistance

Signed-off-by: Sanket Kanjalkar <[email protected]>
Co-authored-by: Codex <[email protected]>