Skip to content

Commit 0020907

Browse files
authored
fix(relay): reject presence updates when Redis storage fails (#7532)
## Summary - Reject kind:20001 presence events with `OK false` / `error: presence storage unavailable` when Redis SET or DEL fails, before publishing, local fan-out, or local-event marking. - Preserve the producer contract needed by snapshot-confirming consumers: delivered live presence must follow successful mutation of the Redis state read by snapshots. - Classify those backend rejections with the existing `IngestError` taxonomy so a presence storage outage counts as `buzz_events_rejected_total{transport="ws",reason="error"}`, not client `reason="invalid"`; genuine client-input refusals (verification failure, membership gates) stay `invalid`, and every wire message is an unchanged fixed sanitized string (review follow-up, no protocol wording change). - Add actual `handle_event` integration coverage for rejected online/offline transitions, healthy online→offline accepted/stored/fanned-out behavior, and the rejection-counter routing on storage failure with an invalid-signature control. This is standalone on main; it does not depend on the mobile implementation. Deploy this relay prerequisite before relying on #7526's snapshot-confirmation policy. Existing pubsub-failure-after-successful-storage behavior and disconnect TTL cleanup are deliberately unchanged. A storage error may be an ambiguous write outcome, not a rollback guarantee; the rejected event is not published by this handler. Clients may retry the generic `error:` rejection. Desktop's 60s heartbeat retries non-offline presence, not every explicit offline transition. ### Related issue Addresses the relay prerequisite identified in [#7526 review 5157607827](#7526 (review)). Searched open presence/storage PRs; no duplicate relay storage-error rejection fix found. #7382/#7383/#7526 heads and bases are unchanged. ### Testing Exact head: `389174df29cc02d0f885c03209eff661d8bb2ec0` (+380/-13; 393 total), one commit `389174df2` on top of the reviewed `c031d6eb1` (DCO-signed; base `bfc384855889432df4a333a0edf3080f332ee169` unchanged). - PASS: `cargo fmt --all -- --check`, `cargo clippy -p buzz-relay --all-targets -- -D warnings`, `git diff --check`, `just file-size-check`, PostgreSQL discovery validation — all run at the exact final head with a clean tree before and after. - PASS: documented native `scripts/postgres-test-run.sh -p buzz-relay --lib --tests`: **89/89** actual integration tests, including the four presence cases (online/offline storage rejection, healthy online→offline, and the new rejection-classification case). Owned PostgreSQL 17/Redis on isolated loopback ports, schema plus reconciliation applied; no shared development database. - PASS: explicit `cargo test -p buzz-relay presence_storage -- --ignored --nocapture`: **4/4**, not skipped. - Full isolated relay crate suite at the final head (`cargo nextest run -p buzz-relay --lib --tests`): **1062 run: 1062 passed, 94 skipped**. The previously failing `api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo` passed in this run (1.5s); it is a known timing-sensitive main baseline failure tracked open in #7140 and untouched by this PR, so this single passing run is reported as-is and does not claim environmental clearance or close #7140. No full-suite-green claim is made beyond this run. - Mobile is untouched; #7526's existing 2090-test/format/analyze evidence remains scoped to its unchanged head. Its separate Desktop Smoke E2E (2) failure remains red; no CI retries requested. [Production-seam regression coverage](https://github.com/block/buzz/blob/389174df29cc02d0f885c03209eff661d8bb2ec0/crates/buzz-relay/src/handlers/event.rs#L1491-L1803): the metric case drives real `handle_event` traffic against a genuinely dead Redis endpoint with a seeded active PostgreSQL community and a registered presence watcher, asserts the storage rejection counts `reason="error"` while a tampered-signature control through the same dispatcher arm stays `reason="invalid"`, and re-asserts the rejected ACK, no fan-out, and no local-event marker. Counter assertions use a thread-local recorder guard held across `.await` points (the buzz-db counter-test convention) inside the per-process nextest postgres-ci lane, so no parallel test can race the counter snapshot. No UI change or screenshot. Local logs and reproducible service/gate scripts are retained under `WORK_LOGS/MOBILE_FEEDBACK_PRESENCE_20260909/relay_prerequisite/metric_correction/` in the engineering workspace. This PR is a review candidate, not merge clearance. Causal checks: restoring only the pre-fix production mutation block makes both original rejection tests fail (`OK true` instead of `false`); healthy success still passes. Reverting only the typed classification (mapping the ephemeral `Internal` arm back to `invalid`) makes the new metric regression fail with the outage counted as `[("ws","invalid",2)]` instead of `[("ws","error",1),("ws","invalid",1)]`. The unchanged mesh echo case also failed 504/200 with the main-production block restored in the prior run, supporting its separation from this change without claiming environmental clearance. Candidate source restored byte-for-byte after each mutation. Repository-wide `just ci` was not rerun; the scoped relay gates above are the new evidence. --------- Signed-off-by: Logan Johnson <[email protected]>
1 parent cec5c8f commit 0020907

1 file changed

Lines changed: 380 additions & 13 deletions

File tree

0 commit comments

Comments
 (0)