Repository navigation
fix(relay): bound each cross-node delivery attempt - #9305
Conversation
|
@codex review |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
A fan-out worker awaited the access-gate read with no limit, so a database connection that stopped answering held the worker forever and its channels got no cross-node live events until a restart. Cut each attempt off after 30 s from when the worker starts it: count it as dropped (reason=stale), resync the syncs it matched, close nothing and keep draining. Signed-off-by: Tom Brow <[email protected]>
68363a6 to
3f00a61
Compare
github-actions
Bot
added
the
codex-security-review-current
🔐 Codex Security Review
Review SummaryOverall Risk: NONE
FindingsNo concrete security, correctness, or reliability findings were identified. Notes
Generated by Codex Security Review | |
kazemek21
left a comment
kazemek21
left a comment
There was a problem hiding this comment.
Kyle reviewed this PR and the Nexus review-tracker notes and approved with no comments.
Posted by Codex (gpt-6.1-sol) on behalf of kylez.
…small * origin/main: fix(db): seek replaceable heads by channel (#9147) buzz-db: check events created_at floor in app writers (phase 1) (#8325) fix(mobile): keep a thread tail behind the keyboard unread (#9295) fix(mobile): restore header cues and per-channel identity check for #9369 (#9370) fix(relay): treat empty filter sets as match-nothing in SQL (#9342) buzz-cli: add --before and --before-id to messages search (#9382) feat(relay): agents cannot write in DMs except with their owner (#9371) fix(relay): bound each cross-node delivery attempt (#9305) feat(mobile): add iOS Home Screen shortcuts for New chat, New DM and the Bestie (#9360) perf(mobile): stop downloading whole videos for posterless stills (#9367) Enforce single-community relay mode (#9149) fix(relay): match #a tag filters in SQL before LIMIT (#9339) Add participant avatars to mobile conversation headers (#9369) fix(mobile): stop asking for an API token when the channel list is refused (#9327) chore(release): release Buzz Relay version 0.3.0 (#9362) Fix gallery video selection, thumbnails, and swipe dismissal (#9356) Signed-off-by: dessert <7d616e7e143e5d408159614dccff1371350a1c1bd36c8de838a45ece5aca0aca@buzz.block.builderlab.xyz>
Context: #9302 (merged) stopped the relay from closing connections when cross-node fan-out falls behind. It also removed the 30 s delivery deadline. This PR is the fix from the review of #9302.
Problem: Since #9302, a fan-out worker waits for the access-gate database read with no time limit. If a database connection stops answering in the middle of that read, the worker never continues. Its queue fills, and one sixteenth of the node's channels get no live events from other nodes until the relay restarts. Before #9302 the deadline cut that delivery off after 30 s.
Fix: Each delivery attempt has a 30 s limit (
MAX_DELIVERY_ATTEMPT), measured from when the worker starts it. When the limit passes, the relay counts the event inbuzz_multinode_fanout_dropped_total{reason="stale"}, resyncs the artifact syncs the event matched, and continues with the next event. It closes no connection. An event that only waits in the queue is still delivered late, as #9302 intends. The queue-full drop and the new stale drop share one helper.Validation
hung_delivery_is_dropped_at_the_bound_and_the_worker_moves_ondrives the production worker with a database that accepts the connection and never answers. It checks that the delivery stops at the limit, is counted, resyncs the matched sync, closes nothing, and that the next event on the same worker is delivered. With the limit removed it fails (the worker waits out the pool's one-hour acquire timeout instead).handlers::multinode_fanouttests pass. The full localbuzz-relaysuite was not run; CI on this head covers it.🤖 Generated with Claude Code