Skip to content

Commit 9b1c4f9

Browse files
wpfleger96DuncanHayt
authored
🤖 perf: bound long-thread aux reads and make query deadlines terminal (#7854)
Long threads (over ~100 replies) time out in desktop. The aux hop that loads reactions and other e-tag references builds its filter as one `tags @> …` term per id, joined with `OR`. Past about 100 ids the planner switches to walking each partition's primary key backward, expecting to stop early. When few rows actually match, it scans the whole community. With a dirty GIN pending list, a 188-reply thread never finishes. ### What changes - **Set-based e-tag reads (`buzz-db`).** The aux e-tag query now puts the id set behind a `MATERIALIZED` CTE and matches with `ANY`. Each e-tag read also runs under a transaction-local `statement_timeout` of 20 s. - **Terminal timeout contract (`buzz-relay`).** When a read hits the deadline, the client gets a terminal timeout (`CLOSED` for REQ, the timeout error for COUNT and search), not a retryable failure. - **Subscription lifecycle (`buzz-relay`).** The per-connection subscription map lock is now the lifecycle lock. Claims, replacement, client `CLOSE`, owner-checked timeout and search cleanup, revoke, and connection teardown all change the map, the registry, topic refcounts and terminal frames while holding it. As a result, a timed-out subscription's `CLOSED` goes out before any replacement with the same id can claim it, and claims made after teardown are refused. - **Dropped terminal frame disconnects (`buzz-relay`).** If the `CLOSED` for a retired subscription (owner-checked timeout or channel eviction) can't be queued, the relay cancels the connection instead of leaving the client holding a subscription that will never receive fan-out. A client-initiated `CLOSE` ack is exempt. - **COUNT under the same deadline (`buzz-db`).** The fast COUNT path runs e-tag counts under the same transaction-local 20 s cap as SELECT, so WS COUNT and HTTP `/count` produce the terminal timeout too. - **Replica reads surface the deadline (`buzz-db`).** When a replica-routed read is cancelled by `statement_timeout` (SQLSTATE `57014`), the error is returned instead of re-running the query on the writer, which would run the same slow statement. Other replica errors still fall back to the writer. `PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md` records the exception. - **Deadline handling (desktop).** Only the relay's `query timed out` counts as a query deadline. A client-side `request timed out` stays retryable. For one-shot reads a deadline is terminal: the global query retry (budget 1), thread-replies retry and reaction hydration don't re-send the same slow query. Reaction ids held back by a deadline are released when the channel is opened again, so reopening retries them while same-channel re-renders don't. Live subscriptions, in both the renderer and the native archive-sync socket, keep treating a deadline `CLOSED` as transient: they back off and resubscribe with the same filter instead of being dropped. - **Deadline handling (mobile).** Mobile recognizes the deadline on both transports: HTTP `503 {"error":"query timed out"}` and the WS `CLOSED` reason. - The root `ProviderScope` retry policy returns no retry for deadline errors and keeps Riverpod's default backoff for everything else. - Background thread-summary recounts don't retry a deadline. - Forum posts polling pauses while the last load settled on a deadline; reopening the forum retries. - A channel's newest-window load surfaces a deadline instead of falling back to legacy WS history, and the channel list's batched queries return "unavailable" on a deadline instead of fanning out to per-filter fallbacks. An unavailable batch is not treated as empty. Known last-message timestamps and unread state are kept, and unread catch-up moves `lastMessageAt` forward (never backward) the same way live events do, so a cold start with no known timestamp still gets one. - A live subscription that gets a deadline `CLOSED` backs off and resubscribes with the same filter. History requests still fail on it. - Settled load failures show a Retry control, matching desktop's thread-replies error card: a shared `LoadErrorView` on the channel timeline, forum thread and forum posts list, and an icon button beside the thread replies summary. While the thread query has no value, the thread page shows the route snapshot plus live, cached and optimistic replies, with the loading or error status still visible. - Other re-sends (reconnect, foreground resume, live events, the unread backstop, scroll) are not suppressed; each is bounded by the relay's 20 s statement deadline. - **Long-thread seeder.** `just seed-long-thread [replies]` posts one thread (default 187 replies) with some reactions to the local dev relay, for reproducing long threads in the apps. It uses a throwaway key, refuses non-loopback relays and ignores ambient `BUZZ_*` credentials. Writes are paced to fit the stock relay rate limit. **Lifecycle concurrency.** The lock serializes one connection's subscription changes. Other connections can still be delayed by pressure on the shared pubsub command queue, and by serial eviction loops waiting behind a blocked connection. This PR does not claim strict isolation between connections. **Recovery vs. load.** Clients stop automatic retries of the same one-shot read, but they don't remember deadlines across other triggers: reconnect, resume, live events and the unread backstop can each send the read again later, bounded by the 20 s statement deadline. A live subscription that hits the deadline resubscribes with its original filter under the existing 1–30 s backoff, and against a filter that keeps timing out it retries at the 30 s cap indefinitely, so the worst case is one bounded query per subscription every 30 s. `since` is deliberately not advanced to now on resubscribe, because that would skip events missed while the subscription was closed. Unchanged from `main`: when mobile's channel history sync fails with cached messages available, the cached messages are shown without the error or Retry control. ### Benchmark Local reproduction on an isolated bench stack (not committed): - PG16, migrations applied, one 188-reply thread plus 5M background events in the same community. - Each request replays the desktop thread-replies filter through the relay, with `include_aux` on. Times are total ms for 3 runs, with a 60 s client cap. GIN pending-list state is controlled with `VACUUM` (clean) or by refilling the pending list (dirty), and recorded per partition with `pgstatginindex`. Dirty runs held 402 pending pages on every partition's tags index at the start and end of each matrix. Clean runs held 0–1 pages (relay startup writes). | build | GIN pending state | limit 10 | 50 | 100 | 200 (188 ids) | |---|---|---|---|---|---| | `main` | clean | 26 / 16 / 13 | 56 / 51 / 52 | 92 / 93 / 95 | 662 / 639 / 641 | | `main` | dirty | 315 / 298 / 327 | 22141 / 15888 / 16769 | 28992 / 27819 / 28080 | timeout ×3 (>60 s) | | this PR | clean | 35 / 21 / 14 | 57 / 49 / 48 | 82 / 80 / 75 | 153 / 150 / 135 | | this PR | dirty | 290 / 254 / 251 | 1257 / 1204 / 1206 | 2432 / 2378 / 2382 | 4333 / 4706 / 5104 | Baseline `origin/main` `b37e47721`; this PR at `5f2650f5f`, whose e-tag read builders are identical to the current head. At head `1ad81d868` against `origin/main` `15d44dcce`, limit 200 measured 145–179 ms clean (`main` 683–715) and 4.41–4.51 s dirty (`main` timeout ×3). Every completed cell returns the same signed event set: 209 events, 187 replies at limit 200. With `include_aux` off, every cell is ≤66 ms. The thread-window loader (`select_aux`) is unchanged from `main`, and its timings and captured plans match `main`'s. Actual aux-hop statements captured from each relay binary with `auto_explain` (analyze + buffers), execution ms: | build | clean, 100 ids | clean, 188 ids | dirty, 100 ids | dirty, 188 ids | |---|---|---|---|---| | `main` (N-way `OR`) | 40 | 298 | 28785 | 62892 | | this PR (`MATERIALIZED` fence + `ANY`) | 23 | 45 | 1992 | 3539 | **What the fence changes** - On `main`, the planner combines `ORDER BY created_at … LIMIT` with an overestimated match count and walks each partition's primary key backward, expecting to stop early. With few real matches it scans the whole community: the dirty 188-id statement touches ~4.9M shared buffers on `main` vs ~0.6M with the fence. - The fence prevents that early-stop plan. Across the measured cases (1–188 ids, both pending states), the planner used the GIN tags index. The fence doesn't force GIN in general; a broad enough filter could still get a different plan. **Tradeoff:** the fence materializes every match before sorting. In this bench small limits stayed within run-to-run noise of `main`. A single hot id (5,000 reactions, limit 100) measured 30–34 ms unfenced vs 35 ms fenced. **Deadline:** each e-tag read (SELECT and COUNT) runs under a transaction-local `statement_timeout` of 20 s. It only tightens: a shorter operator timeout (`BUZZ_DB_STATEMENT_TIMEOUT_MS`) is kept. It's a per-statement limit, not a per-request one, so a request that runs several pages or hops can take longer than 20 s in total. The benchmark does not force the deadline. --------- Signed-off-by: Will Pfleger <[email protected]> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
1 parent cc36d25 commit 9b1c4f9

54 files changed

Lines changed: 3769 additions & 213 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Justfile‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -461,7 +461,8 @@ test-unit:
461461
# `handlers::`: the wider set is mostly Postgres-backed, and five of its
462462
# non-postgres_tests cases only "pass" without a database by waiting out
463463
# the ~30s sqlx acquire timeout, so they do not belong in the infra-free
464-
# unit job either.
464+
# unit job either. The REQ subscription-lifecycle tests are picked by
465+
# exact name for the same reason: the rest of handlers::req needs a DB.
465466
# The third clause adds the NIP-FI HTTP ingress and its router/config
466467
# neighbours: nip_fi_http, nip_fi_config, router, api::parse_query_tests,
467468
# and the Git transport off_mode_precedence_tests. All are infra-free
@@ -472,7 +473,7 @@ test-unit:
472473
# because they live in the binary target; the nested
473474
# `tests::postgres_tests::` stays in the PostgreSQL lane.
474475
cargo nextest run -p buzz-relay --lib --bin buzz-relay \
475-
-E 'test(/^api::admin::/) + test(/^handlers::channel_authz::/) + test(/^handlers::moderation_authz::/) + test(/^handlers::side_effects::tests::/) + test(/^storage_sweep::tests::/) + test(/^nip_fi_http::tests::/) + test(/^nip_fi_config::tests::/) + test(/^readiness::tests::/) + test(/^router::tests::/) + test(/^api::parse_query_tests::/) + test(/^api::git::transport::off_mode_precedence_tests::/) + test(=state::tests::neither_a_confirmed_inactive_community_nor_a_failed_lookup_admits_the_socket) + (kind(bin) & (test(/^tests::/) + test(/^composition_tests::/)) - test(/^tests::postgres_tests::/))'
476+
-E 'test(/^api::admin::/) + test(/^handlers::channel_authz::/) + test(/^handlers::moderation_authz::/) + test(/^handlers::side_effects::tests::/) + test(/^storage_sweep::tests::/) + test(/^nip_fi_http::tests::/) + test(/^nip_fi_config::tests::/) + test(/^readiness::tests::/) + test(/^router::tests::/) + test(/^api::parse_query_tests::/) + test(/^api::git::transport::off_mode_precedence_tests::/) + test(=state::tests::neither_a_confirmed_inactive_community_nor_a_failed_lookup_admits_the_socket) + test(=handlers::req::tests::timed_out_historical_read_deregisters_before_closed) + test(=handlers::req::tests::superseded_timeout_leaves_replacement_intact) + test(=handlers::req::tests::search_claim_retires_live_and_yields_to_replacement) + test(=handlers::req::tests::concurrent_claims_and_stale_teardowns_keep_the_last_owner) + test(=handlers::req::tests::timeout_closed_is_emitted_before_a_replacement_can_claim) + test(=handlers::req::tests::revoke_then_replacement_keeps_replacement_whole) + test(=handlers::req::tests::claims_after_connection_cleanup_are_refused) + test(=handlers::req::tests::dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::revoke_dropped_terminal_frame_cancels_connection) + (kind(bin) & (test(/^tests::/) + test(/^composition_tests::/)) - test(/^tests::postgres_tests::/))'
476477
# ACP author-gate and queue tests protect the trust boundary between
477478
# relay events and agent prompts. They are infra-free; ignored lifecycle
478479
# tests remain excluded and run in their dedicated integration lanes.
@@ -565,6 +566,10 @@ relay: bootstrap _ensure-migrations
565566
set +o allexport
566567
cargo run -p buzz-relay
567568
569+
# Post one root + N replies (and a few reactions) to the local dev relay, for eyeballing long threads
570+
seed-long-thread replies="187":
571+
./scripts/seed-long-thread.sh {{replies}}
572+
568573
# Start the relay with the built web UI served from it
569574
relay-web: bootstrap _ensure-migrations
570575
#!/usr/bin/env bash

‎PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,10 @@ created: 2026-08-28
1212
routing**: the read may be served from a read replica when
1313
`BUZZ_REPLICA_READ_MAX_AGE_MS` is set, subject to the soundness predicate
1414
`RoutePredicate::for_query` derives from the query shape. The seam fails closed
15-
to the writer at every error and is a genuine no-op until the budget is
16-
configured (`crates/buzz-db/src/lib.rs`).
15+
to the writer on replica errors, with one deliberate exception: a statement
16+
cancelled by `statement_timeout` (SQLSTATE `57014`) is surfaced rather than
17+
re-run, because the writer would run the same slow statement. It is a genuine
18+
no-op until the budget is configured (`crates/buzz-db/src/lib.rs`).
1719

1820
## Routing rule
1921

‎crates/buzz-db/src/error.rs‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,5 +84,17 @@ pub enum DbError {
8484
LastOperator,
8585
}
8686

87+
impl DbError {
88+
/// Whether Postgres cancelled the statement (SQLSTATE 57014:
89+
/// `statement_timeout` or an explicit cancel). Re-running such a query on
90+
/// the writer would only repeat the same expensive work.
91+
pub fn is_statement_cancelled(&self) -> bool {
92+
matches!(
93+
self,
94+
DbError::Sqlx(sqlx::Error::Database(db)) if db.code().as_deref() == Some("57014")
95+
)
96+
}
97+
}
98+
8799
/// Convenience alias for `Result<T, DbError>`.
88100
pub type Result<T> = std::result::Result<T, DbError>;

‎crates/buzz-db/src/runtime/migration.rs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -767,6 +767,11 @@ mod postgres_tests {
767767
.as_str()
768768
.contains("CREATE INDEX idx_events_tags_gin"));
769769
assert!(!migrations[0].sql.as_str().contains("idx_events_tags_gin"));
770+
// schema.sql (CI / isolated relay bootstrap) must carry the same index,
771+
// or e-tag reads there run on plans prod never sees.
772+
assert!(include_str!("../../../../schema/schema.sql").contains(
773+
"CREATE INDEX idx_events_tags_gin ON events USING GIN (tags jsonb_path_ops)"
774+
));
770775

771776
// NIP-AM (kind 44200) FTS exclusion: additive migration, never folded
772777
// into 0001 — folding would change 0001's checksum and break brownfield

‎crates/buzz-db/src/runtime/mod.rs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,8 @@ impl ReadSession {
201201
///
202202
/// If the proved replica transaction fails mid-request, the session
203203
/// permanently degrades to the writer and the query is re-run there.
204+
/// Exception: a statement cancelled by `statement_timeout` (57014) is
205+
/// returned as-is, since the writer would run the same slow statement.
204206
/// The writer is always at or ahead of any replica replay position, so
205207
/// the degraded follow-up can only observe *more* than the proof-time
206208
/// snapshot, never less — fresher aux rows, the same failure semantics
@@ -211,6 +213,9 @@ impl ReadSession {
211213
ReadSessionInner::Replica { tx, writer } => {
212214
match event::query_events_on(tx, q).await {
213215
Ok(rows) => return Ok(rows),
216+
// A cancelled statement (timeout) would be just as slow on
217+
// the writer; surface it instead of doubling the cost.
218+
Err(e) if e.is_statement_cancelled() => return Err(e),
214219
Err(e) => {
215220
tracing::warn!(
216221
error = %e,

‎crates/buzz-db/src/runtime/tests.rs‎

Lines changed: 297 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3313,5 +3313,302 @@ async fn floor_guard_blocks_updates_that_move_rows_below_the_fence() {
33133313
drop_scratch_db(&admin, pool, &name).await;
33143314
}
33153315

3316+
// ---- Statement-deadline cancellation vs. replica fallback ----------------
3317+
//
3318+
// A replica read cancelled by `statement_timeout` (SQLSTATE 57014) must be
3319+
// surfaced, not re-run on the writer: the writer would be just as slow and
3320+
// the relay maps it to a distinct non-retried "query timed out". Every other
3321+
// replica failure keeps failing closed to the writer.
3322+
3323+
/// Writer and replica scratch DBs holding one root plus an `e`-tagged reply
3324+
/// that mentions `mentioned` — the writer copy says "writer-reply" so a
3325+
/// writer re-run is observable. Returns the reply-targeted query shape.
3326+
struct EtagFixture {
3327+
admin: PgPool,
3328+
writer: PgPool,
3329+
wname: String,
3330+
replica: PgPool,
3331+
rname: String,
3332+
cid: CommunityId,
3333+
root_hex: String,
3334+
mentioned_hex: String,
3335+
}
3336+
3337+
async fn e_tag_fixture(prefix: &str) -> EtagFixture {
3338+
let admin = PgPool::connect(&admin_url().await)
3339+
.await
3340+
.expect("connect admin");
3341+
let (writer, wname) = create_scratch_db(&admin, &format!("{prefix}_w")).await;
3342+
let (replica, rname) = create_scratch_db(&admin, &format!("{prefix}_r")).await;
3343+
let author = nostr::Keys::generate();
3344+
let mentioned = nostr::Keys::generate();
3345+
let community = Uuid::new_v4();
3346+
let channel = Uuid::new_v4();
3347+
let base = 1_700_000_000u64;
3348+
let root = signed_event_at(&author, "root", base);
3349+
for (pool, content) in [(&writer, "writer-reply"), (&replica, "replica-reply")] {
3350+
seed_community_channel(pool, community, channel, &author).await;
3351+
insert_top_level(pool, community, channel, &root).await;
3352+
let reply = nostr::EventBuilder::new(nostr::Kind::Custom(9), content)
3353+
.tags([
3354+
nostr::Tag::event(root.id),
3355+
nostr::Tag::public_key(mentioned.public_key()),
3356+
])
3357+
.custom_created_at(nostr::Timestamp::from(base + 10))
3358+
.sign_with_keys(&author)
3359+
.expect("sign reply");
3360+
insert_top_level(pool, community, channel, &reply).await;
3361+
crate::insert_mentions(
3362+
pool,
3363+
CommunityId::from_uuid(community),
3364+
&reply,
3365+
Some(channel),
3366+
)
3367+
.await
3368+
.expect("insert mentions");
3369+
}
3370+
EtagFixture {
3371+
admin,
3372+
writer,
3373+
wname,
3374+
replica,
3375+
rname,
3376+
cid: CommunityId::from_uuid(community),
3377+
root_hex: root.id.to_hex(),
3378+
mentioned_hex: mentioned.public_key().to_hex(),
3379+
}
3380+
}
3381+
3382+
impl EtagFixture {
3383+
fn e_tag_query(&self) -> EventQuery {
3384+
let mut q = EventQuery::for_community(self.cid);
3385+
q.e_tags = Some(vec![self.root_hex.clone(), "00".repeat(32)]);
3386+
q
3387+
}
3388+
3389+
/// A routed `Db` whose reader pool connects fresh (so it picks up any
3390+
/// `ALTER DATABASE` settings made on the replica), with the fence open
3391+
/// and a bounded budget so unpinned reads route to the replica.
3392+
async fn routed_db(&self) -> Db {
3393+
let base = admin_url().await;
3394+
let url = format!("{}/{}", &base[..base.rfind('/').expect("path")], self.rname);
3395+
let reader = PgPool::connect(&url).await.expect("connect reader");
3396+
let mut db = Db::from_pools(self.writer.clone(), reader);
3397+
db.fence().force_open_for_tests(chrono::Utc::now());
3398+
db.set_replica_read_max_age_for_tests(Some(std::time::Duration::from_secs(5)));
3399+
db
3400+
}
3401+
3402+
async fn drop(self) {
3403+
drop_scratch_db(&self.admin, self.replica, &self.rname).await;
3404+
drop_scratch_db(&self.admin, self.writer, &self.wname).await;
3405+
}
3406+
}
3407+
3408+
fn contents(rows: &[StoredEvent]) -> Vec<&str> {
3409+
rows.iter().map(|e| e.event.content.as_str()).collect()
3410+
}
3411+
3412+
#[tokio::test]
3413+
#[ignore = "requires Postgres"]
3414+
async fn replica_statement_cancel_propagates_without_writer_rerun() {
3415+
let fx = e_tag_fixture("cxl").await;
3416+
// A short operator cap on the replica (kept, not raised, by the e-tag
3417+
// deadline), plus a lock that parks every `events` read past it: the
3418+
// heartbeat proof still succeeds, then the routed read is cancelled.
3419+
sqlx::query(sqlx::AssertSqlSafe(format!(
3420+
"ALTER DATABASE {} SET statement_timeout = '300ms'",
3421+
fx.rname
3422+
)))
3423+
.execute(&fx.admin)
3424+
.await
3425+
.expect("set replica statement_timeout");
3426+
let db = fx.routed_db().await;
3427+
let q = fx.e_tag_query();
3428+
3429+
// Healthy control: the replica serves before it is locked.
3430+
let rows = db.query_events_routed("t_cxl", &q).await.expect("healthy");
3431+
assert!(
3432+
contents(&rows).contains(&"replica-reply"),
3433+
"fixture must route to replica"
3434+
);
3435+
3436+
let mut locker = fx.replica.begin().await.expect("begin locker");
3437+
sqlx::query("LOCK TABLE events IN ACCESS EXCLUSIVE MODE")
3438+
.execute(&mut *locker)
3439+
.await
3440+
.expect("lock replica events");
3441+
3442+
let cancelled = |r: crate::Result<()>, what: &str| match r {
3443+
Err(e) => assert!(e.is_statement_cancelled(), "{what}: want 57014, got {e}"),
3444+
Ok(()) => panic!("{what}: replica cancel was re-run on the writer"),
3445+
};
3446+
cancelled(
3447+
db.query_events_routed("t_cxl", &q).await.map(drop),
3448+
"query_events_routed",
3449+
);
3450+
cancelled(
3451+
db.query_events_routed_bounded("t_cxl", &q).await.map(drop),
3452+
"query_events_routed_bounded",
3453+
);
3454+
cancelled(
3455+
db.count_events_routed("t_cxl", &q).await.map(drop),
3456+
"count_events_routed",
3457+
);
3458+
let root_id = hex::decode(&fx.root_hex).expect("hex");
3459+
cancelled(
3460+
db.get_events_by_ids_routed("t_cxl", fx.cid, &[root_id.as_slice()])
3461+
.await
3462+
.map(drop),
3463+
"get_events_by_ids_routed",
3464+
);
3465+
3466+
locker.rollback().await.expect("unlock");
3467+
drop(db);
3468+
fx.drop().await;
3469+
}
3470+
3471+
/// [`replica_statement_cancel_propagates_without_writer_rerun`] for the
3472+
/// request-scoped [`ReadSession`]: a cancelled aux read inside the proved
3473+
/// replica transaction surfaces 57014 and does not degrade to the writer.
3474+
#[tokio::test]
3475+
#[ignore = "requires Postgres"]
3476+
async fn read_session_statement_cancel_propagates_without_writer_rerun() {
3477+
let fx = e_tag_fixture("scxl").await;
3478+
let db = fx.routed_db().await;
3479+
let reader_tx = db
3480+
.read_pool
3481+
.as_ref()
3482+
.expect("reader pool")
3483+
.begin()
3484+
.await
3485+
.expect("begin reader tx");
3486+
let mut session = ReadSession {
3487+
inner: ReadSessionInner::Replica {
3488+
tx: reader_tx,
3489+
writer: fx.writer.clone(),
3490+
},
3491+
};
3492+
let rows = session
3493+
.query_events(&fx.e_tag_query())
3494+
.await
3495+
.expect("healthy");
3496+
assert_eq!(contents(&rows), vec!["replica-reply"]);
3497+
3498+
// Force the next statement in the session over a short cap.
3499+
let ReadSessionInner::Replica { tx, .. } = &mut session.inner else {
3500+
unreachable!()
3501+
};
3502+
sqlx::query("SET LOCAL statement_timeout = '50ms'")
3503+
.execute(&mut **tx)
3504+
.await
3505+
.expect("cap session");
3506+
let mut locker = fx.replica.begin().await.expect("begin locker");
3507+
sqlx::query("LOCK TABLE events IN ACCESS EXCLUSIVE MODE")
3508+
.execute(&mut *locker)
3509+
.await
3510+
.expect("lock replica events");
3511+
3512+
match session.query_events(&fx.e_tag_query()).await {
3513+
Err(e) => assert!(e.is_statement_cancelled(), "want 57014, got {e}"),
3514+
Ok(rows) => panic!("cancel re-ran on the writer: {:?}", contents(&rows)),
3515+
}
3516+
assert!(
3517+
session.is_replica(),
3518+
"a cancel must not degrade the session"
3519+
);
3520+
3521+
locker.rollback().await.expect("unlock");
3522+
drop(session);
3523+
drop(db);
3524+
fx.drop().await;
3525+
}
3526+
3527+
#[tokio::test]
3528+
#[ignore = "requires Postgres"]
3529+
async fn replica_non_cancel_failure_still_falls_back_to_writer() {
3530+
let fx = e_tag_fixture("nfb").await;
3531+
let db = fx.routed_db().await;
3532+
let q = fx.e_tag_query();
3533+
let rows = db.query_events_routed("t_nfb", &q).await.expect("healthy");
3534+
assert!(
3535+
contents(&rows).contains(&"replica-reply"),
3536+
"fixture must route to replica"
3537+
);
3538+
3539+
// Break the replica after the proof point (heartbeat table intact).
3540+
sqlx::query("DROP TABLE events CASCADE")
3541+
.execute(&fx.replica)
3542+
.await
3543+
.expect("drop replica events");
3544+
3545+
for rows in [
3546+
db.query_events_routed("t_nfb", &q).await,
3547+
db.query_events_routed_bounded("t_nfb", &q).await,
3548+
] {
3549+
let rows = rows.expect("non-cancel failure must fall back");
3550+
assert!(
3551+
contents(&rows).contains(&"writer-reply"),
3552+
"{:?}",
3553+
contents(&rows)
3554+
);
3555+
}
3556+
assert_eq!(
3557+
db.count_events_routed("t_nfb", &q)
3558+
.await
3559+
.expect("count falls back"),
3560+
1
3561+
);
3562+
3563+
drop(db);
3564+
fx.drop().await;
3565+
}
3566+
3567+
/// The `#p`-joined SELECT (fenced, `e.` column prefix) and the COUNT `ANY`
3568+
/// path, executed with real bound values rather than only rendered SQL.
3569+
#[tokio::test]
3570+
#[ignore = "requires Postgres"]
3571+
async fn e_tag_any_runs_with_real_binds_on_p_join_and_count() {
3572+
let fx = e_tag_fixture("pjn").await;
3573+
let mut q = fx.e_tag_query();
3574+
3575+
let rows = event::query_events(&fx.writer, &q)
3576+
.await
3577+
.expect("e-tag select");
3578+
assert_eq!(contents(&rows), vec!["writer-reply"]);
3579+
assert_eq!(
3580+
event::count_events(&fx.writer, &q)
3581+
.await
3582+
.expect("e-tag count"),
3583+
1
3584+
);
3585+
3586+
q.p_tag_hex = Some(fx.mentioned_hex.clone());
3587+
let rows = event::query_events(&fx.writer, &q)
3588+
.await
3589+
.expect("#p + e-tag select");
3590+
assert_eq!(contents(&rows), vec!["writer-reply"]);
3591+
assert_eq!(
3592+
event::count_events(&fx.writer, &q)
3593+
.await
3594+
.expect("#p + e-tag count"),
3595+
1
3596+
);
3597+
3598+
q.p_tag_hex = Some("11".repeat(32));
3599+
assert!(event::query_events(&fx.writer, &q)
3600+
.await
3601+
.expect("miss")
3602+
.is_empty());
3603+
assert_eq!(
3604+
event::count_events(&fx.writer, &q)
3605+
.await
3606+
.expect("miss count"),
3607+
0
3608+
);
3609+
3610+
fx.drop().await;
3611+
}
3612+
33163613
#[path = "tests/thread_window_postgres_tests.rs"]
33173614
mod thread_window_postgres_tests;

0 commit comments

Comments
 (0)