You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Writer-pointing READ_DATABASE_URL accepted silently; route telemetry reports replica/fresh from the writer #3644
Max's live redteam of #3268 (finding 5, thread cf5deba4 in #buzz-read-only-replica-usage; evidence RESEARCH/PR3268_LIVE_REPLICA_REDTEAM.md): pointing READ_DATABASE_URL at the writer is accepted silently, and route telemetry then reports replica / fresh while debug backend identity shows the writer pod. The freshness fence is trivially satisfied (the "replica" has zero lag because it IS the writer), so nothing is unsafe — but buzz_db_route_decision becomes semantically misleading: operators watching the rollout dashboards would believe replica offload is working when every byte still comes from the writer.
Suggested fix
At boot (or on first routed read), compare backend identity between the writer and reader pools — e.g. pg_control_system().system_identifier + pg_is_in_recovery(), or the existing Aurora identity probe. If the reader connection reports pg_is_in_recovery() = false and matches the writer's identity, either:
log a loud warning and label routes replica_is_writer instead of replica/fresh, or
treat it as a misconfiguration and refuse the reader pool (log + serve all-writer), matching the fail-closed posture everywhere else.
Aurora caveat: cluster-ro legitimately resolves to the writer when the cluster has no readers, so a hard refusal must not crash — degrade to all-writer with a clear log line.
Two things in it deviate from the suggested fix, so flagging them here for whoever picks up the review:
The gate is pg_is_in_recovery() alone, not recovery and an identifier match. system_identifier comes from pg_control_system(), which is superuser-only by default — requiring a match would make the check silently no-op wherever the relay's role isn't a superuser. The identifier is still read when the role permits it and reported in the warning, to separate "the same node as the writer" from "some other primary".
Warn-and-relabel, not refuse-the-reader-pool. Refusing would trade a reporting defect for lost capacity, and it would latch the wrong answer on Aurora, where cluster-ro legitimately resolves to the writer until the cluster has a reader.
One consequence worth calling out: replica_is_writer is a new value on the decision label rather than a new reason, because decision="replica" is what the offload percentage is read off. Dashboards that assume decision ∈ {replica, writer} will need the new value added — nothing in this repo references the metric, so I assume that lives in block-coder-tf-stacks.
Activity
feat(replica): portable heartbeat-token fence with snapshot-local rea…
rshmdev commentedon Jul 31, 2026
Opened #4014 for this.
Two things in it deviate from the suggested fix, so flagging them here for whoever picks up the review:
pg_is_in_recovery()alone, not recovery and an identifier match.system_identifiercomes frompg_control_system(), which is superuser-only by default — requiring a match would make the check silently no-op wherever the relay's role isn't a superuser. The identifier is still read when the role permits it and reported in the warning, to separate "the same node as the writer" from "some other primary".cluster-rolegitimately resolves to the writer until the cluster has a reader.One consequence worth calling out:
replica_is_writeris a new value on thedecisionlabel rather than a newreason, becausedecision="replica"is what the offload percentage is read off. Dashboards that assumedecision ∈ {replica, writer}will need the new value added — nothing in this repo references the metric, so I assume that lives inblock-coder-tf-stacks.feat(replica): portable heartbeat-token fence with snapshot-local rea…
feat(replica): portable heartbeat-token fence with snapshot-local rea…