You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 14a752f
Browse filesBrowse the repository at this point in the historyBrowse files
authored
fix(db): audit partition catalog before creation (#6515)
## Why
The partition manager checks partition existence by name only and treats
Postgres 42P17 "would overlap partition" errors as success. On databases
where a catch-all partition (`events_p_future` /
`delivery_log_p_future`) covers future months, the monthly creation path
silently does nothing while real coverage erodes toward a write outage.
There is currently no visibility into partition catalog health and no
signal before writes start failing.
## What
- **Read-only catalog audit** in `buzz-db` (`partition.rs`): classifies
the live partition catalog for `events` and `delivery_log` via
`pg_inherits` and bound parsing — coverage bounds, catch-all detection,
anomalous children, and trigger parity checked recursively down to
routable leaf partitions.
- **Creation gated on the audit verdict**: the 42P17-to-success swallow
is removed; monthly CREATE runs only for months the audit proves
uncovered, a canonical-name mismatch is an error, and per-month outcome
capture replaces first-error abort. New
`BUZZ_PARTITION_MANAGER_CREATE_ENABLED` kill switch (default `true`).
- **Prometheus metrics** per table: `buzz_partition_serving_safe`,
`buzz_partition_uncovered_months` (audit window is now through +3
months), `buzz_partition_catch_all_covered_months`,
`buzz_partition_anomalous_children`,
`buzz_partition_trigger_parity_missing`/`_extra`,
`buzz_partition_audit_duration_seconds`,
`buzz_partition_audit_runs_total{outcome}`,
`buzz_partition_audit_failures_total`,
`buzz_partition_create_attempts_total`, and
`buzz_partition_audit_last_success_timestamp_seconds` for staleness
alerting.
- **Cached diagnostics**: `/_status.partition_catalog` reports partition
serving-safety, audit age, and freshness. `/_readiness` stays
process-local, consistent with #7341; missing, unsafe, or stale audits
do not remove pods from service.
- **Periodic audit task** in the relay, plus config surface for the
audit interval and the monthly-create kill switch. Catch-alls are
detected from catalog bounds rather than configured names.
- Doc correction: the mesh heartbeat comment no longer claims readiness
gating (it never was gated; behavior unchanged).
## Behavior changes & risk
- No DETACH/DROP/ATTACH and no schema migrations. The only DDL is the
pre-existing monthly CREATE path, now strictly narrower
(audit-proven-uncovered months only) and behind a kill switch.
- Partition audit health does not gate readiness. Until the first
completed audit, read-only retries back off from 5 seconds to a
60-second cap. Any completed audit, including unsafe coverage, restores
the normal interval.
- Partition-overlap errors now surface as errors instead of being
swallowed as success.
- Rollback: revert the commit, or set
`BUZZ_PARTITION_MANAGER_CREATE_ENABLED=false` to disable the creation
path while keeping the audit and metrics.
## Validation
- New Postgres-backed regression tests for catalog classification and
trigger parity (disabled child trigger, `ENABLE ALWAYS` parent, disabled
nested leaf clone in a multi-level layout, healthy-catalog zero-parity
checks). Each new regression was verified to fail against the previous
logic before the fix.
- Local affected-target validation passed against a disposable
current-schema database, with the known mesh forwarded-echo and tracing
callsite failures excluded. This is not a claim that the full suite
passes.
Follow-up work (separate PRs): bounded catch-all advance (empty-only
golden path; non-empty reattach only behind an explicit operator flag,
default off), fresh-install convergence, and proactive coverage-runway
alerting.
---
## Update — 2026-08-21
- Fixed nested-leaf trigger parity auditing so child-only row triggers
are checked on every routable descendant leaf and reported as
`{leaf}:{trigger}` under the owning root child.
- Added a Postgres regression test for a child-only trigger on a nested
leaf; mutation-reverting to immediate-child-only extra-trigger scanning
makes it fail.
Generated with Codex
Signed-off-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz>
Signed-off-by: Ian Oberst <[email protected]>
Signed-off-by: Ian Oberst <[email protected]>
Co-authored-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz>
Co-authored-by: Codex <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
0 commit comments