You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c99872f
Browse filesBrowse the repository at this point in the historyBrowse files
fix(relay): delete the startup NIP-33 d_tag backfill (#8468)
## Summary
Delete the NIP-33 `d_tag` backfill that ran on every relay boot.
On every boot, before the listener opened, the relay ran:
```sql
SELECT DISTINCT community_id FROM events
WHERE kind BETWEEN 30000 AND 39999 AND d_tag IS NULL
```
No index covers `d_tag IS NULL`. The only `d_tag` index is the partial
`idx_events_parameterized`, which is `WHERE d_tag IS NOT NULL`. So this
query scanned every partition of `events`. In production it averaged
about 11s per boot, max 19s
(`buzz_startup_phase_seconds{phase="d_tag_backfill"}`), and found
nothing.
These rows shouldn't exist anymore:
- The backfill was the one-time migration for #246, the change that
added `d_tag`.
- In `crates/buzz-db/src`, every production write path sets the column
for kinds 30000–39999 through `extract_d_tag`: the main insert,
`replaceable.rs`, `channel_members.rs`, `relay_members.rs`, and
`push.rs`. A new source test,
`every_production_event_insert_names_d_tag`, now enforces that.
- The `events` inserts that omit `d_tag` are all test fixtures.
`scripts/backfill-d-tag.sql` is still there as the manual fallback for
any database that predates #246 and somehow never booted a relay since.
### What got simpler
- Removed `Db::backfill_d_tags` and its admission test.
- Removed the source-shape assertion in `observability_source.rs`.
- Removed the `d_tag_backfill` startup phase.
- `StartupStep::ALL` is now a slice, so removing a step no longer means
editing a hand-kept array length.
### Notes
- The `buzz_startup_phase_seconds{phase="d_tag_backfill"}` series stops
reporting.
- **Rebased onto main after #8467 merged.** Both remove a startup phase.
With the fixed-length `ALL` array the two merged cleanly in git into a
17-entry `[Self; 18]`, which doesn't compile. This PR makes `ALL` a
slice.
- **Before merging**, run this read-only query against production and
confirm it returns `0`: `SELECT count(*) FROM events WHERE kind BETWEEN
30000 AND 39999 AND d_tag IS NULL`. It scans every partition, so use a
read replica. If it isn't `0`, run `scripts/backfill-d-tag.sql` first.
The cutover script copies legacy `d_tag` values as-is, so only this
query proves historical data is clean.
- **Self-hosted installs** should run the same check before upgrading.
### Related issue
None found. Companion to #8467, which also targets relay startup time.
### Testing
- New `every_production_event_insert_names_d_tag`
(`crates/buzz-db/tests/observability_source.rs`): every production
`INSERT INTO events` must name `d_tag` in its column list. Dropping
`d_tag` from `push.rs` makes it fail and name the function.
- Runtime E2E against a real relay: boot has no `d_tag_backfill` phase;
kind:30023 replacement via WebSocket and HTTP keeps only the newer
event, an unrelated `d` survives, and an older replay can't win
(confirmed in SQL).
- Stacked on #8467 at `501dfd83`: a fresh two-replica boot runs all
enabled startup phases, with neither `nip43_reconcile` nor
`d_tag_backfill` present. Both replicas pass readiness and return the
same signed message over authenticated HTTP and WebSocket, and outsiders
are denied.
- Blox, at `501dfd83`: `just fmt-check clippy security-review-check
file-size-check` passes. Every `test-unit` step and `just test` pass
except the `buzz-acp` `git_bootstrap` native-git tests, which fail the
same way at base `b6aa5539` on Blox, and one `buzz-agent` catalog timing
test that fails only under full-suite load and passes when run alone.
- `e27139de` adds only a doc comment on
`every_production_event_insert_names_d_tag`, saying which SQL shapes the
text match covers. No code changed, so the results above still apply.
---------
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
0 commit comments