Skip to content

perf(db): plan multi-value #d reads with their values - #9131

Merged
TheSentinel454 merged 1 commit into
mainfrom
elrond/dtag-custom-plan
Nov 6, 2026
Merged

TheSentinel454 merged 1 commit into
mainfrom
elrond/dtag-custom-plan

Conversation

@TheSentinel454

Copy link
Copy Markdown
Contributor

Summary

A REQ with several #d values for a NIP-33 kind runs as a named, cached statement. A client resolving channel metadata for up to 128 channels sends this kind of REQ. After a few runs on a connection, Postgres can switch the statement to a generic plan, which ignores the bound values:

  • The d match stops being an index condition on idx_events_parameterized.
  • The generic plan instead walks the author's rows in created_at order and filters d_tag row by row.
  • The LIMIT is one row per requested value, so the walk never stops early. It reads every stored version of the kind, not one probe per value.

This change makes three fixes:

  • Multi-value #d reads now run as unnamed statements, as tag-value matches already do (plans_with_bound_values), so Postgres plans every call with its values. This covers both the page SELECT and the COUNT.
  • The values are bound as one array (d_tag = ANY($n)) instead of one bind per value. The SQL text no longer changes with the number of values, which matches how channel scope is bound.
  • The page SELECT and the COUNT now share one push_d_tag_filter helper instead of two copies of the d predicate.

What got simpler:

  • One helper replaces the duplicated d predicate blocks.
  • A #d read no longer produces a different statement for every value count.

Related issue

N/A. None found.

Testing

  • New unit test d_tag_sql_does_not_depend_on_value_count: the SQL text is identical for 2 and 128 values, for both the page query and the count query.
  • Postgres test value_matched_queries_are_not_cached_as_prepared_statements, extended from the tag-value test. It runs each #d page and count query 6 times and asserts no d_tag = ANY( statement is left in pg_prepared_statements. It also checks the page and count results.
  • Mutants, each caught by the matching test:
    • dropping d_tags from plans_with_bound_values fails the Postgres test;
    • going back to a per-value IN (...) list fails the unit test.
  • Gates run at 5f71e71:
    • cargo fmt --check
    • cargo clippy -p buzz-db -p buzz-relay --all-targets -D warnings
    • buzz-db unit tests: 215 passed
    • scripts/postgres-test-run.sh -p buzz-relay -p buzz-db --lib --tests: 1291 passed
    • scripts/test-postgres-test-discovery.sh
  • Local EXPLAIN ANALYZE on synthetic data (50k versions of one author's kind 39000 rows, 128 #d values):
    • With force_custom_plan, d_tag = ANY(...) is in the Index Cond on every partition.
    • With force_generic_plan, the partition holding the rows drops d_tag to a row filter.
    • On larger data the generic plan can switch to an ordered created_at walk.

🤖 Generated with Claude Code

@TheSentinel454
TheSentinel454 marked this pull request as ready for review November 5, 2026 19:54
@TheSentinel454
TheSentinel454 requested a review from a team as a code owner November 5, 2026 19:54
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

  • Exact PR diff: c39ee507b27354b34baec635c8835c7ebe6f1de0...49bb40fef8fc99b405c55a206a853f00f29cab3c
  • Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.


Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability findings in the authorized PR range. The d-tag binding changes preserve existing filtering and community scoping across event queries and counts.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

  • Read-only source review; tests and database query plans were not executed.

Generated by Codex Security Review |
Requested by: @TheSentinel454 |
Workflow run

@TheSentinel454
TheSentinel454 force-pushed the elrond/dtag-custom-plan branch from 5f71e71 to 28d9f34 Compare November 5, 2026 23:52
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 5, 2026
A REQ with several `#d` values for NIP-33 kinds (a client resolving channel
metadata for up to 128 channels, say) ran as a named, cached statement. After
a few runs on a connection, Postgres can switch it to a generic plan. That plan
walks the author's `created_at` index and filters `d_tag` row by row. Its LIMIT
(one row per value) is never reached early, so it reads every stored version
of the kind instead of probing `idx_events_parameterized` once per value.

Multi-value `#d` reads now run as unnamed statements, as tag-value matches
already do, so every call is planned with its values. The values are also
bound as one array (`d_tag = ANY($n)`), so the number of values no longer
changes the SQL text. The page SELECT and the COUNT share one helper for the
`d` predicate instead of two copies.

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
@TheSentinel454
TheSentinel454 force-pushed the elrond/dtag-custom-plan branch from 28d9f34 to 49bb40f Compare November 6, 2026 00:00

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 LGTM. Every read path that takes d_tags (page SELECT, COUNT, thread replies, which were already unnamed) now plans with its values, and d_tag = ANY($n) on the TEXT column stays an index condition on idx_events_parameterized. The shared push_d_tag_filter removes the duplicated predicate, and both tests pin the behavior that matters.

@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
@TheSentinel454
TheSentinel454 merged commit bd1b17d into main Nov 6, 2026
90 checks passed
@TheSentinel454
TheSentinel454 deleted the elrond/dtag-custom-plan branch November 6, 2026 00:41
tellaho added a commit that referenced this pull request Nov 6, 2026
* origin/main: (54 commits)
  release(mobile): push candidate tags as the operator, not the App (#9084)
  test(mobile): check posterless video previews renew the relay assertion (#9144)
  feat(relay): log WebSocket EVENT writes on the access target (#9154)
  perf(db): plan multi-value #d reads with their values (#9131)
  fix(mobile): confirm before starting a Huddle from the header (#9155)
  feat(cli): send NIP-FI evidence via credential chain and Buzz desktop (#8999)
  fix(mobile): find verified people who have no profile (#9086)
  feat(protocol): define compact membership discovery (BUZZ-615) (#9117)
  perf(db): hydrate search hits with exact event timestamps (#9126)
  perf(db): prune post-commit push enqueue by signed event timestamp (#9128)
  perf(db): use exact event timestamps for transactional deletion (#9130)
  perf(redis): index mesh ready records and share one Redis connection (#9113)
  fix(mobile): delay iOS message actions until a deliberate hold (#9049)
  feat(cli): user status expiry and status read (#8777)
  feat(cli): send a thread message to the channel (#8778)
  feat(relay): add kind 30180 custom emoji entry (#8783)
  feat(cli): read-only unread state (NIP-RS) (#8789)
  feat(cli): report a message to relay moderators (#8829)
  buzz-db: read explicit #h sets one channel at a time (#9110)
  feat(cli): list huddles and their participants (#8830)
  ...

Signed-off-by: Taylor Ho <[email protected]>

This branch was previously deployed

1 inactive deployment
codex-review — 49bb40fe Deployed Nov 6, 2026 by TheSentinel454 via Run Codex Security Review #13257
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-security-review-current The posted Codex security review matches its recorded range.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants