Skip to content

perf(loadgen): spread seed writes across channel creators - #9206

Merged
TheSentinel454 merged 6 commits into
mainfrom
elrond/loadgen-channel-creators
Nov 6, 2026
Merged

TheSentinel454 merged 6 commits into
mainfrom
elrond/loadgen-channel-creators

Conversation

@TheSentinel454

@TheSentinel454 TheSentinel454 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What

buzz-loadgen seed signed every channel create, member add, and thread root with the community owner. In private channels without human members, the owner also signed all the history. The relay limits HTTP calls per pubkey (default 300 a minute), so a community with thousands of channels can't be seeded against a relay that keeps its default limits. The seeder doesn't retry a 429, so the seed fails partway, and a torn-down run_id can't be reused.

Take one community with 20,000 private channels: the owner alone signs more than 40,000 writes, which is over two hours of its quota. Human authors had the same problem on a smaller scale. A human in hundreds of consecutive private channels signed hundreds of history posts within seconds.

Now the community owner creates its first 8 channels, and each further block of 8 gets its own creator identity (Role::Creator, derived from run_id). A creator signs its channels' kind:9007 and kind:9000 events and their thread roots, and run looks roots up by the creator. That keeps every creator well under one minute of quota, whatever the community's channel count.

History and replies still rotate across humans: a private channel's own human members, or its creator when it has none. Because only humans look roots up, the creator signs nothing but roots in their channels. No amount of history or replies can push a root out of run's newest-100 lookup.

Human signers can still bunch up, for example a human in hundreds of consecutive channels or a deep thread. To cover that, a seed write the relay throttles now waits out the retry in <n>s window the relay names and tries again, up to 10 times, before it counts as failed.

Compatibility: a community with up to 8 channels is seeded as before, apart from the retries. An existing fixture with that shape keeps working with run, including its root lookups.

What got simpler

Nothing got simpler. This adds a creator role and a retry loop, which is the smallest change that makes large fixtures seedable against default limits.

Validation

  • cargo fmt --check, cargo clippy -p buzz-test-client --all-targets -D warnings, and cargo test -p buzz-test-client --lib loadgen, run on a Linux workstation at the head commit.
  • no_channel_creator_signs_more_seed_writes_than_one_minute_of_its_quota seeds a private community of 400 channels against a fake relay. The community has wide channels, a tier of humans in 280 channels each, and runs at 2x. The test requires every creator to sign fewer than 300 writes. With the owner signing everything, it fails: the owner signs all 7,212 writes.
  • throttled_writes_wait_out_the_quota_and_a_lasting_quota_fails_the_seed: when every write gets two 429s it is retried and the seed passes. When the quota never clears, each write is tried 11 times and the seed fails with the quota error.
  • private_channels_add_every_member_before_anything_is_posted now uses 20 channels. It checks that every create, member add and root is signed by its channel's creator, that the owner keeps channels 0–7, and that every other post is by a human member.
  • root_lookups_search_the_channel_creators_posts checks that run searches for roots by the owner in a community's first block and by the block's creator after it, across two communities.

🤖 Generated with Claude Code

The relay limits HTTP calls per pubkey, so a community with thousands of
channels cannot be seeded when one owner signs every channel create,
member add and thread root (and, in private channels, the history of
every channel without human members). A tier human in hundreds of
consecutive private channels also signed hundreds of history posts
within seconds.

The community owner now creates its first 8 channels and each further
block of 8 gets its own derived creator identity. A creator signs its
channels' kind:9007, kind:9000 and thread roots, and in private channels
their history and replies; `run` looks roots up by it. Communities of up
to 8 channels are seeded exactly as before.

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
@TheSentinel454
TheSentinel454 marked this pull request as ready for review November 6, 2026 18:45
@TheSentinel454
TheSentinel454 requested a review from a team as a code owner November 6, 2026 18:45
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

  • Exact PR diff: a25f3cd1d7942f9ec666910471232b8193774496...34e2e653d72357c1ed7314e1ed78c84ad68935e6
  • Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.


Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability findings were identified in the authorized PR range.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

  • Review used read-only source and history inspection. Builds and tests were not executed, as instructed.

Generated by Codex Security Review |
Requested by: @TheSentinel454 |
Workflow run

Review follow-up. Private-channel history and replies go back to the
channel's human members (the creator only when it has none), so in a
channel humans read, the creator signs nothing but roots and no volume
of posts crowds them out of run's newest-100 root lookup. A seed write
the relay throttles now waits out the window it names and retries, up to
10 times, so a signer whose writes bunch up (a human in hundreds of
consecutive channels, a deep thread) slows down instead of failing.

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
…-signed per try

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approving. The creator blocks keep every signer's create, member-add and root writes well under the per-pubkey HTTP quota, the 429 retry parses the relay's retry in <n>s body correctly, and root lookups in run follow the same channel_creator mapping as the seeder. Communities with 8 or fewer channels (including the seeded bb-public staging p90 fixture) keep the owner as signer and root author, so they're unaffected.

Nonblocking:

  • As the Codex review notes, a private fixture seeded before this change with more than 8 channels can't be reseeded or have its roots found, because its channels belong to the owner rather than the new creators. A line in perf/LOADGEN.md saying such fixtures need a fresh run_id would save someone a confusing failure.
  • Retry sleeps are counted inside the seed write's latency, and retried 429s never reach the throttle stats, so a throttled seed reports slow writes with 0 throttles.

A private fixture seeded before channel creators keeps its old owner on
channels past the first 8, so reseeding it fails and run loses those
roots. Bump the local-desktop run_id and document the fresh run_id.

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
A throttled seed post retried by paced never reached the stats, and its
latency included the quota wait, so a throttled seed reported slow writes
and 0 throttles. Count each retried rejection as a throttled seed write and
time only the accepted attempt.

Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Luke Tornquist <[email protected]>
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
@github-actions github-actions Bot added codex-security-review-current The posted Codex security review matches its recorded range. and removed codex-security-review-current The posted Codex security review matches its recorded range. labels Nov 6, 2026
@TheSentinel454
TheSentinel454 enabled auto-merge (squash) November 6, 2026 19:33
@TheSentinel454
TheSentinel454 merged commit e670b67 into main Nov 6, 2026
91 checks passed
@TheSentinel454
TheSentinel454 deleted the elrond/loadgen-channel-creators branch November 6, 2026 19:48
bradseiler added a commit that referenced this pull request Nov 6, 2026
* origin/main:
  fix(relay): set received_at in the system-channel fan-out test helper (#9214)
  feat(mobile): move Activity to a Home bell and align Home spacing (#9212)
  Allow operators to set initial NIP-FI policy on create (#9211)
  perf(loadgen): spread seed writes across channel creators (#9206)
  feat(relay): runtime-tunable history-read and AUTH concurrency caps via feature flags (#9170)

Signed-off-by: Brad Seiler <[email protected]>

This branch was previously deployed

1 inactive deployment
codex-review — 34e2e653 Deployed Nov 6, 2026 by TheSentinel454 via Run Codex Security Review #13585
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-security-review-current The posted Codex security review matches its recorded range.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants