Repository navigation
perf(loadgen): spread seed writes across channel creators - #9206
Conversation
The relay limits HTTP calls per pubkey, so a community with thousands of channels cannot be seeded when one owner signs every channel create, member add and thread root (and, in private channels, the history of every channel without human members). A tier human in hundreds of consecutive private channels also signed hundreds of history posts within seconds. The community owner now creates its first 8 channels and each further block of 8 gets its own derived creator identity. A creator signs its channels' kind:9007, kind:9000 and thread roots, and in private channels their history and replies; `run` looks roots up by it. Communities of up to 8 channels are seeded exactly as before. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
🔐 Codex Security Review
Review SummaryOverall Risk: NONE
FindingsNo concrete security, correctness, or reliability findings were identified. Notes
Generated by Codex Security Review | |
Review follow-up. Private-channel history and replies go back to the channel's human members (the creator only when it has none), so in a channel humans read, the creator signs nothing but roots and no volume of posts crowds them out of run's newest-100 root lookup. A seed write the relay throttles now waits out the window it names and retries, up to 10 times, so a signer whose writes bunch up (a human in hundreds of consecutive channels, a deep thread) slows down instead of failing. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
…-signed per try Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
wpfleger96
left a comment
There was a problem hiding this comment.
🤖 Approving. The creator blocks keep every signer's create, member-add and root writes well under the per-pubkey HTTP quota, the 429 retry parses the relay's retry in <n>s body correctly, and root lookups in run follow the same channel_creator mapping as the seeder. Communities with 8 or fewer channels (including the seeded bb-public staging p90 fixture) keep the owner as signer and root author, so they're unaffected.
Nonblocking:
- As the Codex review notes, a private fixture seeded before this change with more than 8 channels can't be reseeded or have its roots found, because its channels belong to the owner rather than the new creators. A line in
perf/LOADGEN.mdsaying such fixtures need a freshrun_idwould save someone a confusing failure. - Retry sleeps are counted inside the seed write's latency, and retried 429s never reach the throttle stats, so a throttled seed reports slow writes with 0 throttles.
A private fixture seeded before channel creators keeps its old owner on channels past the first 8, so reseeding it fails and run loses those roots. Bump the local-desktop run_id and document the fresh run_id. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
A throttled seed post retried by paced never reached the stats, and its latency included the quota wait, so a throttled seed reported slow writes and 0 throttles. Count each retried rejection as a throttled seed write and time only the accepted attempt. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
* origin/main: fix(relay): set received_at in the system-channel fan-out test helper (#9214) feat(mobile): move Activity to a Home bell and align Home spacing (#9212) Allow operators to set initial NIP-FI policy on create (#9211) perf(loadgen): spread seed writes across channel creators (#9206) feat(relay): runtime-tunable history-read and AUTH concurrency caps via feature flags (#9170) Signed-off-by: Brad Seiler <[email protected]>
What
buzz-loadgen seedsigned every channel create, member add, and thread root with the community owner. In private channels without human members, the owner also signed all the history. The relay limits HTTP calls per pubkey (default 300 a minute), so a community with thousands of channels can't be seeded against a relay that keeps its default limits. The seeder doesn't retry a 429, so the seed fails partway, and a torn-downrun_idcan't be reused.Take one community with 20,000 private channels: the owner alone signs more than 40,000 writes, which is over two hours of its quota. Human authors had the same problem on a smaller scale. A human in hundreds of consecutive private channels signed hundreds of history posts within seconds.
Now the community owner creates its first 8 channels, and each further block of 8 gets its own creator identity (
Role::Creator, derived fromrun_id). A creator signs its channels' kind:9007 and kind:9000 events and their thread roots, andrunlooks roots up by the creator. That keeps every creator well under one minute of quota, whatever the community's channel count.History and replies still rotate across humans: a private channel's own human members, or its creator when it has none. Because only humans look roots up, the creator signs nothing but roots in their channels. No amount of history or replies can push a root out of
run's newest-100 lookup.Human signers can still bunch up, for example a human in hundreds of consecutive channels or a deep thread. To cover that, a seed write the relay throttles now waits out the
retry in <n>swindow the relay names and tries again, up to 10 times, before it counts as failed.Compatibility: a community with up to 8 channels is seeded as before, apart from the retries. An existing fixture with that shape keeps working with
run, including its root lookups.What got simpler
Nothing got simpler. This adds a creator role and a retry loop, which is the smallest change that makes large fixtures seedable against default limits.
Validation
cargo fmt --check,cargo clippy -p buzz-test-client --all-targets -D warnings, andcargo test -p buzz-test-client --lib loadgen, run on a Linux workstation at the head commit.no_channel_creator_signs_more_seed_writes_than_one_minute_of_its_quotaseeds a private community of 400 channels against a fake relay. The community has wide channels, a tier of humans in 280 channels each, and runs at 2x. The test requires every creator to sign fewer than 300 writes. With the owner signing everything, it fails: the owner signs all 7,212 writes.throttled_writes_wait_out_the_quota_and_a_lasting_quota_fails_the_seed: when every write gets two 429s it is retried and the seed passes. When the quota never clears, each write is tried 11 times and the seed fails with the quota error.private_channels_add_every_member_before_anything_is_postednow uses 20 channels. It checks that every create, member add and root is signed by its channel's creator, that the owner keeps channels 0–7, and that every other post is by a human member.root_lookups_search_the_channel_creators_postschecks thatrunsearches for roots by the owner in a community's first block and by the block's creator after it, across two communities.🤖 Generated with Claude Code