Skip to content

fix(identity): report an issuer that keeps rate-limiting an agent as a limit - #2365

Merged
ngthuydiem merged 2 commits into
mainfrom
fizz/agent-grant-limit
Nov 13, 2026
Merged

ngthuydiem merged 2 commits into
mainfrom
fizz/agent-grant-limit

Conversation

@ngthuydiem

@ngthuydiem ngthuydiem commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A local agent enrolls with its owner's organization and mints short-lived assertions before every write. When the issuer answers both of those steps with HTTP 429, the shared retrying helper exhausts its three attempts and reports network-unavailable. The owner then sees "Your organization's identity service is unreachable", even though the issuer answered every request.

An issuer can answer 429 for request throttling, and also for a per-owner cap on enrolled agents. The cap case never clears by waiting, so "unreachable" sends people looking for a network problem. In practice it showed up as some of an owner's agents being unable to post while their other agents worked.

Change

  • protocol::Failure gains a rate_limited flag. It is set when the last answer before the helper gave up was a 429. That covers retry exhaustion and also the 30s deadline firing while waiting on Retry-After.
  • The agent's refused() message reports a rate-limited failure as a limit: "Your organization is limiting this agent's requests; try again later. If it keeps failing, you may have reached your organization's agent limit".
  • The failure code is unchanged (network-unavailable / network-timeout). Human session and assertion flows, and the renderer's transient classification, therefore behave exactly as before. The enrollment reservation still treats the failure as transient.
  • One sentence added to docs/identity.md.

Non-goals: revoking a deleted agent's grant with the issuer, and changing retry policy.

Validation

  • New test an_issuer_that_keeps_rate_limiting_is_not_reported_as_unreachable covers three cases. The enrollment path with 429 ×3 reports the limit. 429, 429, 503 still reports unreachable, because only the last answer counts. The assertion-mint path with 429 ×3 reports the limit. The test fails with the message mapping disabled and passes with it.
  • New test a_retry_after_past_the_deadline_keeps_the_rate_limit covers a 429 with Retry-After: 60 that outlasts the 30s deadline. It fails against the earlier early-return timeout branch and passes now. It passed 20 of 20 repeated runs.
  • cargo test --lib federated_identity: 156 passed (rebased on 51d6395).
  • cargo test --lib -- --skip media_preparation --skip window_state: 1135 passed. On this macOS host, converts_tiled_heic_without_cropping also fails on main (063fccd), and window_state tests segfault headless, so I skipped both.
  • cargo clippy --lib --tests -- -D warnings: clean.
  • An agent review found one blocker: a long Retry-After hit the deadline path. That is fixed, along with its style suggestions.

Not yet done: exercising this in the running app against a live issuer, and a human test. That's why this is a draft.

Follow-up

Deleting a local agent only removes its grant locally, so an issuer-side per-owner cap keeps counting deleted agents. Revoking at deletion needs an issuer revoke route that accepts the adapter session the way the other adapter calls already send it. It also needs handling for deletion while signed out or offline. That belongs in a separate change.

🤖 Generated with Claude Code

@ngthuydiem

Copy link
Copy Markdown
Contributor Author

🤖 Evidence at 5e624cc:

$ cargo test --lib federated_identity::agent::tests::an_issuer_that_keeps_rate_limiting
test federated_identity::agent::tests::an_issuer_that_keeps_rate_limiting_is_not_reported_as_unreachable ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 835 filtered out; finished in 2.35s

@ngthuydiem
ngthuydiem marked this pull request as ready for review November 4, 2026 03:32
@ngthuydiem
ngthuydiem requested a review from a team as a code owner November 4, 2026 03:32

@kazemek21 kazemek21 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kyle reviewed this PR and the Nexus review-tracker notes and approved with the non-blocking comment below.

Posted by Codex (gpt-6.1-sol) on behalf of kylez

},
)
})?
.map_err(|failure| Failure {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve the rate-limit flag on deadline expiry

If a 429 carries Retry-After: 60, the 30-second timeout returns through .map_err(...)? before this mapping runs. Failure::from("network-timeout") therefore retains rate_limited: false, and the agent still reports the issuer as unreachable. Should you also carry the flag through the timeout branch and cover that path with a long-Retry-After case?

Posted by Codex (gpt-6.1-sol) on behalf of kylez

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fixed in 481bebb: the timeout branch now returns Err into the same mapping instead of returning early with ?, so the 429 flag survives the deadline. Added a_retry_after_past_the_deadline_keeps_the_rate_limit (429 with Retry-After: 60, then advance past DEADLINE). It fails against the old code and passes now.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket, Mantis and Groot reviews.

One P2 finding is recorded inline, corroborating the existing reviewer’s deadline-expiry finding. No additional actionable defects found in the five-file change, retry classification, enrollment reservation cleanup or assertion-mint propagation.

Reviewed head 5e624ccd5c2e71a7cd9a03e631bcd63fea18087a, base 08718270346ee60b88ae176ef60845c923bf563e (merge base 063fccd5f94c93bfb538e8271bec8b29383c0b6c). Public description and feature commit inspected; no attached images or concrete public-material privacy finding.

Source-only: no tests, app launches or live issuer requests performed. Hosted checks at this head showed CI required successful; Windows native validation and flake checks were skipped. The added test covers retry exhaustion, not deadline expiry. Runtime error/retry presentation, focus/accessibility and human acceptance remain unverified. This is a COMMENT review, not approval or merge authorization.

)
})?
.map_err(|failure| Failure {
rate_limited: rate_limited.load(std::sync::atomic::Ordering::Relaxed),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 P2 — Preserve the rate-limit flag when the deadline expires. Corroborates the existing finding.

A 429 with Retry-After: 60 makes the 30-second outer timeout expire during the retry sleep. The preceding timeout mapper creates Failure::from("network-timeout") with rate_limited: false, and its ? returns before this new mapper runs. Enrollment and assertion minting therefore still report “Your organization's identity service is unreachable”, despite the documented deadline behavior.

Copy the observed flag into the timeout failure too, or flatten both error paths before applying this annotation. Add a controlled-clock case for 429 plus a Retry-After exceeding the deadline; the current test fixture sends no Retry-After header and only covers exhausted retries.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fixed in 481bebb: the timeout branch now returns Err into the same mapping instead of returning early with ?, so the 429 flag survives the deadline. Added a_retry_after_past_the_deadline_keeps_the_rate_limit (429 with Retry-After: 60, then advance past DEADLINE). It fails against the old code and passes now.

Fizz and others added 2 commits November 13, 2026 10:05
…a limit

Co-authored-by: Diem Nguyen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Fizz <6be7be23ec5bba868f3e1db5ec86013d76bd1a293dce5b435a192158fec13641@buzz.block.builderlab.xyz>
Co-authored-by: Diem Nguyen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Fizz <6be7be23ec5bba868f3e1db5ec86013d76bd1a293dce5b435a192158fec13641@buzz.block.builderlab.xyz>
@ngthuydiem
ngthuydiem force-pushed the fizz/agent-grant-limit branch from 5e624cc to 481bebb Compare November 13, 2026 18:06
@ngthuydiem
ngthuydiem merged commit c1a21a2 into main Nov 13, 2026
46 checks passed
@ngthuydiem
ngthuydiem deleted the fizz/agent-grant-limit branch November 13, 2026 23:50
wpfleger96 added a commit that referenced this pull request Nov 14, 2026
…y-unreads

* origin/main: (24 commits)
  [1/2] feat(agents): add a recommended agent setup (core) (#3344)
  test(browser): survive the live-stream restart a page-clock jump causes (#3442)
  chore(renovate): hold rusqlite below 0.41 for mesh-llm (#3438)
  test(browser): serve the diffs and CLI reload fixtures outside the navigation budget (#3445)
  test(browser): play the draft clip at rate 0 instead of looping it before the Play click (#3446)
  fix(status): reopen a status whose preset deadlines coincide as Today (#3449)
  fix(messages): tolerate a cancelled reaction-name chunk load (#3450)
  fix(pairing): tolerate /proc/net/tcp rows printed twice or missed (#3451)
  test(browser): wait on the signal each test asserts in local-agent-activity and message-navigation (#3444)
  Expose native channel headings to Solo sources (#3523)
  test(browser-host): run the 100,000-channel broker scale check alone in the measurements lane (#3447)
  fix(solo): keep every contributed destination action (#3516)
  fix(identity): report an issuer that keeps rate-limiting an agent as a limit (#2365)
  fix(identity): let a stuck at-launch sign-in copy its link (#2453)
  fix(profiles): open avatar previews on hover in compact density (#3510)
  feat(solo): smooth chat recency reordering (#3502)
  Reject self-mentions that cannot wake another session (#3513)
  feat(status): offer don't clear for every status (#3515)
  Let plugin panels name their tab with panels.useTabTitle (#3251)
  fix(agents): report a conversation wake that fails before its turn starts (#2192)
  ...

# Conflicts:
#	tests/browser/message-management.spec.mjs
wpfleger96 added a commit that referenced this pull request Nov 14, 2026
* origin/main: (27 commits)
  [2/2] feat(agents): offer Automatic (Recommended) in Create agent and Identity (#3345)
  fix(solo): preserve and resume unsent chat drafts (#3525)
  fix(builderlab): give refused app sign-ins and Identity links a route to Organization sign-in (#3078)
  fix(agents): say why Import an agent can't import here, and link Manage plugins (#3073)
  [1/2] feat(agents): add a recommended agent setup (core) (#3344)
  test(browser): survive the live-stream restart a page-clock jump causes (#3442)
  chore(renovate): hold rusqlite below 0.41 for mesh-llm (#3438)
  test(browser): serve the diffs and CLI reload fixtures outside the navigation budget (#3445)
  test(browser): play the draft clip at rate 0 instead of looping it before the Play click (#3446)
  fix(status): reopen a status whose preset deadlines coincide as Today (#3449)
  fix(messages): tolerate a cancelled reaction-name chunk load (#3450)
  fix(pairing): tolerate /proc/net/tcp rows printed twice or missed (#3451)
  test(browser): wait on the signal each test asserts in local-agent-activity and message-navigation (#3444)
  Expose native channel headings to Solo sources (#3523)
  test(browser-host): run the 100,000-channel broker scale check alone in the measurements lane (#3447)
  fix(solo): keep every contributed destination action (#3516)
  fix(identity): report an issuer that keeps rate-limiting an agent as a limit (#2365)
  fix(identity): let a stuck at-launch sign-in copy its link (#2453)
  fix(profiles): open avatar previews on hover in compact density (#3510)
  feat(solo): smooth chat recency reordering (#3502)
  ...

# Conflicts:
#	src-tauri/src/federated_identity/agent/tests.rs
#	src-tauri/src/federated_identity/mod.rs
#	src-tauri/src/federated_identity/protocol.rs
joahg added a commit to joahg/buzz-app that referenced this pull request Nov 14, 2026
* origin/main: (133 commits)
  Expose ordinary chats without workspace headings (block#3530)
  [2/2] feat(agents): offer Automatic (Recommended) in Create agent and Identity (block#3345)
  fix(solo): preserve and resume unsent chat drafts (block#3525)
  fix(builderlab): give refused app sign-ins and Identity links a route to Organization sign-in (block#3078)
  fix(agents): say why Import an agent can't import here, and link Manage plugins (block#3073)
  [1/2] feat(agents): add a recommended agent setup (core) (block#3344)
  test(browser): survive the live-stream restart a page-clock jump causes (block#3442)
  chore(renovate): hold rusqlite below 0.41 for mesh-llm (block#3438)
  test(browser): serve the diffs and CLI reload fixtures outside the navigation budget (block#3445)
  test(browser): play the draft clip at rate 0 instead of looping it before the Play click (block#3446)
  fix(status): reopen a status whose preset deadlines coincide as Today (block#3449)
  fix(messages): tolerate a cancelled reaction-name chunk load (block#3450)
  fix(pairing): tolerate /proc/net/tcp rows printed twice or missed (block#3451)
  test(browser): wait on the signal each test asserts in local-agent-activity and message-navigation (block#3444)
  Expose native channel headings to Solo sources (block#3523)
  test(browser-host): run the 100,000-channel broker scale check alone in the measurements lane (block#3447)
  fix(solo): keep every contributed destination action (block#3516)
  fix(identity): report an issuer that keeps rate-limiting an agent as a limit (block#2365)
  fix(identity): let a stuck at-launch sign-in copy its link (block#2453)
  fix(profiles): open avatar previews on hover in compact density (block#3510)
  ...

Signed-off-by: Joah Gerstenberg <[email protected]>

# Conflicts:
#	src/bundled/codex/prompt.test.ts
#	src/bundled/goose/runtime.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants