Skip to content

fix(kernel): cascade parent /stop into agent_send subagents (#3044 follow-up)#3048

Merged
houko merged 7 commits into
mainfrom
fix/hand-cancel-cascade
Apr 24, 2026
Merged

fix(kernel): cascade parent /stop into agent_send subagents (#3044 follow-up)#3048
houko merged 7 commits into
mainfrom
fix/hand-cancel-cascade

Conversation

@houko

@houko houko commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #3047. Addresses the second complaint in issue #3044 — when a parent agent had invoked a hand / subagent via `agent_send` and the user clicked stop, the hand's loop kept running up to `MAX_ITERATIONS=50` more turns, burning tokens that should never have been spent.

Root cause

`agent_loop.rs:2814` checks `opts.interrupt.is_cancelled()` at each iteration boundary, but every new turn allocates a fresh `SessionInterrupt` via `SessionInterrupt::new()` in the kernel dispatch path. The child's cancel flag was completely disconnected from the parent's. `stop_agent_run(parent)` only cancelled the parent's own flag, so the child happily kept iterating.

Design: one-way cancel cascade

`SessionInterrupt` now carries an optional upstream reference:

```rust
pub struct SessionInterrupt {
flag: Arc,
upstream: Option<Arc>,
}
```

  • `is_cancelled()` returns `true` when either the local flag or the upstream is set.
  • `cancel()` only touches the local flag. Upstream is read-only.

Result: parent cancel → child aborts; child cancel → parent unaffected.

Plumbing

  • `SessionInterrupt::new_with_upstream(parent)` — new constructor.
  • `kernel::send_message_as(agent_id, msg, parent_agent_id)` — new public entry that looks up the parent's live `SessionInterrupt` in the existing `session_interrupts` `DashMap` and threads it as upstream into the callee's loop via `execute_llm_agent`.
  • `send_message_full_with_upstream` — inner non-streaming variant that accepts the optional upstream. Existing `send_message_full` delegates with `None`, so all 9 existing call sites are unchanged.
  • `KernelHandle::send_to_agent_as` — new trait method with a default impl that falls back to `send_to_agent` for any implementation that doesn't care about cascade.
  • `tool_agent_send` now routes through `send_to_agent_as` when `caller_agent_id` is available in `ToolExecContext`. System-initiated calls (no caller) keep the legacy path.

Scope

This covers `agent_send`-style synchronous delegation, which is the path the issue reporter's scenario exercises. Hand agents triggered by autonomous events (cron, event subscription) run with no parent in flight and correctly retain their own lifecycle — cascade doesn't apply there.

The streaming entry (`send_message_streaming_with_sender_and_session`) is not touched in this PR. `tool_agent_send` uses the non-streaming `send_to_agent` path, so the fix covers the user-visible bug. A separate follow-up can extend cascade to the streaming path if someone invokes subagents via streaming.

Test plan

  • `cargo test -p librefang-runtime --lib interrupt::` — 11 passed including 4 new cascade tests:
    • `upstream_cancel_cascades_to_child`
    • `child_cancel_does_not_leak_to_upstream`
    • `child_reset_does_not_affect_upstream`
    • `multiple_siblings_share_upstream`
  • `cargo check -p librefang-kernel -p librefang-runtime --lib`
  • `cargo clippy -p librefang-runtime -p librefang-kernel -p librefang-kernel-handle --lib -- -D warnings`
  • `cargo fmt --check`
  • Live integration: parent agent calls a hand via `agent_send`, user clicks stop mid-flight, verify hand's loop exits immediately rather than running to 50 iterations.

Addresses the second half of issue #3044: after a user clicks stop on a
parent agent, any hand / subagent the parent had invoked via agent_send
kept running its own independent loop for up to MAX_ITERATIONS=50 more
turns, burning tokens that should never have been spent.

Root cause: every new turn allocated a fresh SessionInterrupt via
`SessionInterrupt::new()`, so the child's cancel flag was completely
disconnected from the parent's. stop_agent_run(parent) only cancelled
the parent's own flag.

Fix: one-way cancel cascade.

- `SessionInterrupt` gains an optional `upstream: Option<Arc<AtomicBool>>`.
  `is_cancelled()` returns true when either the local flag OR the upstream
  is set, but `cancel()` only affects the local flag. So parent→child
  cancel propagates; child→parent does not leak.
- New `SessionInterrupt::new_with_upstream(parent)` constructor.
- Kernel gains `send_message_as(agent_id, message, parent_agent_id)` and
  `send_message_full_with_upstream` that look up the parent's live
  SessionInterrupt from `session_interrupts` and thread it to the
  callee's loop via `execute_llm_agent`. Existing `send_message` entries
  are unchanged — they call through with `upstream_interrupt = None`.
- `KernelHandle` trait gets a default `send_to_agent_as`; the real
  kernel impl overrides it to invoke the cascade path.
- `tool_agent_send` now passes the `caller_agent_id` already available
  in ToolExecContext through to `send_to_agent_as`. System-initiated
  calls (no caller) fall back to the legacy `send_to_agent`.

The cascade only covers `agent_send`-style synchronous dispatch. Hand
agents invoked through autonomous triggers (cron, event) run with no
parent in flight, so they continue to own their own lifecycle — which
is correct.

Files
- librefang-runtime/src/interrupt.rs: upstream field, new_with_upstream,
  is_cancelled reads both flags, 4 new tests (cascade, no reverse leak,
  reset isolation, sibling sharing).
- librefang-kernel-handle/src/lib.rs: send_to_agent_as trait method
  with default fallback to send_to_agent.
- librefang-kernel/src/kernel/mod.rs: send_message_as public entry,
  send_message_full_with_upstream inner path, execute_llm_agent takes
  Option<SessionInterrupt>, KernelHandle impl override.
- librefang-runtime/src/tool_runner.rs: tool_agent_send routes to the
  cascade-aware method when caller_agent_id is known.
@github-actions github-actions Bot added size/M 50-249 lines changed ready-for-review PR is ready for maintainer review area/runtime Agent loop, LLM drivers, WASM sandbox area/kernel Core kernel (scheduling, RBAC, workflows) labels Apr 24, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 83339ab001

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/librefang-runtime/src/interrupt.rs
Comment thread crates/librefang-kernel/src/kernel/mod.rs
Addresses review feedback on #3048:

- Add two kernel-level tests exercising the `session_interrupts` lookup
  + `SessionInterrupt::new_with_upstream` chain that `send_message_as`
  uses internally. Catches regressions where any hop drops the upstream.
  Full-turn testing requires a configured LLM provider; the rest of the
  chain (`execute_llm_agent` wiring) is type-threaded and covered by the
  primitive unit tests in librefang-runtime.
- Add `TODO(#3044)` comment on the streaming entry so the next caller
  that adds streaming-based inter-agent dispatch extends the cascade.
- Resolve parent_agent_id symmetrically with the target via
  `resolve_agent_identifier` so callers can pass a name/alias and
  cascade still works (current callers pass UUIDs, but the asymmetry
  was a footgun).
- Emit a `tracing::trace!` when the default `KernelHandle::send_to_agent_as`
  impl falls through to `send_to_agent`, so non-standard handles that
  don't support cascade are diagnosable in logs.
@github-actions github-actions Bot added size/L 250-999 lines changed and removed size/M 50-249 lines changed labels Apr 24, 2026
…n tests

Addresses review findings on #3048:

- Revert the stricter parent_agent_id resolution introduced in
  d85ec09. `resolve_agent_identifier` requires the agent to be
  present in the registry, which regressed the "parent absent → no
  cascade but call proceeds" contract: a parent whose entry was
  removed mid-flight (e.g. /kill racing pending agent_send) still
  holds a live SessionInterrupt via the in-flight turn, but registry
  lookup fails. Now: try the resolver first (for name/alias support),
  fall back to bare UUID parse so stale-but-valid UUIDs still thread
  through to the session_interrupts lookup.

- Rename the earlier tests to honestly reflect what they test:
  `cascade_primitives_via_session_interrupts_dashmap` and
  `no_upstream_when_parent_has_no_active_turn` describe the primitive
  coverage rather than claiming to test `send_message_as` end-to-end.

- Add two tests that actually invoke `KernelHandle::send_to_agent_as`:
  * `send_to_agent_as_tolerates_unregistered_parent_uuid` — exercises
    the P1 fallback: a valid UUID whose agent was never registered
    must not short-circuit the call; the child-not-found failure is
    what surfaces.
  * `send_to_agent_as_rejects_unparseable_parent_id` — garbage parent
    id surfaces a clear error and does not panic.

A true end-to-end test (stubbed agent polls interrupt, parent cancel
mid-flight, observe child loop exit) still needs a minimal LLM driver
stub which doesn't exist in this crate; the primitive + public-method
coverage is the most we can do without introducing test infra for a
heavyweight concern.
@houko
houko force-pushed the fix/hand-cancel-cascade branch from f2f9e8c to 5a70753 Compare April 24, 2026 08:44
The 50-iteration cap lived in two hardcoded spots: `MAX_ITERATIONS`
const in agent_loop.rs and `AutonomousConfig::default().max_iterations`
in the types crate. Worse, even after dedup, operators still couldn't
change it without recompiling.

Changes:
- Introduce `AutonomousConfig::DEFAULT_MAX_ITERATIONS` as the single
  policy constant in librefang-types. agent_loop.rs's `MAX_ITERATIONS`
  now re-exports it.
- Add `KernelConfig.agent_max_iterations: Option<u32>` — operator
  override in config.toml. Defaults to None (use library default).
- Add `LoopOptions.max_iterations: Option<u32>` so the kernel can thread
  the config value down to the agent loop. All 4 LoopOptions
  construction sites in kernel/mod.rs now read
  `self.config.load().agent_max_iterations` (or the cached `cfg` in
  execute_llm_agent).
- Resolution order in `run_agent_loop`: per-agent `manifest.autonomous
  .max_iterations` > operator `opts.max_iterations` > library default.

Tests:
- `test_default_max_iterations_is_policy_constant` — tripwire for
  accidental policy bumps.
- `max_iterations_resolution_prefers_manifest_over_opts` /
  `_falls_back_to_opts` / `_falls_back_to_default_when_nothing_set` —
  cover the new resolution chain.
- Existing `test_autonomous_config_defaults` and
  `test_max_iterations_constant` now reference the shared constant.

Operators who want to rein in runaway loops can set
`agent_max_iterations = 15` (or whatever) in `~/.librefang/config.toml`
without touching agent manifests or the source tree.
@houko
houko force-pushed the fix/hand-cancel-cascade branch from 5a70753 to 2f7bee5 Compare April 24, 2026 08:44

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f7bee5fc5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/librefang-runtime/src/agent_loop.rs
houko added 2 commits April 24, 2026 17:53
Surfaces the new `KernelConfig.agent_max_iterations` knob (introduced
in the previous commit on this branch) in the schema-driven config UI
so operators can adjust the agent-loop iteration cap from the dashboard
without editing config.toml by hand.

- routes/config.rs `config_schema()` — new field in the "general" section:
  numeric, 1..=500, step 1. Accepts null to clear the override and fall
  back to `AutonomousConfig::DEFAULT_MAX_ITERATIONS`.
- routes/config.rs `full_config()` — emits the current value in the
  config payload so the UI can render it.

Schema is still hand-authored per field. A follow-up PR will look at
deriving it from `KernelConfig` directly (e.g. via `schemars`) to end
the pattern of "add field → also edit two places in config.rs". Out of
scope for this PR — tracked separately.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ae53ba8e7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

/// cheap models to bound cost per turn; raise it for long-horizon
/// autonomous agents. `None` means "use the compiled-in default".
#[serde(default, skip_serializing_if = "Option::is_none")]
pub agent_max_iterations: Option<u32>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wire agent_max_iterations into config reload diffing

agent_max_iterations is added to KernelConfig, but the reload pipeline never considers it (build_reload_plan has no branch for this field). In reload_config, the new config is only swapped into self.config when should_apply_hot(...) sees at least one hot action, so editing only this key via /api/config/set can return success while the runtime keeps using the old iteration cap. Please add this field to reload diff handling (or otherwise force a config swap when it changes) so the new limit actually takes effect without a restart.

Useful? React with 👍 / 👎.

The E2E test builds a deeply-nested future through kernel -> runtime ->
agent_loop. Adding fields to LoopOptions (max_iterations) and
SessionInterrupt (upstream) pushed the type-layout query past the
default 128-depth limit on all three test runners (Ubuntu/macOS/Windows)
plus the Quality clippy job.

Follows rustc's own suggestion to bump the limit to 256. No logic change.
@houko
houko merged commit 2cb5dfe into main Apr 24, 2026
19 checks passed
@houko
houko deleted the fix/hand-cancel-cascade branch April 24, 2026 11:46
@github-actions github-actions Bot removed the ready-for-review PR is ready for maintainer review label Apr 24, 2026
houko added a commit that referenced this pull request Apr 26, 2026
PR review caught 1 stale-doc contradiction + 7 undocumented behaviour
changes from the fix-PR cohort.

- #3114 — configuration/core: 'api_key empty = unauthenticated' was
  stale; now documents the loopback-only policy (non-loopback bypass
  was closed)
- #3170 — features/observability: auto_start is opt-in; home_dir-
  scoped Docker labels; RAII cleanup
- #2989 — api/agents POST /message: per-request session_id override
- #3048 — Interrupting a Running Agent Turn: /stop cascades into
  agent_send subagents (recursive)
- #3056 + #3000 — desktop: dedicated Connection screen via custom
  URI scheme with retry / open-in-browser / uninstall
- #2952 — triggers: response routes to agent's home channel
- #2955 — triggers: task_posted assignee_match = 'self' | 'any'
- #3069 — providers/local: 60-second reprobe + manual 'Test
  connection' refresh

All sections mirrored in docs/src/app/zh/.
houko added a commit that referenced this pull request Apr 26, 2026
…pi / observability (#3189)

* docs(providers): add Novita / SearXHG / Bedrock entries

* docs(config): document [parallel_tools] section + max_history_messages

* docs(scheduler): cron multi-delivery + pre_script + silent_marker

* docs: live context.md, chat attachments, CLI slash commands

Salvages three of the six items from the (quota-killed) misc-features
agent run. Each is a focused append, no rewrite of existing content.

- agent/templates: live Live Context section explains the per-turn
  re-read of `context.md` (#3115), the .identity/ vs root path lookup,
  the 32 KB cap with stale-cache fallback, and the cache_context opt-out.
- integrations/api/agents: attachment-resolution table (image / PDF /
  text+code) covering the supported MIME / extension list and the
  200 KB truncation, plus a note about adjacent-text coalescing for
  small chat-tuned local models (#3094).
- integrations/cli/commands: 'Slash Commands in Chat' section listing
  the registry-backed CLI commands (/model /status /help /clear /kill
  /exit) and pointing at channel-side commands for the broader set
  (#3122 #3123).

* docs: tool invoke API, trajectory export, doctor audit, CLI default-pick, SDK codegen

Picks up the items the second misc-features agent didn't get to before
hitting the org quota.

- integrations/api/system: POST /api/tools/{name}/invoke (#3025) — fail-
  closed allowlist, agent_id requirement for approval-gated tools.
- integrations/api/agents: GET /sessions/{id}/trajectory (#3097) —
  json/jsonl formats, redactor pipeline, audit use case.
- operations/troubleshooting: 'Audit checks' subsection enumerating the
  three registered AuditChecks (#3082) — VaultKey, ApiListenAddr,
  ConfigTomlSchema — with what each asserts and how to extend.
- configuration/providers/tools: 'CLI logins as first-class default
  providers' section (#3061) — detection priority claude-code > codex-
  cli > gemini-cli > qwen-code, override via [default_model] in
  config.toml.
- integrations/sdk: 'Auto-generated from openapi.json' section (#3046) —
  scripts/codegen-sdks.py, pre-commit hook, openai-tag skip rule.

* docs(zh): mirror providers / config-core / cron / features-index sections

* docs(zh): mirror agent / api / cli / providers-tools / troubleshooting / sdk additions

Round 2 of the zh translation pass — picks up the seven mid-priority
files that the first commit didn't cover. All sections mirror the EN
content from #3189 but read naturally as Chinese (terms like agent /
session / fallback / NoEntry left in English where they're API
surface, prose translated to 中文).

- agent/templates: Live Context section, .identity/ vs root fallback,
  cache_context = true opt-out.
- api/agents: attachment-resolution table + trajectory export endpoint
  (#3025 / #3094 / #3097).
- api/system: POST /api/tools/{name}/invoke endpoint + fail-closed
  allowlist explanation.
- cli/commands: 'Slash Commands in Chat' section listing CLI registry
  commands and pointing at channel commands.
- providers/tools: 'CLI logins as first-class default providers'
  detection priority and override.
- troubleshooting: 'Audit checks' subsection + table.
- sdk: 'Auto-generated from openapi.json' section.

* docs: refresh /new slash-command semantics (#3071 review fix)

PR review on #3189 (houko) caught two stale references describing the
pre-#3071 behavior of /new ('wipe the session' / 'clear history'). The
new semantics fork off a fresh session while the previous one stays
resumable on the same channel — phrasing updated in 4 places (en + zh
× 2 files), plus a one-line clarifier in the slash-command roster
section so the cross-reference doesn't silently inherit the old
mental model:

- configuration/channels: 'wipe the session' → 'fork off into a new
  session (the prior conversation stays resumable on the channel)'
- integrations/api/realtime: '(clear history)' → 'forks into a fresh
  session — the prior session is preserved and remains resumable'
- integrations/cli/commands roster paragraph: explicit note that /new
  is a fork, not a wipe.

Same wording mirrored to docs/src/app/zh/.

* docs(channels): 9 channel-related Apr 22-25 PRs (en + zh)

Covers the channel + bridge feature wave:
- #3020 per-agent ChannelOverrides — fixes the contradicting 'without
  modifying the agent manifest' phrasing in both en and zh.
- #3077 prefix_agent_name + Signal plain-text default
- #3009 Slack reactions_enabled + #3005 Feishu @mention preservation
  consolidated into a 'Reactions and Processing State' section
- #3008 Signal media attachments
- #3012 WhatsApp send_voice + dm_policy / group_policy
- #3011 Webhook deliver_only mode
- #2972 channel file downloads (file_download_dir / max_bytes config)

* docs(config-core): provider timeout / browser CDP / cron session caps (en + zh)

Three new `config.toml` sections that landed Apr 22-23:
- #3004 `[provider_request_timeout_secs]` map
- #2993 / #2991 `[browser].cdp_endpoint` + `cdp_auth_token_env`
- #2994 `[kernel].cron_session_max_tokens` / `cron_session_max_messages`

* docs(api): GET /attach + Owner Notice envelope (en + zh)

- #3078 SSE attach endpoint for multi-client co-watching — describes
  the per-session SessionStreamHub fan-out, attacher behaviour, and
  the curl example.
- #2965 ReplyEnvelope.owner_notice + notify_owner tool — message body
  field, StreamEvent variant, WhatsApp OWNER_JIDS fan-out.

* docs(observability + ops): Tempo stack, cache_hit_ratio metric, daemon log tee (en + zh)

- #3064 + #3149 — bundled observability stack (otel-collector / Tempo
  / Grafana), bring-up commands, [telemetry] config, business spans,
  cache_hit_ratio metric formula and where it surfaces
- #3022 — `librefang start --foreground` now tees logs to
  ~/.librefang/logs/daemon-YYYY-MM-DD.log; same-day append; 7-day
  rotation pruned on daemon start

* docs: hooks transform / lazy tools / mcp taint / capability detection / providers extras

Final batch of catch-up docs covering the remaining Apr 22-25 user-facing
features:

- agent/hooks: transform_tool_result hook contract (#3003)
- agent/templates: tool_search / tool_load lazy loading + lazy_tools
  opt-out (#3047)
- integrations/mcp-a2a: TaintRuleId enum + per-tool skip_rules policy
  (#2999)
- configuration/features: compaction summaries in user's conversation
  language (#3007)
- configuration/providers/local: embedding auto-detection priority
  list (#3099) + Ollama capability detection (Modality / metadata
  pipeline, #3074 / #3133 / #3134 / #3140)
- configuration/providers/management: custom image-gen provider via
  generic OpenAI-compat driver (#2998) + Moonshot file-upload helper
  via /v1/files (#2966)

All sections mirrored to docs/src/app/zh/.

* docs: address houko fix-PR audit (8 items, en + zh)

PR review caught 1 stale-doc contradiction + 7 undocumented behaviour
changes from the fix-PR cohort.

- #3114 — configuration/core: 'api_key empty = unauthenticated' was
  stale; now documents the loopback-only policy (non-loopback bypass
  was closed)
- #3170 — features/observability: auto_start is opt-in; home_dir-
  scoped Docker labels; RAII cleanup
- #2989 — api/agents POST /message: per-request session_id override
- #3048 — Interrupting a Running Agent Turn: /stop cascades into
  agent_send subagents (recursive)
- #3056 + #3000 — desktop: dedicated Connection screen via custom
  URI scheme with retry / open-in-browser / uninstall
- #2952 — triggers: response routes to agent's home channel
- #2955 — triggers: task_posted assignee_match = 'self' | 'any'
- #3069 — providers/local: 60-second reprobe + manual 'Test
  connection' refresh

All sections mirrored in docs/src/app/zh/.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/kernel Core kernel (scheduling, RBAC, workflows) area/runtime Agent loop, LLM drivers, WASM sandbox size/L 250-999 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant