-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
[Feature]: agent_end (or new turn_end) hook payload should expose per-turn tool-call outcomes + completion_reason #87207
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:needs-security-reviewClawSweeper marked this issue as needing security-sensitive review.ClawSweeper marked this issue as needing security-sensitive review.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.This issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:needs-security-reviewClawSweeper marked this issue as needing security-sensitive review.ClawSweeper marked this issue as needing security-sensitive review.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.This issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Problem
Plugins can subscribe to
agent_endto observe when a turn finishes, but the payload is structurally insufficient for any observability use case that needs per-turn outcome data. Today'sagent_endpayload (as observed against OC 2026.5.22) carries:Plus
ctxwith session identity. Notably missing:"stop" | "max_tokens" | "tool_use" | "error"from the LLM provider)Tool-use and tool-result blocks DO appear inline inside
messages[]as Anthropic-format content blocks, so a careful plugin can parse them out of the message array. But that approach has costs:completion_reasonin any payload field, so plugins can't distinguish "model decided to stop" from "ran out of context" from "tool-use loop continues."Concrete ask
Either of:
Option A: Enrich
agent_endpayload (preferred for backward compatibility)Option B: New
turn_endhook with the rich payloadIf extending
agent_endis undesirable (semver, downstream coupling), add a complementaryturn_endhook that fires alongsideagent_endand carries the new fields. Plugins opt in by subscribing.Why this matters
Concrete consumer: Evolve is building a per-turn struggle/cost analyzer (see linked spec). It needs tool_error_count + tool_retry_count + completion_reason as primary features. Right now we read these by post-hoc-parsing
messages[]content blocks — works, but couples our plugin to a content-block schema we don't own and miss per-tool durations entirely.More broadly, this is the foundational data shape any cost/performance/quality observability plugin needs. Cascadia, Helicone-style traces, FrugalGPT-style cascading, RouteLLM-style classifier training data — all of them want tool-outcome data per turn as first-class fields, not parsed-out-of-message-blocks heuristics. Same need is implied by issue #82548 (AI safety and quality observability events) but at a finer granularity.
What I'm NOT asking for
before_tool_call,after_tool_call). The hook-governance discussion explicitly classifies these as out-of-scope for the plugin API. Per-turn aggregate data onagent_end(orturn_end) is the right interception point.Workaround we're shipping while waiting
For our struggle detector, we currently parse
tool_useandtool_resultblocks out ofmessages[]and count them. Works for Anthropic-shape providers; will need a fallback path for other backends. Per-tool wall-clock duration we don't have a workaround for — we fall back tototal_durationMs / tool_call_countas a proxy. Acceptable for v1 but lossy.Acceptance criteria
agent_end(or newturn_end) payload includes the four new field families aboveagent_endcontinue to work unchanged (additive change)Environment
openclawglobal on macOS)openclaw/plugin-sdkshipping with that release@openclaw/evolve(private; not relevant to the ask)api.on("agent_end", (event, ctx) => console.log(Object.keys(event)))produces onlymessages,success,durationMs.