feat(hooks): per-turn usageState on reply_payload_sending by Marvinthebored · Pull Request #89629 · openclaw/openclaw

Marvinthebored · 2026-06-02T23:13:41Z

What

Attaches a per-turn execution snapshot to the reply_payload_sending plugin hook as an optional usageState field, so hook consumers can render per-response model/usage/context metadata without extra side calls.

agent-runner records the snapshot after the model turn. dispatch attaches it to the live reply_payload_sending event only when it can correlate the payload to the exact agent run id.

Contract

usageState is best-effort live turn metadata. It is omitted for durable/recovered replay and any delivery path without exact run correlation.

Fields are optional/additive. Existing hook consumers are unaffected.

Per-turn atoms included:

model/routing: provider, model, resolved/requested refs, reasoning effort, fast mode, fallback, auth mode, override source
usage/context: token usage, last-call usage, cache usage, context budget, compaction count
cost/timing: estimated turn cost, duration
identity: agent id/session id/chat type plus configured agent identity

Provider subscription/limit windows are intentionally not included. The current run path does not carry exact credential identity for every provider/account, so attaching quota windows here would risk showing another profile's limits.

Review response

Maintainer repair on 2026-06-13:

Removed the provider-limit helper and usageState.limits contract from this PR.
Changed usageState handoff to exact run-id correlation only; no session-key fallback.
Shared the generated run id from onAgentRunStart into the delivery hook, so normal live replies still get the snapshot even when the caller did not provide replyOptions.runId.
Documented the live/best-effort hook behavior in docs/plugins/hooks.md.
Trimmed excessive comments and removed stale limit tests/code.

Verification

node scripts/run-vitest.mjs src/auto-reply/dispatch.test.ts src/auto-reply/reply/reply-usage-state.test.ts src/plugins/wired-hooks-reply-payload-sending.test.ts src/auto-reply/reply/dispatch-from-config.test.ts
tsgo -p tsconfig.core.json --declaration false --singleThreaded --checkers 1
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --prompt 'Review PR 89629 after maintainer repair. Focus on exact run correlation for reply usageState, provider limit scoping, public hook docs, and regressions introduced by the repair commit.'

Real behavior proof

Behavior or issue addressed: This PR exposes live per-turn usageState on reply_payload_sending so hook consumers can render model, usage, context, cost, timing, routing, and identity atoms without side calls. Provider subscription/limit windows are intentionally not part of this PR after maintainer repair because the live run path does not carry exact credential identity for every provider/account.

Real environment tested: The contributor tested the usageState contract on a live OpenClaw gateway with downstream footer consumption in Telegram and Discord. The PR itself has no in-core UX; the proof uses a hook consumer to show the live contract data reaching real channel replies.

Exact steps or command run after this patch: Built and deployed the branch to a live OpenClaw gateway, sent real replies through Telegram and Discord, and observed a downstream reply_payload_sending footer consumer render the live usageState atoms.

Evidence after fix: Live downstream screenshots from the contributor show the footer rendered from usageState data: Telegram https://raw.githubusercontent.com/Marvinthebored/openclaw/proof-assets/pr-proof/proof_telegram2.png and Discord https://raw.githubusercontent.com/Marvinthebored/openclaw/proof-assets/pr-proof/proof_discord1.jpeg

Observed result after fix: Real Telegram and Discord replies included the footer generated from the hook usageState data. After maintainer repair, the supported contract is the same live per-turn usageState delivery minus provider limit windows; uncorrelated and durable replay paths intentionally omit it.

What was not tested: No fresh maintainer-run live gateway screenshot was captured after removing the provider-limit field. The live proof covers downstream usageState delivery; scoped maintainer tests cover the repaired exact-run correlation and omission behavior.

clawsweeper · 2026-06-02T23:15:29Z

Codex review: needs real behavior proof before merge. Reviewed June 13, 2026, 4:50 AM ET / 08:50 UTC.

Summary
Review failed before ClawSweeper could summarize the requested change.

PR surface: Source +228, Tests +93, Docs +4. Total +325 across 8 files.

Reproducibility: unclear. The review failed before ClawSweeper could establish a reproduction path.

Review metrics: none identified.

Merge readiness
Overall: 🌊 off-meta tidepool
Proof: 🌊 off-meta tidepool
Patch quality: 🌊 off-meta tidepool
Result: rating does not apply to this item.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] No close action taken because the review did not complete.

Maintainer options:

Decide the mitigation before merge
Retry the Codex review after fixing the execution failure.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] Review did not complete, so no work-lane recommendation was made.

Review details

Best possible solution:

Retry the Codex review after fixing the execution failure.

Do we have a high-confidence way to reproduce the issue?

Unclear. The review failed before ClawSweeper could establish a reproduction path.

Is this the best way to solve the issue?

Unclear. Retry the review first so ClawSweeper can evaluate the actual issue and fix direction.

AGENTS.md: unclear because the file could not be read completely.

Codex review notes: model internal, reasoning high; reviewed against 7bd533a80e9d.

Label changes

Label changes:

add rating: 🌊 off-meta tidepool: Overall readiness is 🌊 off-meta tidepool; proof is 🌊 off-meta tidepool and patch quality is 🌊 off-meta tidepool.
remove mantis: telegram-visible-proof: Current Telegram visible-proof status is not_needed.
remove P2: Current review triage priority is none.
remove rating: 🧂 unranked krab: Current PR rating is rating: 🌊 off-meta tidepool, so this older rating label is no longer current.
remove merge-risk: 🚨 compatibility: Current PR review selected no merge-risk labels.
remove merge-risk: 🚨 auth-provider: Current PR review selected no merge-risk labels.
remove status: 📣 needs proof: Current PR status no longer selects a status label.

Label justifications:

rating: 🌊 off-meta tidepool: Overall readiness is 🌊 off-meta tidepool; proof is 🌊 off-meta tidepool and patch quality is 🌊 off-meta tidepool.

Evidence reviewed

PR surface:

Source +228, Tests +93, Docs +4. Total +325 across 8 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	5	242	14	+228
Tests	2	93	0	+93
Docs	1	4	0	+4
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	8	339	14	+325

What I checked:

failure reason: timeout.
codex failure detail: Codex review failed for this PR: spawnSync /opt/hostedtoolcache/node/24.16.0/x64/bin/node ETIMEDOUT.
codex stderr: No stderr captured.
codex stdout: ly/directive-handling.model.ts:92: params.authMode,\nsrc/auto-reply/reply/directive-handling.model.ts:121: params.authMode,\nsrc/auto-reply/reply/directive-handling.model.ts:433: const authMode: ModelAuthDetailMode = "verbose";\nsrc/auto-reply/reply/directive-handling.model.ts:451: authMode,\nsrc/agents/tools/media-generate-tool-actions-shared.ts:13:import type { AuthProfileStore } from "../auth-profiles/types.js";\nsrc/agents/tools/manifest-capability-availability.ts:19:import { listProfilesForProvider } from "../auth-profiles/profile-list.js";\nsrc/agents/tools/manifest-capability-availability.ts:20:import type { AuthProfileStore } from "../auth-profiles/types.js";\nsrc/agents/tools/pdf-tool.test.ts:12:import type { AuthProfileStore } from "../auth-profiles/types.js";\nsrc/agents/tools/pdf-tool.test.ts:219: type: "api_key",\nsrc/agents/tools/model-config.helpers.ts:20:} from "../auth-profile.
process error code: ETIMEDOUT.

Likely related people:

unknown: Codex failed before it could trace repository history. (role: review did not complete; confidence: low)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

…ffort delivery Addresses review feedback on openclaw#89629. 1) Provider-limit resolution no longer defaults to OAuth. getProviderUsageLimits and getProviderUsageLimitsCached resolved with `credentialType ?? "oauth"`, and the agent-runner snapshot call passed no credential type, so an api-key OpenAI turn could borrow cached OAuth/ChatGPT usage windows. Drop the "oauth" default (missing credential type => no OpenAI usage provider) and thread the turn's authMode through at the call site. Adds provider-usage.limits.test.ts covering api-key/no-credential (no fetch), oauth/token (resolves), and non-OpenAI. 2) usageState is documented as best-effort, present only on live dispatcher delivery. Routed durable and recovered queue replays re-run this hook as a stateless transform over the original payload (see QueuedDeliveryPayload); a point-in-time usage snapshot is not stateless and would replay stale after a restart, so it is intentionally omitted there. Consumers must treat the field as optional. Co-Authored-By: Claude Opus 4.8 <[email protected]>

Peetiegonzalez · 2026-06-03T14:51:53Z

The reason we push for this in core is that it is a very simple way to allow user(agent)-defined formatting of the system state input contract. Always-on, without having to type /status and get a wall-of-text.

Default /usage display is lacking in information and typically ugly. As LLM-designed UX tends to be. Here, we add a translation layer with fallback to default. If there is no translation definition file present, default works just fine.

With user input, it has the potential to greatly improve UX with an always-on dashboard in common communications channels, without introducing an executable payload via a plugin. It's just a .json file in the user directory, carrying an equivalence dictionary.

NB: these are user-defined by translating the state input contract, they can be as colourful or boring as you like.

…ffort delivery Addresses review feedback on openclaw#89629. 1) Provider-limit resolution no longer defaults to OAuth. getProviderUsageLimits and getProviderUsageLimitsCached resolved with `credentialType ?? "oauth"`, and the agent-runner snapshot call passed no credential type, so an api-key OpenAI turn could borrow cached OAuth/ChatGPT usage windows. Drop the "oauth" default (missing credential type => no OpenAI usage provider) and thread the turn's authMode through at the call site. Adds provider-usage.limits.test.ts covering api-key/no-credential (no fetch), oauth/token (resolves), and non-OpenAI. 2) usageState is documented as best-effort, present only on live dispatcher delivery. Routed durable and recovered queue replays re-run this hook as a stateless transform over the original payload (see QueuedDeliveryPayload); a point-in-time usage snapshot is not stateless and would replay stale after a restart, so it is intentionally omitted there. Consumers must treat the field as optional. Co-Authored-By: Claude Opus 4.8 <[email protected]>

…on reply_payload_sending Attach a per-turn execution snapshot to the reply_payload_sending hook as `usageState`, so a plugin (or the future in-core /usage renderer) can render a per-response usage readout as a pure consumer of the contract — no side calls. Recorded in agent-runner, consumed in dispatch. Fields: provider, model, resolvedRef/requested, reasoningEffort, fastMode, fallbackUsed, is_override (overrideSource), authMode, compactionCount, contextTokenBudget, token usage, turn cost (USD), duration, owning agentId/sessionId, chatType, the agent identity (name/emoji), and the active provider's subscription `limits` windows. reply_payload_sending is the one reply hook universal across every surface (incl. the Codex app-server, which emits no llm_output/agent_end), so it is the correct harness-agnostic place for per-turn usage. Limits are resolved by a core-internal non-blocking SWR helper (src/infra/provider-usage.limits.ts) and attached to the snapshot — no new plugin-SDK accessor. All fields optional/additive. Co-Authored-By: Claude Opus 4.8 <[email protected]>

…ffort delivery Addresses review feedback on openclaw#89629. 1) Provider-limit resolution no longer defaults to OAuth. getProviderUsageLimits and getProviderUsageLimitsCached resolved with `credentialType ?? "oauth"`, and the agent-runner snapshot call passed no credential type, so an api-key OpenAI turn could borrow cached OAuth/ChatGPT usage windows. Drop the "oauth" default (missing credential type => no OpenAI usage provider) and thread the turn's authMode through at the call site. Adds provider-usage.limits.test.ts covering api-key/no-credential (no fetch), oauth/token (resolves), and non-OpenAI. 2) usageState is documented as best-effort, present only on live dispatcher delivery. Routed durable and recovered queue replays re-run this hook as a stateless transform over the original payload (see QueuedDeliveryPayload); a point-in-time usage snapshot is not stateless and would replay stale after a restart, so it is intentionally omitted there. Consumers must treat the field as optional. Co-Authored-By: Claude Opus 4.8 <[email protected]>

…limits requestShaping.authMode is the auth *mechanism* (e.g. "auth-profile" for a configured auth profile), not the credential *type* resolveUsageProviderId expects. Gating limits on it === "oauth"/"token" dropped 📊 for legit OAuth (profile-based) turns. Map it: api-key/aws-sdk -> no usage provider (cannot borrow cached oauth windows); oauth/token/auth-profile/absent -> usage-eligible, with the real credential re-checked at fetch time. Co-Authored-By: Claude Opus 4.8 <[email protected]>

…eState context.used_tokens / pct_used were derived from the snapshot's aggregate prompt total (cacheRead+cacheWrite+input). Over a multi-call tool-loop turn that is the run AGGREGATE, overstating window occupancy (often past 100%) so a footer's context gauge pins full while /status shows the true ~7%. Add two optional fields to PluginHookReplyUsageState and populate them in the reply path: - contextUsedTokens: the final call's prompt size (agentMeta.promptTokens) = real end-of-turn occupancy, a point-in-time state, not the aggregate. - lastUsage: the final model call's usage only (vs `usage`, the turn aggregate), so a footer can render the last exchange's i/o + cache. Both optional and additive; consumers fall back to the aggregate when absent (correct for single-call turns). Renderer consumption lands separately (openclaw#89835). Co-Authored-By: Claude Opus 4.8 <[email protected]>

A missing per-turn authMode was mapped to "oauth", so an OpenAI api-key turn that arrived without an explicit auth mechanism could resolve and display ChatGPT subscription windows that aren't its own — served straight from the 60s limits cache, which the "re-checked at fetch time" guard does not cover. Treat a genuinely absent signal as non-eligible (same as api-key): no usage provider resolves and the footer omits limit windows. Present mechanisms are unchanged — oauth/auth-profile/token stay eligible, and only OpenAI is gated on the credential type so other providers are unaffected. A real oauth/profile turn always carries its mechanism; one arriving blank is an upstream tagging bug to fix at the source. Inverts the now-incorrect "absent => oauth-eligible" test into regression coverage for the absent/api-key case. Co-Authored-By: Claude Opus 4.8 <[email protected]>

obviyus · 2026-06-13T08:55:44Z

Landed via rebase onto main.

Scoped tests: node scripts/run-vitest.mjs src/auto-reply/dispatch.test.ts src/auto-reply/reply/reply-usage-state.test.ts src/plugins/wired-hooks-reply-payload-sending.test.ts src/auto-reply/reply/dispatch-from-config.test.ts; tsgo -p tsconfig.core.json --declaration false --singleThreaded --checkers 1; git diff --check; .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --prompt 'Pre-land review for PR 89629 after rebase. Focus on reply_payload_sending usageState run correlation, optional live-only contract, provider limits removal, docs accuracy, and regressions.'
Changelog: not updated; PR body carries the user-facing release-note context for this hook contract.
Land commit: 7a86ed4
Merge commit: 1c65500

Thanks @Marvinthebored!

openclaw-barnacle Bot added the size: S label Jun 2, 2026

Marvinthebored mentioned this pull request Jun 2, 2026

feat(plugin-sdk): provider usage-limits read accessor for plugins #89631

Closed

openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label Jun 2, 2026

Marvinthebored force-pushed the pr-reply-usagestate branch from 301834c to a7ca7f5 Compare June 3, 2026 01:36

Marvinthebored changed the title ~~feat(hooks): expose per-turn usageState on reply_payload_sending~~ feat(hooks): per-turn usageState (with provider usage limits) on reply_payload_sending Jun 3, 2026

openclaw-barnacle Bot added size: M and removed size: S labels Jun 3, 2026

clawsweeper Bot added the merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. label Jun 3, 2026

Marvinthebored force-pushed the pr-reply-usagestate branch from a7ca7f5 to 79ed4ad Compare June 3, 2026 04:53

Marvinthebored changed the title ~~feat(hooks): per-turn usageState (with provider usage limits) on reply_payload_sending~~ feat(hooks): per-turn usageState (provider limits + rich atoms) on reply_payload_sending Jun 3, 2026

clawsweeper Bot added the merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. label Jun 3, 2026

Marvinthebored mentioned this pull request Jun 3, 2026

feat(usage): native templated /usage full footer renderer #89835

Merged

openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 3, 2026

Marvinthebored force-pushed the pr-reply-usagestate branch from a3b31df to 69a8974 Compare June 4, 2026 02:17

obviyus force-pushed the pr-reply-usagestate branch from 4b7b510 to 750fbf6 Compare June 13, 2026 08:28

obviyus changed the title ~~feat(hooks): per-turn usageState (provider limits + rich atoms) on reply_payload_sending~~ feat(hooks): per-turn usageState on reply_payload_sending Jun 13, 2026

openclaw-barnacle Bot added triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. and removed proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 13, 2026

obviyus self-assigned this Jun 13, 2026

Peetiegonzalez and others added 6 commits June 13, 2026 14:10

fix(hooks): tighten reply usage state correlation

7a86ed4

obviyus force-pushed the pr-reply-usagestate branch from 750fbf6 to 7a86ed4 Compare June 13, 2026 08:46

openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 13, 2026

Marvinthebored mentioned this pull request Jun 30, 2026

pipeline: normalized provider→channel stream grammar (core) #93342

Closed

clawsweeper Bot mentioned this pull request Jul 7, 2026

[Feature]: Expose sessionKey/runId to outbound message hook for deterministic verification #18571

Open

clawsweeper Bot mentioned this pull request Jul 17, 2026

fix(auto-reply): prevent reply hook memory growth during bursts #109610

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(hooks): per-turn usageState on reply_payload_sending#89629

feat(hooks): per-turn usageState on reply_payload_sending#89629
obviyus merged 6 commits into
openclaw:mainfrom
Marvinthebored:pr-reply-usagestate

Marvinthebored commented Jun 2, 2026 •

edited by obviyus

Loading

Uh oh!

clawsweeper Bot commented Jun 2, 2026 •

edited

Loading

Uh oh!

Peetiegonzalez commented Jun 3, 2026 •

edited

Loading

Uh oh!

obviyus commented Jun 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

Marvinthebored commented Jun 2, 2026 • edited by obviyus Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Contract

Review response

Verification

Real behavior proof

Uh oh!

clawsweeper Bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Peetiegonzalez commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

obviyus commented Jun 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Marvinthebored commented Jun 2, 2026 •

edited by obviyus

Loading

clawsweeper Bot commented Jun 2, 2026 •

edited

Loading

Peetiegonzalez commented Jun 3, 2026 •

edited

Loading