Skip to content

fix(subagents): record sub-agent LLM token usage to daily stats (#1597)#1600

Merged
Aaronontheweb merged 2 commits into
netclaw-dev:devfrom
Aaronontheweb:fix/subagent-token-usage-stats
Jul 8, 2026
Merged

fix(subagents): record sub-agent LLM token usage to daily stats (#1597)#1600
Aaronontheweb merged 2 commits into
netclaw-dev:devfrom
Aaronontheweb:fix/subagent-token-usage-stats

Conversation

@Aaronontheweb

Copy link
Copy Markdown
Collaborator

Closes #1597.

Problem

Sub-agent LLM calls never recorded token usage, so every sub-agent's input/output tokens were invisible to netclaw stats.

This is a pre-existing gap, not a regression. git log -S for ISessionMetrics/UsageDetails/SessionMetrics scoped to SubAgentActor.cs returns zero commits across all history — the actor never had an ISessionMetrics dependency and silently discarded the ChatResponse.Usage it already receives from StreamingResponseReader. The recent observability PRs (#1428, #1468, #1472/#1499) only added logging and pruned dead OTel Activity objects; none ever touched token tracking.

Fix — record at the source

ISessionMetrics (the DailyStatsPublisher) is a process-wide singleton that the parent session and every sub-agent share. So the correct fix is to record each sub-agent LLM call once, in the child, exactly as the main session records its own calls:

  • SubAgentActor — inject ISessionMetrics?; record response.Usage on every LlmResponseReceived (one per LLM turn: tool-call turns, retries, the forced-no-tools final turn, and repair turns); add cumulative input/output token totals to the completion summary log.
  • SubAgentSpawner — inject the DI singleton and pass it through CreateProps. No Program.cs change: DI already registers the singleton and injects it into the optional parameter, matching SessionCatalogService/MemoryCurationWorkerService.

Why not also aggregate in the parent (the issue's suggestion B)?

Because the parent and child share the same singleton, re-recording propagated totals in the parent would double-count every sub-agent token in netclaw stats. Folding sub-agent tokens into the parent's UsageOutput would also corrupt the parent's context-window percentage — the sub-agent has its own context window. Recording once at the source is both correct and minimal.

Tests

Four regression tests, each verified to fail if the recording is removed (neutralizing the call made all four fail, incl. the summary log showing inputTokens=0, outputTokens=0):

  • Records_token_usage_to_session_metrics_on_text_response — single-call billing.
  • Records_token_usage_for_every_llm_call_across_the_turn_loop — sums usage across a tool-call turn + final-text turn (the crux: internal calls, not just the final output).
  • Completion_summary_reports_cumulative_token_totals — summary log carries token totals.
  • Spawned_sub_agent_bills_its_llm_calls_to_session_metrics — full SubAgentSpawner → CreateProps → actor wiring, so dropping the pass-through anywhere in that chain fails here.

Adds a UsageOverride hook to the sub-agent test FakeChatClient and extracts a shared RecordingSessionMetrics test helper.

Validation

  • Full Netclaw.Actors.Tests: 2551 passed, 0 failed; Netclaw.Daemon builds clean.
  • dotnet slopwatch analyze: 0 issues. Copyright headers: all present.

Follow-up

Filed #1599 to consolidate the three divergent FakeChatClient doubles into a shared Netclaw.Tests.Utilities helper (deliberately kept out of this PR to stay focused).

…law-dev#1597)

Sub-agent LLM calls never recorded token usage, so every sub-agent's
input/output tokens were invisible to `netclaw stats`. This was a
pre-existing gap, not a regression: SubAgentActor never had an
ISessionMetrics dependency and discarded the ChatResponse.Usage it already
receives from StreamingResponseReader. The recent observability PRs
(netclaw-dev#1428, netclaw-dev#1468, netclaw-dev#1472/netclaw-dev#1499) only added logs and pruned dead OTel Activities;
none ever touched token tracking.

Fix records at the source, mirroring the main session:
- Inject ISessionMetrics into SubAgentActor (via SubAgentSpawner/CreateProps).
  DI already registers it as a process-wide singleton, so no Program.cs change.
- Record response.Usage on every LlmResponseReceived (one per LLM turn:
  tool-call turns, retries, the forced-no-tools final turn, repair turns).
- Add cumulative input/output token totals to the completion summary log.

Recorded in the child, not propagated to the parent: ISessionMetrics is the
SAME process-wide singleton both share, so re-recording in the parent would
double-count, and folding sub-agent tokens into the parent's UsageOutput
would corrupt its context-window percentage (the sub-agent has its own
context window).

Regression coverage (all four fail if the recording is removed):
- SubAgentActor bills usage per LLM call and sums across the turn loop.
- Completion summary log carries token totals.
- Full spawner->CreateProps->actor wiring bills tokens to the spawner's metrics.

Adds a UsageOverride hook to the sub-agent test FakeChatClient and extracts a
shared RecordingSessionMetrics test helper.
@Aaronontheweb Aaronontheweb added subagents spawn_agent, SubAgentActor, definition loader, discovery context layer, and related features observability labels Jul 8, 2026

@Aaronontheweb Aaronontheweb left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Aaronontheweb
Aaronontheweb enabled auto-merge (squash) July 8, 2026 16:34
@Aaronontheweb
Aaronontheweb merged commit 96dc538 into netclaw-dev:dev Jul 8, 2026
15 checks passed
Aaronontheweb added a commit that referenced this pull request Jul 8, 2026
… (#1600)

Sub-agent LLM calls never recorded token usage, so every sub-agent's
input/output tokens were invisible to `netclaw stats`. This was a
pre-existing gap, not a regression: SubAgentActor never had an
ISessionMetrics dependency and discarded the ChatResponse.Usage it already
receives from StreamingResponseReader. The recent observability PRs
(#1428, #1468, #1472/#1499) only added logs and pruned dead OTel Activities;
none ever touched token tracking.

Fix records at the source, mirroring the main session:
- Inject ISessionMetrics into SubAgentActor (via SubAgentSpawner/CreateProps).
  DI already registers it as a process-wide singleton, so no Program.cs change.
- Record response.Usage on every LlmResponseReceived (one per LLM turn:
  tool-call turns, retries, the forced-no-tools final turn, repair turns).
- Add cumulative input/output token totals to the completion summary log.

Recorded in the child, not propagated to the parent: ISessionMetrics is the
SAME process-wide singleton both share, so re-recording in the parent would
double-count, and folding sub-agent tokens into the parent's UsageOutput
would corrupt its context-window percentage (the sub-agent has its own
context window).

Regression coverage (all four fail if the recording is removed):
- SubAgentActor bills usage per LLM call and sums across the turn loop.
- Completion summary log carries token totals.
- Full spawner->CreateProps->actor wiring bills tokens to the spawner's metrics.

Adds a UsageOverride hook to the sub-agent test FakeChatClient and extracts a
shared RecordingSessionMetrics test helper.
Aaronontheweb added a commit to Aaronontheweb/netclaw that referenced this pull request Jul 8, 2026
Reconciles cherry-picked dev commits (version-bump-to-beta.2, Grpc.Tools
2.82.0, SkillServer 0.4.0-beta.3, BOM frontmatter tests,
memory-relevance-gate opsx artifacts) plus two commits we lacked:
sub-agent LLM token usage daily stats fix (netclaw-dev#1600) and Netclaw.SkillClient
0.4.0-beta.3 -> 0.4.0-beta.4 bump (netclaw-dev#1596).

Conflict resolutions:
- Directory.Build.props: kept VersionSuffix alpha.onnx.1 (ours); we remain
  the alpha.onnx.1 experimental prerelease, not beta.2.
- Directory.Packages.props: took Netclaw.SkillClient 0.4.0-beta.4 (theirs).
- RELEASE_NOTES.md: kept the alpha.onnx.1 section on top, followed by a
  single 0.25.0-beta.2 section (content was identical on both sides), then
  the older history unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

observability subagents spawn_agent, SubAgentActor, definition loader, discovery context layer, and related features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sub-agent LLM calls never record or emit token usage data (invisible to 'netclaw stats')

1 participant