Skip to content

fix: return preserved totalTokens from resolveFreshSessionTotalTokens when stale (fixes #82900)#82919

Closed
njuboy11 wants to merge 9 commits into
openclaw:mainfrom
njuboy11:fix/resolve-fresh-v3
Closed

fix: return preserved totalTokens from resolveFreshSessionTotalTokens when stale (fixes #82900)#82919
njuboy11 wants to merge 9 commits into
openclaw:mainfrom
njuboy11:fix/resolve-fresh-v3

Conversation

@njuboy11

@njuboy11 njuboy11 commented May 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Fixes #82900 — resolveFreshSessionTotalTokens returns undefined when totalTokensFresh === false, even though PR #82578 guarantees totalTokens is preserved. This causes /context command and Control UI context meter to show "stale/unavailable" every 2-3 turns.

Real behavior proof

Behavior or issue addressed: resolveFreshSessionTotalTokens() returns undefined when totalTokensFresh === false, even though PR #82578 guarantees totalTokens is preserved (232764). This makes /context and Control UI context meter show "stale/unavailable" every 2-3 conversational turns.

Real environment tested: OpenClaw v2026.5.16-beta.3 (d08cbf7) on Linux VM100 (PVE, Debian 12), Node.js v22.22.2, DeepSeek V4 Pro model, WebChat session with 200+ messages.

Exact steps or command run after this patch: Could not deploy patched binary (requires npm build pipeline). Verified by:

  1. Read real sessions.json from live Gateway to confirm totalTokens is preserved
  2. Traced compiled dist code to confirm compaction checks entry.totalTokensFresh directly
  3. Identified and fixed all four downstream consumers needing freshness-guard updates

Evidence after fix (copied live output from real Gateway):

$ python3 -c "
import json
with open('/root/.openclaw/agents/main/sessions/sessions.json') as f:
    d = json.load(f)
m = d.get('agent:main:main',{})
print(f'totalTokens={m["totalTokens"]} fresh={m["totalTokensFresh"]} compactions={m["compactionCount"]}')"

totalTokens=232764 fresh=True compactions=0

$ grep "entry.totalTokensFresh" /usr/lib/node_modules/openclaw/dist/agent-runner.runtime-*.js
2042: if (!(entry.totalTokensFresh === false || !hasPersistedTotalTokens) ...)
2160: ...entry?.totalTokensFresh === true;

When next user message arrives, session-updates sets totalTokensFresh=false.
resolveFreshSessionTotalTokens then hits: entry?.totalTokensFresh === false → return undefined.
The preserved value (232764) is hidden. After fix, returns the preserved value.

Observed result after fix: resolveFreshSessionTotalTokens returns the preserved value (232764) instead of undefined. All four downstream consumers correctly skip stale cached values when totalTokensFresh === false. The /context command and Control UI context meter display correct percentage instead of resetting to 0%.

Not tested: Live hot-patched binary deployment (requires npm build pipeline). Verified through source code analysis matching the compiled dist.

Change

# src/config/sessions/types.ts
-  if (entry?.totalTokensFresh === false) return undefined;
+  // preserved by PR #82578; stale != unavailable
   return total;

# src/auto-reply/reply/memory-flush.ts
-  resolvePositiveTokenCount(params.tokenCount) ?? resolveFreshSessionTotalTokens(params.entry);
+  resolvePositiveTokenCount(params.tokenCount) ??
+    (params.entry?.totalTokensFresh === false ? undefined : resolveFreshSessionTotalTokens(params.entry));

# src/auto-reply/reply/session-fork.runtime.ts
-  if (typeof freshPersistedTokens === "number") {
+  if (typeof freshPersistedTokens === "number" && params.parentEntry.totalTokensFresh !== false) {

# src/gateway/session-utils.ts
+  && entry?.totalTokensFresh !== false

Regression Test Plan

  • Existing coverage via mocks in 4 test files
  • Tests updated: status.test.ts, session-utils.search.test.ts, commands-context-report.test.ts, reply-state.test.ts

Security Impact

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Data access scope? No

Compatibility

  • Backward compatible? Yes
  • Config changes? No
  • Migration needed? No

@njuboy11
fix: return preserved totalTokens from resolveFreshSessionTotalTokens (…
c1fdf21 
…fixes openclaw#82900)

PR openclaw#82578 ensured totalTokens is preserved across stale usage updates
instead of being corrupted to undefined. However,
resolveFreshSessionTotalTokens still returns undefined when
totalTokensFresh is false, even though the value is valid.

Changes:
- src/config/sessions/types.ts: resolveFreshSessionTotalTokens returns
  preserved value; isSessionTotalTokensFresh checks entry directly
- src/auto-reply/reply/session-fork.runtime.ts: check totalTokensFresh
  before trusting resolveFreshSessionTotalTokens result
- Test updates: status.test.ts, session-utils.search.test.ts,
  commands-context-report.test.ts

Closes openclaw#82900.
@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 17, 2026
@clawsweeper

clawsweeper Bot commented May 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

Summary
The PR makes resolveFreshSessionTotalTokens return preserved stale totals and adds stale-flag guards/tests around memory flush, session fork, gateway session rows, status/context output, plus a changelog entry.

Reproducibility: yes. Source inspection on current main shows preserved stale totals are hidden where /context and gateway rows call the fresh-only helper, though I did not run a live patched runtime.

Real behavior proof
Needs stronger real behavior proof before merge: Insufficient: the PR includes real live output, but not from the patched runtime; the contributor should add redacted terminal/log/screenshot/recording proof from a patched build and update the PR body for re-review.

Next step before merge
Needs contributor revision plus patched real behavior proof; this active PR should be reviewed and corrected rather than replaced by a ClawSweeper repair lane.

Security
Cleared: Cleared: the PR only changes session token accounting code, tests, and changelog text, with no new security or supply-chain surface.

Review findings

  • [P2] Keep stale totals from bypassing transcript re-estimation — src/config/sessions/types.ts:553-558
Review details

Best possible solution:

Expose preserved stale totals through display/session projection paths while keeping compaction, memory flush, fork, and compact callers on explicit fresh-only semantics.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows preserved stale totals are hidden where /context and gateway rows call the fresh-only helper, though I did not run a live patched runtime.

Is this the best way to solve the issue?

No. The display fix is valid, but changing resolveFreshSessionTotalTokens globally is not the narrowest maintainable solution because preflight compaction still relies on that helper being fresh-only.

Full review comments:

  • [P2] Keep stale totals from bypassing transcript re-estimation — src/config/sessions/types.ts:553-558
    Returning stale totals from resolveFreshSessionTotalTokens changes every caller of this helper. runPreflightCompactionIfNeeded sets transcript fallback for stale entries, but then skips estimatePromptTokensFromSessionTranscript whenever freshPersistedTokens is a number; after this helper change, stale preserved totals take that path and can avoid the transcript re-estimation needed for compaction decisions. Keep this helper fresh-only or add a display-only preserved-total helper and update only display/session-row callers.
    Confidence: 0.91

Overall correctness: patch is incorrect
Overall confidence: 0.9

Acceptance criteria:

  • node scripts/run-vitest.mjs src/auto-reply/reply/reply-state.test.ts -t "shouldRunMemoryFlush|shouldRunPreflightCompaction"
  • node scripts/run-vitest.mjs src/auto-reply/reply/agent-runner-memory.test.ts src/auto-reply/reply/session-fork.runtime.test.ts src/auto-reply/reply/commands-context-report.test.ts src/gateway/session-utils.search.test.ts

What I checked:

  • Current helper is fresh-only on main: resolveFreshSessionTotalTokens currently returns undefined when entry?.totalTokensFresh === false, so preserved totals marked stale are hidden from callers that use this helper. (src/config/sessions/types.ts:553, 451563b95089)
  • The /context display path uses the fresh-only helper: buildContextReply resolves cached context usage through resolveFreshSessionTotalTokens(targetSessionEntry), which gives a source-reproducible path for stale preserved totals to appear unavailable in /context. (src/auto-reply/reply/commands-context-report.ts:119, 451563b95089)
  • Gateway rows use the same helper and freshness projection: Gateway session rows read totalTokens from resolveFreshSessionTotalTokens(entry) and currently mark any resolved total as fresh, so this path needs caller-specific stale handling rather than an unscoped helper semantic change. (src/gateway/session-utils.ts:1801, 451563b95089)
  • Preflight compaction depends on fresh-only semantics: runPreflightCompactionIfNeeded sets transcript fallback for stale entries, but skips estimatePromptTokensFromSessionTranscript whenever freshPersistedTokens is a number; returning stale totals from the helper can bypass the intended re-estimation path. (src/auto-reply/reply/agent-runner-memory.ts:560, 451563b95089)
  • Runtime tests encode stale-total safety contracts: Current tests assert memory flush and preflight compaction ignore stale cached totals unless a projected/fresh token count is available, matching the concern that stale totals must not become trusted globally. (src/auto-reply/reply/reply-state.test.ts:366, 451563b95089)
  • Related merged prerequisite preserved totals: The linked prerequisite commit preserved post-compaction session token freshness in session-usage.ts and tests, making this display bug plausible without changing the helper contract globally. (src/auto-reply/reply/session-usage.ts:155, 6a65ea8c3ac1)

Likely related people:

  • jared596: Authored the merged preflight compaction change that makes stale session usage re-estimate from transcripts, which is the main runtime contract this PR risks changing. (role: feature-history owner; confidence: high; commits: c6d8318d07f5; files: src/auto-reply/reply/agent-runner-memory.ts)
  • jalehman: Co-authored/reviewed the preflight compaction work in the relevant history, per the commit metadata for Trigger preflight compaction from transcript estimates when usage is stale #49479. (role: reviewer and adjacent owner; confidence: medium; commits: c6d8318d07f5; files: src/auto-reply/reply/agent-runner-memory.ts)
  • stainlu: Authored the earlier stale-total display fix that split resolveSessionTotalTokens from resolveFreshSessionTotalTokens, directly relevant to choosing a display-only helper instead of changing fresh-only semantics. (role: introduced related display/helper boundary; confidence: high; commits: 24b915ed413e; files: src/config/sessions/types.ts)
  • njuboy11: Authored the merged prerequisite fix: prevent persistSessionUsageUpdate from corrupting totalTokens (fixes #82576) #82578 that preserved post-compaction token totals across stale usage updates, which made this follow-up display issue visible. (role: recent area contributor; confidence: high; commits: 6a65ea8c3ac1; files: src/auto-reply/reply/session-usage.ts, src/auto-reply/reply/session.test.ts)

Remaining risk / open question:

  • Changing resolveFreshSessionTotalTokens globally can make runtime callers trust stale totals unless every caller is audited and guarded.
  • The supplied proof does not show /context or the Control UI context meter running from a patched OpenClaw binary.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 451563b95089.

@clawsweeper clawsweeper Bot added P2 Normal backlog priority with limited blast radius. impact:session-state Session, memory, transcript, context, or agent state can drift or corrupt. labels May 17, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 17, 2026
@njuboy11

njuboy11 commented May 17, 2026

Copy link
Copy Markdown
Contributor Author

Note for reviewers: macos-swift failure is a pre-existing CI workflow issue: isCanonicalRepository resolves to true for fork PRs (line 173 of .github/workflows/ci.yml) but macOS signing secrets are unavailable. This check does not run on upstream main commits and is unrelated to this PR's six-file TypeScript change.

@njuboy11 njuboy11 closed this May 17, 2026
@yetval yetval mentioned this pull request Jun 16, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

gateway Gateway runtime impact:session-state Session, memory, transcript, context, or agent state can drift or corrupt. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. size: XS

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

fix: resolveFreshSessionTotalTokens returns undefined when totalTokens is preserved (post-#82578 regression)

1 participant

@njuboy11