fix: distinguish whether base already includes output in token projection#55722
fix: distinguish whether base already includes output in token projection#55722luoshan5261486 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1048152c98
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -604,6 +605,7 @@ export async function runMemoryFlushIfNeeded(params: { | |||
| promptTokensSnapshot, | |||
| transcriptOutputTokens, | |||
| promptTokenEstimate, | |||
| entry?.totalTokensFresh, | |||
There was a problem hiding this comment.
Always include last output for fresh token snapshots
SessionEntry.totalTokens is explicitly treated as a prompt/context-only snapshot (it excludes completion tokens in src/agents/usage.ts:173-175), so passing entry?.totalTokensFresh as baseIncludesOutput here drops transcriptOutputTokens in the common fresh path. That underestimates projected context by the previous assistant reply, which can delay or prevent memory-flush triggering when sessions are near the threshold.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR adds a Confidence Score: 3/5Not safe to merge: the stale-then-upgraded path now silently drops output tokens from the MemoryFlush projection, re-introducing underestimation for sessions whose token data was freshly hydrated from the transcript in the same call. One P1 regression (output tokens dropped on the totalTokensFresh: false → true path) and one P2 no-op finding. The P1 directly undermines the stated goal of the fix — memory flush will still trigger late for sessions in the stale state. src/auto-reply/reply/agent-runner-memory.ts — lines 603-609 (MemoryFlush projection) and lines 362-365 (PreflightCompaction call).
|
| Filename | Overview |
|---|---|
| src/auto-reply/reply/agent-runner-memory.ts | Introduces baseIncludesOutput parameter to fix MemoryFlush double-counting when totalTokensFresh is already true on entry, but introduces a regression when totalTokensFresh starts as false and is mutated to true within the same call, causing output tokens to be dropped. PreflightCompaction change is functionally a no-op. |
Comments Outside Diff (1)
-
src/auto-reply/reply/agent-runner-memory.ts, line 603-609 (link)Regression: output tokens dropped after in-function
totalTokensFreshupgradeWhen
entry.totalTokensFreshis initiallyfalse, the following execution path produces an incorrect projection after this PR:hasFreshPersistedPromptTokens = false(line 507-508, evaluated before the update block)shouldReadTranscript = true→ transcript is read, yieldingtranscriptPromptTokens = X(prompt-only) andtranscriptOutputTokens = Y- The
shouldPersistTranscriptPromptTokensblock (lines 566-593) updatesentrysoentry.totalTokensFresh = true promptTokensSnapshot = X(line 595-598, onlytranscriptPromptTokenscontributes sincehasFreshPersistedPromptTokenswas false)entry?.totalTokensFreshis nowtrue→baseIncludesOutput = true→ outputYis skipped
The projected count is therefore
X + estimateinstead of the correctX + Y + estimate.Before this PR, the same code path called
resolveEffectivePromptTokens(X, Y, estimate)(no fourth argument), giving the correctX + Y + estimate.The root cause is that
entry?.totalTokensFreshat line 608 reflects the in-flight mutation at line 570, not the value that governed howpromptTokensSnapshotwas built. To avoid the mismatch,baseIncludesOutputshould be derived from the freshness state that was actually used to constructpromptTokensSnapshot:const snapshotBaseIncludesOutput = hasFreshPersistedPromptTokens && (persistedPromptTokens ?? 0) >= (transcriptPromptTokens ?? 0); const projectedTokenCount = hasFreshPromptTokensSnapshot ? resolveEffectivePromptTokens( promptTokensSnapshot, transcriptOutputTokens, promptTokenEstimate, snapshotBaseIncludesOutput, ) : undefined;
snapshotBaseIncludesOutputistrueonly whenpromptTokensSnapshotwas taken frompersistedPromptTokens(which already includes output). In the stale-then-upgraded path it evaluates tofalse, sotranscriptOutputTokensis correctly added.Prompt To Fix With AI
This is a comment left during a code review. Path: src/auto-reply/reply/agent-runner-memory.ts Line: 603-609 Comment: **Regression: output tokens dropped after in-function `totalTokensFresh` upgrade** When `entry.totalTokensFresh` is initially `false`, the following execution path produces an incorrect projection after this PR: 1. `hasFreshPersistedPromptTokens = false` (line 507-508, evaluated before the update block) 2. `shouldReadTranscript = true` → transcript is read, yielding `transcriptPromptTokens = X` (prompt-only) and `transcriptOutputTokens = Y` 3. The `shouldPersistTranscriptPromptTokens` block (lines 566-593) updates `entry` so `entry.totalTokensFresh = true` 4. `promptTokensSnapshot = X` (line 595-598, only `transcriptPromptTokens` contributes since `hasFreshPersistedPromptTokens` was false) 5. `entry?.totalTokensFresh` is now `true` → `baseIncludesOutput = true` → output `Y` is **skipped** The projected count is therefore `X + estimate` instead of the correct `X + Y + estimate`. **Before this PR**, the same code path called `resolveEffectivePromptTokens(X, Y, estimate)` (no fourth argument), giving the correct `X + Y + estimate`. The root cause is that `entry?.totalTokensFresh` at line 608 reflects the in-flight mutation at line 570, not the value that governed how `promptTokensSnapshot` was built. To avoid the mismatch, `baseIncludesOutput` should be derived from the freshness state that was actually used to construct `promptTokensSnapshot`: ```typescript const snapshotBaseIncludesOutput = hasFreshPersistedPromptTokens && (persistedPromptTokens ?? 0) >= (transcriptPromptTokens ?? 0); const projectedTokenCount = hasFreshPromptTokensSnapshot ? resolveEffectivePromptTokens( promptTokensSnapshot, transcriptOutputTokens, promptTokenEstimate, snapshotBaseIncludesOutput, ) : undefined; ``` `snapshotBaseIncludesOutput` is `true` only when `promptTokensSnapshot` was taken from `persistedPromptTokens` (which already includes output). In the stale-then-upgraded path it evaluates to `false`, so `transcriptOutputTokens` is correctly added. How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/agent-runner-memory.ts
Line: 603-609
Comment:
**Regression: output tokens dropped after in-function `totalTokensFresh` upgrade**
When `entry.totalTokensFresh` is initially `false`, the following execution path produces an incorrect projection after this PR:
1. `hasFreshPersistedPromptTokens = false` (line 507-508, evaluated before the update block)
2. `shouldReadTranscript = true` → transcript is read, yielding `transcriptPromptTokens = X` (prompt-only) and `transcriptOutputTokens = Y`
3. The `shouldPersistTranscriptPromptTokens` block (lines 566-593) updates `entry` so `entry.totalTokensFresh = true`
4. `promptTokensSnapshot = X` (line 595-598, only `transcriptPromptTokens` contributes since `hasFreshPersistedPromptTokens` was false)
5. `entry?.totalTokensFresh` is now `true` → `baseIncludesOutput = true` → output `Y` is **skipped**
The projected count is therefore `X + estimate` instead of the correct `X + Y + estimate`.
**Before this PR**, the same code path called `resolveEffectivePromptTokens(X, Y, estimate)` (no fourth argument), giving the correct `X + Y + estimate`.
The root cause is that `entry?.totalTokensFresh` at line 608 reflects the in-flight mutation at line 570, not the value that governed how `promptTokensSnapshot` was built. To avoid the mismatch, `baseIncludesOutput` should be derived from the freshness state that was actually used to construct `promptTokensSnapshot`:
```typescript
const snapshotBaseIncludesOutput =
hasFreshPersistedPromptTokens &&
(persistedPromptTokens ?? 0) >= (transcriptPromptTokens ?? 0);
const projectedTokenCount = hasFreshPromptTokensSnapshot
? resolveEffectivePromptTokens(
promptTokensSnapshot,
transcriptOutputTokens,
promptTokenEstimate,
snapshotBaseIncludesOutput,
)
: undefined;
```
`snapshotBaseIncludesOutput` is `true` only when `promptTokensSnapshot` was taken from `persistedPromptTokens` (which already includes output). In the stale-then-upgraded path it evaluates to `false`, so `transcriptOutputTokens` is correctly added.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: src/auto-reply/reply/agent-runner-memory.ts
Line: 362-365
Comment:
**PreflightCompaction fix is a no-op**
The `false` passed as `baseIncludesOutput` here has no effect. `lastOutputTokens` is still `undefined`, so `output = Math.max(0, undefined ?? 0) = 0` regardless of the flag. The calculated projection is identical before and after the PR for this path.
The PR description says this path suffered from "missing output" because `lastOutputTokens` was `undefined`, but that problem remains — the actual output tokens are never sourced for PreflightCompaction. If the intent is to eventually supply `transcriptOutputTokens` here (as done for MemoryFlush), the call site would need a corresponding transcript read and the actual output value passed as the second argument.
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "Merge branch 'openclaw:main' into fix/me..." | Re-trigger Greptile
| const projectedTokenCount = | ||
| typeof transcriptPromptTokens === "number" | ||
| ? resolveEffectivePromptTokens(transcriptPromptTokens, undefined, promptTokenEstimate) | ||
| ? resolveEffectivePromptTokens(transcriptPromptTokens, undefined, promptTokenEstimate, false) | ||
| : undefined; |
There was a problem hiding this comment.
PreflightCompaction fix is a no-op
The false passed as baseIncludesOutput here has no effect. lastOutputTokens is still undefined, so output = Math.max(0, undefined ?? 0) = 0 regardless of the flag. The calculated projection is identical before and after the PR for this path.
The PR description says this path suffered from "missing output" because lastOutputTokens was undefined, but that problem remains — the actual output tokens are never sourced for PreflightCompaction. If the intent is to eventually supply transcriptOutputTokens here (as done for MemoryFlush), the call site would need a corresponding transcript read and the actual output value passed as the second argument.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/agent-runner-memory.ts
Line: 362-365
Comment:
**PreflightCompaction fix is a no-op**
The `false` passed as `baseIncludesOutput` here has no effect. `lastOutputTokens` is still `undefined`, so `output = Math.max(0, undefined ?? 0) = 0` regardless of the flag. The calculated projection is identical before and after the PR for this path.
The PR description says this path suffered from "missing output" because `lastOutputTokens` was `undefined`, but that problem remains — the actual output tokens are never sourced for PreflightCompaction. If the intent is to eventually supply `transcriptOutputTokens` here (as done for MemoryFlush), the call site would need a corresponding transcript read and the actual output value passed as the second argument.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bacd1e182e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ? resolveEffectivePromptTokens(transcriptPromptTokens, undefined, promptTokenEstimate) | ||
| ? resolveEffectivePromptTokens({ | ||
| basePromptTokens: transcriptPromptTokens, | ||
| lastOutputTokens: transcriptOutputTokens, |
There was a problem hiding this comment.
Replace undeclared token variable in preflight projection
runPreflightCompactionIfNeeded now passes transcriptOutputTokens to resolveEffectivePromptTokens, but that identifier is not declared anywhere in this function. When the transcript-fallback path executes (for stale/missing totalTokens with transcript data), this evaluates an undefined symbol and can throw a ReferenceError, aborting the preflight compaction check instead of producing a token projection.
Useful? React with 👍 / 👎.
Bug Description
The
memoryFlushmechanism is designed to write durable memories tomemory/YYYY-MM-DD.mdbefore a session compacts. However, the token projection calculation used by the flush check systematically underestimates the true token count, causingmemory/directories to remain empty even after multiple compaction events.Root Cause
resolveEffectivePromptTokenshad two issues:totalTokensFresh === falsepath:lastOutputTokenswas passed asundefined, so previous round's LLM output (typically 5,000–15,000 tokens) was completely missing from the projection.totalTokensFresh === truepath:totalTokensalready includes previous prompt+output, buttranscriptOutputTokenswas added again, causing double-counting.These two bugs meant flush triggered far later than intended (or not at all), while compaction would fire first.
Fix
Added a
baseIncludesOutput?: booleanparameter toresolveEffectivePromptTokens:baseIncludesOutput === true: skip addinglastOutputTokens(avoids double-counting)baseIncludesOutput === false | undefined: addlastOutputTokens(fixes missing output)Three code paths now produce correct projections:
Files Changed
src/auto-reply/reply/agent-runner-memory.tsFixes: #55679