Skip to content

fix: distinguish whether base already includes output in token projection#55722

Closed
luoshan5261486 wants to merge 1 commit into
openclaw:mainfrom
luoshan5261486:fix/memory-flush-token-projection
Closed

fix: distinguish whether base already includes output in token projection#55722
luoshan5261486 wants to merge 1 commit into
openclaw:mainfrom
luoshan5261486:fix/memory-flush-token-projection

Conversation

@luoshan5261486

Copy link
Copy Markdown

Bug Description

The memoryFlush mechanism is designed to write durable memories to memory/YYYY-MM-DD.md before a session compacts. However, the token projection calculation used by the flush check systematically underestimates the true token count, causing memory/ directories to remain empty even after multiple compaction events.

Root Cause

resolveEffectivePromptTokens had two issues:

  1. totalTokensFresh === false path: lastOutputTokens was passed as undefined, so previous round's LLM output (typically 5,000–15,000 tokens) was completely missing from the projection.
  2. totalTokensFresh === true path: totalTokens already includes previous prompt+output, but transcriptOutputTokens was added again, causing double-counting.

These two bugs meant flush triggered far later than intended (or not at all), while compaction would fire first.

Fix

Added a baseIncludesOutput?: boolean parameter to resolveEffectivePromptTokens:

  • When baseIncludesOutput === true: skip adding lastOutputTokens (avoids double-counting)
  • When baseIncludesOutput === false | undefined: add lastOutputTokens (fixes missing output)

Three code paths now produce correct projections:

Path totalTokensFresh Before After
PreflightCompaction false Missing output ❌ Correct ✅
MemoryFlush true Double-counting ❌ Correct ✅
MemoryFlush false Correct Correct

Files Changed

  • src/auto-reply/reply/agent-runner-memory.ts

Fixes: #55679

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1048152c98

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@@ -604,6 +605,7 @@ export async function runMemoryFlushIfNeeded(params: {
promptTokensSnapshot,
transcriptOutputTokens,
promptTokenEstimate,
entry?.totalTokensFresh,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Always include last output for fresh token snapshots

SessionEntry.totalTokens is explicitly treated as a prompt/context-only snapshot (it excludes completion tokens in src/agents/usage.ts:173-175), so passing entry?.totalTokensFresh as baseIncludesOutput here drops transcriptOutputTokens in the common fresh path. That underestimates projected context by the previous assistant reply, which can delay or prevent memory-flush triggering when sessions are near the threshold.

Useful? React with 👍 / 👎.

@greptile-apps

greptile-apps Bot commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a baseIncludesOutput flag to resolveEffectivePromptTokens and threads it through both MemoryFlush and PreflightCompaction call sites to avoid double-counting output tokens in the token projection used by the flush gate.\n\n- MemoryFlush (fresh path) — correctly fixed: when entry.totalTokensFresh is true on arrival, persistedPromptTokens already encompasses output, so the new baseIncludesOutput = true correctly suppresses re-adding transcriptOutputTokens.\n- MemoryFlush (stale path) — regression introduced: when totalTokensFresh starts as false, the shouldPersistTranscriptPromptTokens block (lines 566-593) mutates entry.totalTokensFresh = true mid-function. Line 608 then reads the mutated value (true) as baseIncludesOutput, but promptTokensSnapshot was built entirely from transcriptPromptTokens (prompt-only). The result is that transcriptOutputTokens is silently dropped — a regression from pre-PR behaviour.\n- PreflightCompaction — no-op: passing false for baseIncludesOutput while lastOutputTokens remains undefined produces identical arithmetic to before.

Confidence Score: 3/5

Not safe to merge: the stale-then-upgraded path now silently drops output tokens from the MemoryFlush projection, re-introducing underestimation for sessions whose token data was freshly hydrated from the transcript in the same call.

One P1 regression (output tokens dropped on the totalTokensFresh: false → true path) and one P2 no-op finding. The P1 directly undermines the stated goal of the fix — memory flush will still trigger late for sessions in the stale state.

src/auto-reply/reply/agent-runner-memory.ts — lines 603-609 (MemoryFlush projection) and lines 362-365 (PreflightCompaction call).

Important Files Changed

Filename Overview
src/auto-reply/reply/agent-runner-memory.ts Introduces baseIncludesOutput parameter to fix MemoryFlush double-counting when totalTokensFresh is already true on entry, but introduces a regression when totalTokensFresh starts as false and is mutated to true within the same call, causing output tokens to be dropped. PreflightCompaction change is functionally a no-op.

Comments Outside Diff (1)

  1. src/auto-reply/reply/agent-runner-memory.ts, line 603-609 (link)

    P1 Regression: output tokens dropped after in-function totalTokensFresh upgrade

    When entry.totalTokensFresh is initially false, the following execution path produces an incorrect projection after this PR:

    1. hasFreshPersistedPromptTokens = false (line 507-508, evaluated before the update block)
    2. shouldReadTranscript = true → transcript is read, yielding transcriptPromptTokens = X (prompt-only) and transcriptOutputTokens = Y
    3. The shouldPersistTranscriptPromptTokens block (lines 566-593) updates entry so entry.totalTokensFresh = true
    4. promptTokensSnapshot = X (line 595-598, only transcriptPromptTokens contributes since hasFreshPersistedPromptTokens was false)
    5. entry?.totalTokensFresh is now truebaseIncludesOutput = true → output Y is skipped

    The projected count is therefore X + estimate instead of the correct X + Y + estimate.

    Before this PR, the same code path called resolveEffectivePromptTokens(X, Y, estimate) (no fourth argument), giving the correct X + Y + estimate.

    The root cause is that entry?.totalTokensFresh at line 608 reflects the in-flight mutation at line 570, not the value that governed how promptTokensSnapshot was built. To avoid the mismatch, baseIncludesOutput should be derived from the freshness state that was actually used to construct promptTokensSnapshot:

    const snapshotBaseIncludesOutput =
      hasFreshPersistedPromptTokens &&
      (persistedPromptTokens ?? 0) >= (transcriptPromptTokens ?? 0);
    
    const projectedTokenCount = hasFreshPromptTokensSnapshot
      ? resolveEffectivePromptTokens(
          promptTokensSnapshot,
          transcriptOutputTokens,
          promptTokenEstimate,
          snapshotBaseIncludesOutput,
        )
      : undefined;

    snapshotBaseIncludesOutput is true only when promptTokensSnapshot was taken from persistedPromptTokens (which already includes output). In the stale-then-upgraded path it evaluates to false, so transcriptOutputTokens is correctly added.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/auto-reply/reply/agent-runner-memory.ts
    Line: 603-609
    
    Comment:
    **Regression: output tokens dropped after in-function `totalTokensFresh` upgrade**
    
    When `entry.totalTokensFresh` is initially `false`, the following execution path produces an incorrect projection after this PR:
    
    1. `hasFreshPersistedPromptTokens = false` (line 507-508, evaluated before the update block)
    2. `shouldReadTranscript = true` → transcript is read, yielding `transcriptPromptTokens = X` (prompt-only) and `transcriptOutputTokens = Y`
    3. The `shouldPersistTranscriptPromptTokens` block (lines 566-593) updates `entry` so `entry.totalTokensFresh = true`
    4. `promptTokensSnapshot = X` (line 595-598, only `transcriptPromptTokens` contributes since `hasFreshPersistedPromptTokens` was false)
    5. `entry?.totalTokensFresh` is now `true``baseIncludesOutput = true` → output `Y` is **skipped**
    
    The projected count is therefore `X + estimate` instead of the correct `X + Y + estimate`.
    
    **Before this PR**, the same code path called `resolveEffectivePromptTokens(X, Y, estimate)` (no fourth argument), giving the correct `X + Y + estimate`.
    
    The root cause is that `entry?.totalTokensFresh` at line 608 reflects the in-flight mutation at line 570, not the value that governed how `promptTokensSnapshot` was built. To avoid the mismatch, `baseIncludesOutput` should be derived from the freshness state that was actually used to construct `promptTokensSnapshot`:
    
    ```typescript
    const snapshotBaseIncludesOutput =
      hasFreshPersistedPromptTokens &&
      (persistedPromptTokens ?? 0) >= (transcriptPromptTokens ?? 0);
    
    const projectedTokenCount = hasFreshPromptTokensSnapshot
      ? resolveEffectivePromptTokens(
          promptTokensSnapshot,
          transcriptOutputTokens,
          promptTokenEstimate,
          snapshotBaseIncludesOutput,
        )
      : undefined;
    ```
    
    `snapshotBaseIncludesOutput` is `true` only when `promptTokensSnapshot` was taken from `persistedPromptTokens` (which already includes output). In the stale-then-upgraded path it evaluates to `false`, so `transcriptOutputTokens` is correctly added.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/agent-runner-memory.ts
Line: 603-609

Comment:
**Regression: output tokens dropped after in-function `totalTokensFresh` upgrade**

When `entry.totalTokensFresh` is initially `false`, the following execution path produces an incorrect projection after this PR:

1. `hasFreshPersistedPromptTokens = false` (line 507-508, evaluated before the update block)
2. `shouldReadTranscript = true` → transcript is read, yielding `transcriptPromptTokens = X` (prompt-only) and `transcriptOutputTokens = Y`
3. The `shouldPersistTranscriptPromptTokens` block (lines 566-593) updates `entry` so `entry.totalTokensFresh = true`
4. `promptTokensSnapshot = X` (line 595-598, only `transcriptPromptTokens` contributes since `hasFreshPersistedPromptTokens` was false)
5. `entry?.totalTokensFresh` is now `true``baseIncludesOutput = true` → output `Y` is **skipped**

The projected count is therefore `X + estimate` instead of the correct `X + Y + estimate`.

**Before this PR**, the same code path called `resolveEffectivePromptTokens(X, Y, estimate)` (no fourth argument), giving the correct `X + Y + estimate`.

The root cause is that `entry?.totalTokensFresh` at line 608 reflects the in-flight mutation at line 570, not the value that governed how `promptTokensSnapshot` was built. To avoid the mismatch, `baseIncludesOutput` should be derived from the freshness state that was actually used to construct `promptTokensSnapshot`:

```typescript
const snapshotBaseIncludesOutput =
  hasFreshPersistedPromptTokens &&
  (persistedPromptTokens ?? 0) >= (transcriptPromptTokens ?? 0);

const projectedTokenCount = hasFreshPromptTokensSnapshot
  ? resolveEffectivePromptTokens(
      promptTokensSnapshot,
      transcriptOutputTokens,
      promptTokenEstimate,
      snapshotBaseIncludesOutput,
    )
  : undefined;
```

`snapshotBaseIncludesOutput` is `true` only when `promptTokensSnapshot` was taken from `persistedPromptTokens` (which already includes output). In the stale-then-upgraded path it evaluates to `false`, so `transcriptOutputTokens` is correctly added.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/auto-reply/reply/agent-runner-memory.ts
Line: 362-365

Comment:
**PreflightCompaction fix is a no-op**

The `false` passed as `baseIncludesOutput` here has no effect. `lastOutputTokens` is still `undefined`, so `output = Math.max(0, undefined ?? 0) = 0` regardless of the flag. The calculated projection is identical before and after the PR for this path.

The PR description says this path suffered from "missing output" because `lastOutputTokens` was `undefined`, but that problem remains — the actual output tokens are never sourced for PreflightCompaction. If the intent is to eventually supply `transcriptOutputTokens` here (as done for MemoryFlush), the call site would need a corresponding transcript read and the actual output value passed as the second argument.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Merge branch 'openclaw:main' into fix/me..." | Re-trigger Greptile

Comment on lines 362 to 365
const projectedTokenCount =
typeof transcriptPromptTokens === "number"
? resolveEffectivePromptTokens(transcriptPromptTokens, undefined, promptTokenEstimate)
? resolveEffectivePromptTokens(transcriptPromptTokens, undefined, promptTokenEstimate, false)
: undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 PreflightCompaction fix is a no-op

The false passed as baseIncludesOutput here has no effect. lastOutputTokens is still undefined, so output = Math.max(0, undefined ?? 0) = 0 regardless of the flag. The calculated projection is identical before and after the PR for this path.

The PR description says this path suffered from "missing output" because lastOutputTokens was undefined, but that problem remains — the actual output tokens are never sourced for PreflightCompaction. If the intent is to eventually supply transcriptOutputTokens here (as done for MemoryFlush), the call site would need a corresponding transcript read and the actual output value passed as the second argument.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/auto-reply/reply/agent-runner-memory.ts
Line: 362-365

Comment:
**PreflightCompaction fix is a no-op**

The `false` passed as `baseIncludesOutput` here has no effect. `lastOutputTokens` is still `undefined`, so `output = Math.max(0, undefined ?? 0) = 0` regardless of the flag. The calculated projection is identical before and after the PR for this path.

The PR description says this path suffered from "missing output" because `lastOutputTokens` was `undefined`, but that problem remains — the actual output tokens are never sourced for PreflightCompaction. If the intent is to eventually supply `transcriptOutputTokens` here (as done for MemoryFlush), the call site would need a corresponding transcript read and the actual output value passed as the second argument.

How can I resolve this? If you propose a fix, please make it concise.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bacd1e182e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

? resolveEffectivePromptTokens(transcriptPromptTokens, undefined, promptTokenEstimate)
? resolveEffectivePromptTokens({
basePromptTokens: transcriptPromptTokens,
lastOutputTokens: transcriptOutputTokens,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Replace undeclared token variable in preflight projection

runPreflightCompactionIfNeeded now passes transcriptOutputTokens to resolveEffectivePromptTokens, but that identifier is not declared anywhere in this function. When the transcript-fallback path executes (for stale/missing totalTokens with transcript data), this evaluates an undefined symbol and can throw a ReferenceError, aborting the preflight compaction check instead of producing a token projection.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: memoryFlush never triggers — token projection underestimates by ignoring output tokens

1 participant