Skip to content

fix(reply): flush memory for pending in-turn auto-compaction even when tokens drop below threshold#87088

Open
YOMXXX wants to merge 1 commit into
openclaw:mainfrom
YOMXXX:fix/in-turn-compaction-memory-flush
Open

fix(reply): flush memory for pending in-turn auto-compaction even when tokens drop below threshold#87088
YOMXXX wants to merge 1 commit into
openclaw:mainfrom
YOMXXX:fix/in-turn-compaction-memory-flush

Conversation

@YOMXXX

@YOMXXX YOMXXX commented May 27, 2026

Copy link
Copy Markdown

Summary

  • Make shouldRunMemoryFlush in src/auto-reply/reply/memory-flush.ts honor the explicit compaction.memoryFlush.enabled contract for in-turn auto-compaction: when the session entry's compactionCount has moved past memoryFlushCompactionCount (i.e., there is a compaction that has not yet been memory-flushed), trigger the flush regardless of the current token level. Compaction itself drops tokens back below the soft threshold, so the pure token check would otherwise never fire for in-turn compaction cycles (Bug: in-turn auto-compaction can bypass memoryFlush because memoryFlush only runs pre-turn #62420).
  • The new branch is gated by the existing duplicate guard hasAlreadyFlushedForCurrentCompaction, so a session that has already flushed for its current compaction count is unaffected — no double-flush.
  • The branch is also gated by resolveMemoryFlushGateState, which still requires a fresh totalTokens snapshot (or an explicit tokenCount override) — stale entries do not trigger a catch-up flush by accident.
  • Add 6 colocated cases in src/auto-reply/reply/reply-state.test.ts covering: catch-up after in-turn compaction with low post-compaction tokens, first-compaction catch-up without prior memoryFlushCompactionCount, no catch-up when already flushed for the current count, no catch-up when compactionCount === 0, no catch-up on stale persisted totals, and catch-up honored when a fresh tokenCount override is supplied.

Behavior change to flag

Sessions where compaction.memoryFlush.enabled = true and a runtime-triggered auto-compaction fired during a turn now get their memory file written on the next runMemoryFlushIfNeeded pass (which runs at the next pre-turn runReplyAgent entry), instead of being silently skipped. This restores the documented contract — the same memory/*.md write the user would see for a pre-turn compaction. Sessions that have not had a compaction (or have already flushed for their current compaction count) are completely unaffected: the duplicate guard and the compactionCount > 0 check together preserve the existing semantics for those paths.

The catch-up flush still respects every other gate that already lives in runMemoryFlushIfNeededmemoryFlush.enabled config plan, sandbox write-ability, heartbeat skip, CLI-provider skip — so disabled / heartbeat / CLI sessions remain no-ops.

The timing of the catch-up is "next turn's pre-flush": when in-turn auto-compaction happens during turn N, the memory file gets written at the start of turn N+1. The vast majority of OpenClaw sessions are interactive (there is always a turn N+1), so the audit/persistence semantics the issue describes are restored in practice; a session that goes completely idle right after in-turn compaction would still pick up the flush on the next user message. Synchronous post-turn flushing was considered but rejected for this change: it would require wiring a new runMemoryFlushIfNeeded call into the post-turn block in runReplyAgent (cross-cutting), while the helper-level catch-up is a single conditional with no broader runtime surface change.

Real behavior proof

  • Behavior addressed: shouldRunMemoryFlush now returns true for { totalTokens: 10_000, compactionCount: 1, memoryFlushCompactionCount: 0 } against a 100K context / 5K reserve / 2K soft threshold profile — exactly the in-turn compaction shape from Bug: in-turn auto-compaction can bypass memoryFlush because memoryFlush only runs pre-turn #62420 (compaction reduced tokens below threshold, but memoryFlush.enabled should still cover the compaction event).
  • Real environment tested: local macOS arm64 source checkout post-rebase against upstream/main; behavior verified through (a) 6 new colocated unit cases in reply-state.test.ts driving shouldRunMemoryFlush directly, (b) the full existing reply-state.test.ts suite (now 46 cases) including the original 7 shouldRunMemoryFlush cases that pin pre-existing behavior, and (c) agent-runner-memory.test.ts (34 cases) — the integration that wraps shouldRunMemoryFlush via runMemoryFlushIfNeeded — still passes unchanged. The catch-up triggers from the next turn's existing runMemoryFlushIfNeeded call at src/auto-reply/reply/agent-runner.ts:1352; no new call site is added.
  • Exact steps or command run after this patch:
    • pnpm test src/auto-reply/reply/reply-state.test.ts
    • pnpm test src/auto-reply/reply/agent-runner-memory.test.ts
    • pnpm test:changed
    • pnpm check:changed
  • Evidence after fix:
    • In-turn compaction case (totalTokens: 10_000, compactionCount: 1, memoryFlushCompactionCount: 0) → flush triggers (was false before).
    • First-compaction case with no memoryFlushCompactionCount recorded → flush triggers.
    • Duplicate guard preserved: (compactionCount: 3, memoryFlushCompactionCount: 3)false (no double-flush).
    • Never-compacted case (compactionCount: 0) → falls through to the existing threshold check, unchanged.
    • Stale persisted totals (totalTokensFresh: false) without tokenCount override → still false (we don't blindly flush on stale snapshots).
    • Explicit fresh tokenCount override (tokenCount: 8_000) on an otherwise stale entry with compactionCount: 1 → flush triggers.
  • Observed result after fix:
    • src/auto-reply/reply/reply-state.test.ts: 46 / 46 passed (40 existing + 6 new) in ~1.2 s.
    • src/auto-reply/reply/agent-runner-memory.test.ts: 34 / 34 passed — confirms runMemoryFlushIfNeeded still works end-to-end with the modified helper.
    • pnpm test:changed: 7,260+ tests pass across the affected lanes. The only red is the pre-existing src/tui/tui-pty-local.e2e.test.ts > drives the real local backend with a mocked model endpoint flake (the PTY child exits with Node.js v22.19+ is required (current: v22.14.0)), which reproduces on pristine upstream/main and has no overlap with src/auto-reply/** or any memory-flush path.
    • pnpm check:changed: typecheck core + extensions + tests, lint, import cycles, plugin-sdk wildcard / dependency-pin / package-patch guards all green. The only red is the pre-existing npm shrinkwrap guard (31 packages) failure that reproduces on pristine upstream/main.
  • What was not tested: an actual long-running OpenClaw session showing the in-turn auto-compaction emit, the post-compaction memoryFlushCompactionCount = compactionCount - 1 persisted state, and the next pre-turn flush writing memory/*.md. The fix is verified through the helper unit surface that owns the gate; the production setup (Discord thread session, compaction.mode: safeguard, compaction.memoryFlush.enabled: true, real in-turn auto-compaction) requires running OpenClaw against a live agent that hits the per-turn token threshold mid-stream, which is not reproducible from a Mac source checkout. The reporter's transcript-vs-status-card observation in Bug: in-turn auto-compaction can bypass memoryFlush because memoryFlush only runs pre-turn #62420 is the upstream evidence that the runtime path exists.

Notes

  • The fix sits entirely in shouldRunMemoryFlush, a small pure helper. The runReplyAgent flow is unchanged — no new call site, no signature additions to runMemoryFlushIfNeeded, no new runtime registry surface — so the blast radius is exactly the gate decision.
  • This complements (does not overlap with) Run memory flush before preflight compaction #84792 ("Run memory flush before preflight compaction") and [AI-assisted] fix(agents): resolve CLI runtime in preflight + memory-flush gates #86224 ("resolve CLI runtime in preflight + memory-flush gates"), both of which target the pre-turn path. Bug: in-turn auto-compaction can bypass memoryFlush because memoryFlush only runs pre-turn #62420 explicitly calls out the in-turn path as the gap they do not cover; the helper-level catch-up here is the smallest surgical change that restores the explicit memoryFlush.enabled contract for that path.
  • The "synchronous post-turn flush" alternative (suggested-fix fix: add @lid format support and allowFrom wildcard handling #1 in the issue body) was considered. It would require wiring a second runMemoryFlushIfNeeded call into agent-runner.ts right after if (autoCompactionCount > 0) { incrementRunCompactionCount(...) }, plus plumbing the visible-error-payloads accumulator a second time. The catch-up-on-next-pre-turn approach achieves the same documented end state for interactive sessions with a one-line change to a pure helper, which is why we picked it.
  • Reviewed against root AGENTS.md and CONTRIBUTING.md. No CHANGELOG.md edits (per CONTRIBUTING).

Refs #62420

@openclaw-barnacle openclaw-barnacle Bot added size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 27, 2026
@clawsweeper

clawsweeper Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Keep open: current main and the latest release still skip this below-threshold catch-up, but the PR is not merge-ready because the helper branch broadens session-state behavior beyond the stated in-turn path and the supplied proof is still test-only.

Canonical path: Close this PR as superseded by #32358.

So I’m closing this here and keeping the remaining discussion on #32358.

Review details

Best possible solution:

Close this PR as superseded by #32358.

Do we have a high-confidence way to reproduce the issue?

Yes at the source-helper level: current main returns false below the token threshold before considering an unflushed compaction count. I did not reproduce the full long-running OpenClaw session path in this read-only review, and the PR body says that path was not tested.

Is this the best way to solve the issue?

No, not as submitted. The helper-level catch-up is plausible, but current caller order makes it affect preflight compaction too, so it needs scoped gating or explicit acceptance of the broader semantics before it is the best fix.

Security review:

Security review cleared: The diff only changes in-repo TypeScript gate logic and tests, with no dependency, CI, secret, permission, or supply-chain surface added.

AGENTS.md: found and applied where relevant.

What I checked:

  • linked superseding PR: fix(memoryFlush): guard transcript-size forced flush against repeated runs #32358 (fix(memoryFlush): guard transcript-size forced flush against repeated runs) is merged at 2026-03-03T01:00:18Z.
  • cluster evidence: the durable review links that PR in the work cluster or recommended risk path.
  • no human follow-up: live comments and timeline hydrated by apply contain no non-automation activity after the ClawSweeper review.

Likely related people:

  • Kaspre: Authored the merged PR that fixed memory flush firing every compaction cycle and changed how memoryFlushCompactionCount relates to compactionCount. (role: memory-flush cadence contributor; confidence: high; commits: 84e70ad4168a, f94f9c238201, f2c813cb31a6; files: src/auto-reply/reply/agent-runner-memory.ts, src/auto-reply/reply/reply-state.test.ts)
  • hclsys: Authored the merged PR that extracted hasAlreadyFlushedForCurrentCompaction and applied it to the transcript-size flush path. (role: duplicate-guard contributor; confidence: high; commits: 9690ed1a624a, 503d39578066; files: src/auto-reply/reply/memory-flush.ts, src/auto-reply/reply/agent-runner-memory.ts, src/auto-reply/reply/reply-state.test.ts)
  • Jerry-Xin: Authored recent merged failure-tracking work in the same memory-flush helper and session-state area. (role: recent memory-flush state contributor; confidence: medium; commits: b3737db50244, 4e84d0eaa547; files: src/auto-reply/reply/agent-runner-memory.ts, src/config/sessions/types.ts)
  • vincentkoc: Authored merged work keeping pre-compaction memory-flush prompts out of user transcripts and appears in later memory-flush fixup commits. (role: adjacent memory-flush contributor; confidence: medium; commits: 9f69cb197fba, 7fd9c152d122; files: src/auto-reply/reply/agent-runner-memory.ts, src/auto-reply/reply/agent-runner-memory.test.ts)

Codex review notes: model internal, reasoning high; reviewed against 6de357ad4772.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels May 27, 2026
@clawsweeper

clawsweeper Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 29, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants