fix(agents): preserve re-sent user prompt during compaction transcript rotation by yetval · Pull Request #93732 · openclaw/openclaw

yetval · 2026-06-16T17:57:54Z

Summary

With agents.defaults.compaction.truncateAfterCompaction enabled, a compaction can silently drop the user's most recent request from the successor transcript. The agent then continues having lost what the user asked for.

The trigger is a re-sent prompt: the user sends prompt P, the agent responds, the conversation continues, and the user re-sends an identical P (>=24 chars, within the 60s window). When compaction then fires with the boundary falling between the two copies, both copies are removed and P never appears in the successor.

Root cause

buildSuccessorEntries in src/agents/embedded-agent-runner/compaction-successor-transcript.ts collects ids to drop:

// before
const removedIds = new Set<string>();
const duplicateUserMessageIds = collectDuplicateUserMessageEntryIdsForCompaction(branch);
for (const entry of allEntries) {
  if (
    (summarizedBranchIds.has(entry.id) && entry.type === "message") ||
    staleStateEntryIds.has(entry.id) ||
    duplicateUserMessageIds.has(entry.id)
  ) {
    removedIds.add(entry.id);
  }
}

collectDuplicateUserMessageEntryIdsForCompaction (src/agents/embedded-agent-runner/compaction-duplicate-user-messages.ts) keeps the first occurrence of a user message and flags later same-window repeats. Running it over the full branch means: when P's first copy lives in the summarized prefix and the later copy lives in the kept tail, the helper treats the kept-tail copy as the duplicate and adds it to duplicateUserMessageIds.

That kept-tail copy is then removed via duplicateUserMessageIds, while the earlier copy is independently removed via summarizedBranchIds (summarized prefix message). Both verbatim instances are gone. The compaction summary is generic prose that does not contain the prompt text, so the request is lost entirely.

Fix

Anchor dedup within the kept region only. The summarized prefix is already dropped via summarizedBranchIds, so the dedup pass should not consider those entries when deciding which kept-tail copy is the "first" surviving instance:

// after
const keptBranchEntries = branch.filter((entry) => !summarizedBranchIds.has(entry.id));
const duplicateUserMessageIds =
  collectDuplicateUserMessageEntryIdsForCompaction(keptBranchEntries);

Now the first surviving copy in the kept tail is preserved; only genuine later repeats within the kept region are still dropped, which is exactly the deduplication the helper was added for.

Why this is the right boundary

The defect is in how buildSuccessorEntries feeds the helper, not in the helper itself. collectDuplicateUserMessageEntryIdsForCompaction correctly keeps the first occurrence and drops later repeats; it was just being shown entries (the summarized prefix) that are already removed by a different rule, so its "first occurrence" anchored on a copy that no longer survives. Filtering the input to the kept region fixes the input contract without changing the helper's semantics.
The other two removal rules are untouched: summarized-prefix messages and stale state entries are still removed exactly as before. Only the duplicate-detection input is narrowed.
Within-kept-region dedup still works: a prompt re-sent twice inside the kept tail still drops the later copy, so the original duplicate-inflation fix is preserved.

Verification

node scripts/run-vitest.mjs src/agents/embedded-agent-runner/compaction-successor-transcript.duplicate-prompt-loss.test.ts -> 1 passed. This new regression test fails on pristine main (expected false to be true) and passes with this patch.
node scripts/run-vitest.mjs src/agents/embedded-agent-runner/compaction-successor-transcript.test.ts -> existing 12 tests still pass.
node scripts/run-vitest.mjs src/agents/embedded-agent-runner/compaction-duplicate-user-messages.test.ts -> 3 tests still pass.
oxfmt --check on the changed files -> clean.
run-oxlint.mjs on the changed files -> 0 findings.
tsgo core (tsconfig.core.json) and core test (tsconfig.core.test.json) -> clean.

Real behavior proof

Behavior addressed: with truncateAfterCompaction enabled, a re-sent user prompt whose earlier copy lands in the summarized prefix and whose later copy lands in the kept tail is no longer dropped from the successor transcript.

Real environment tested: drove the real production function rotateTranscriptAfterCompaction end to end against a real SessionManager, building the transcript (prompt, assistant, "go on", assistant, compaction at that boundary, re-sent identical prompt, assistant), then read the written successor file back with the real readTranscriptFileState and extracted the successor's user turns. Run against pristine main and the patched branch with identical inputs.

Exact steps or command run after this patch: append the two-copy transcript through SessionManager, call rotateTranscriptAfterCompaction({ sessionManager, sessionFile }), load the resulting successor file, and list the role === "user" turn texts.

Evidence after fix:

# BEFORE (pristine main d1e20d2f29)
SUCCESSOR_USER_TURNS=["go on"]
PROMPT_PRESENT=false

# AFTER (this patch, identical inputs)
SUCCESSOR_USER_TURNS=["go on","Please refactor the authentication module right now"]
PROMPT_PRESENT=true

Observed result after fix: the successor transcript that previously contained only "go on" (the actual request "Please refactor the authentication module right now" absent) now retains the request.

What was not tested: a live gateway run that triggers real model-driven compaction with truncateAfterCompaction set; this change is at the transcript-rotation layer and is covered above by driving the real rotation/read functions plus the colocated regression test.

…otation Duplicate user-message detection ran over the full branch, so when a prompt was re-sent within the 60s window its earlier copy in the summarized prefix and the later copy in the kept tail were both removed: the summarized copy via summarizedBranchIds and the kept copy as a duplicate. With truncateAfterCompaction enabled the prompt then vanished from the successor transcript entirely. Restrict dedup to the kept region so the first surviving copy is preserved.

clawsweeper · 2026-06-16T18:34:18Z

Codex review: needs maintainer review before merge. Reviewed June 16, 2026, 2:41 PM ET / 18:41 UTC.

Summary
The PR narrows duplicate user-message pruning during compaction transcript rotation to the unsummarized kept branch entries and adds a regression test for preserved re-sent prompts.

PR surface: Source +2, Tests +79. Total +81 across 2 files.

Reproducibility: yes. source-reproducible: current main removes summarized-prefix messages and duplicate ids computed from the full branch, so a first copy in the summarized prefix and a re-sent kept-tail copy can both disappear. I did not execute tests because this review was constrained to read-only inspection.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] A live model-driven gateway compaction was not exercised in this read-only review; the supplied proof covers the production rotate/read layer touched by the patch.

Maintainer options:

Decide the mitigation before merge
Land the narrow caller-side filter with regression coverage after normal CI/maintainer review, keeping duplicate pruning semantics unchanged for duplicate retries that both survive in the successor transcript.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No repair job is needed; the PR is a small correct implementation awaiting normal maintainer review and CI.

Security
Cleared: The diff only changes transcript pruning logic and adds a Vitest regression test; it does not touch secrets, dependencies, CI, package metadata, or external execution paths.

Review details

Best possible solution:

Land the narrow caller-side filter with regression coverage after normal CI/maintainer review, keeping duplicate pruning semantics unchanged for duplicate retries that both survive in the successor transcript.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main removes summarized-prefix messages and duplicate ids computed from the full branch, so a first copy in the summarized prefix and a re-sent kept-tail copy can both disappear. I did not execute tests because this review was constrained to read-only inspection.

Is this the best way to solve the issue?

Yes; filtering the duplicate collector input in buildSuccessorEntries is the best narrow boundary because the helper still drops true duplicates inside the kept region and the separate compaction-input dedupe path remains unchanged.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 617c9d4b7fb4.

Label changes

Label justifications:

P1: The PR addresses a silent loss of a recent user request from successor agent transcripts in a shipped opt-in compaction path.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body supplies structured before/after terminal output from real SessionManager rotation and transcript reading, showing the prompt absent before and present after the patch.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies structured before/after terminal output from real SessionManager rotation and transcript reading, showing the prompt absent before and present after the patch.

Evidence reviewed

PR surface:

Source +2, Tests +79. Total +81 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	3	1	+2
Tests	1	79	0	+79
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	82	1	+81

What I checked:

Repository policy read: Read the full root AGENTS.md and scoped src/agents/AGENTS.md; the review followed the required full-path, proof, and owner-history checks for agent runtime changes. (AGENTS.md:1, 617c9d4b7fb4)
Current-main bug source: On current main, buildSuccessorEntries computes summarizedBranchIds and then calls collectDuplicateUserMessageEntryIdsForCompaction(branch), so summarized-prefix messages still anchor duplicate detection even though summarized messages are later removed. (src/agents/embedded-agent-runner/compaction-successor-transcript.ts:177, 617c9d4b7fb4)
Duplicate helper contract: collectDuplicateUserMessageEntryIdsForCompaction keeps the first same-window long user prompt and adds later matching entry ids to duplicateIds, which explains why a kept-tail copy is removed when the first copy was in the summarized prefix. (src/agents/embedded-agent-runner/compaction-duplicate-user-messages.ts:90, 617c9d4b7fb4)
Patch boundary: The PR head filters branch to entries not in summarizedBranchIds before duplicate collection and leaves the helper semantics unchanged; the added test drives rotateTranscriptAfterCompaction and reads the successor transcript back. (src/agents/embedded-agent-runner/compaction-successor-transcript.ts:174, 0ccc8f269652)
Runtime callers checked: Rotation is called from the embedded compaction path, queued context-engine compaction path, and automatic run attempt path only after successful compaction when truncateAfterCompaction is enabled, so the change is at the shared successor transcript builder boundary. (src/agents/embedded-agent-runner/compact.ts:1479, 617c9d4b7fb4)
Shipped behavior still affected: The latest release tag v2026.6.8 contains the same full-branch duplicate collection in compaction-successor-transcript.ts, so the PR is not obsolete on shipped code. (src/agents/embedded-agent-runner/compaction-successor-transcript.ts, 844f405ac1be)

Likely related people:

steipete: Authored the merged compacted session transcript rotation PR that introduced the successor transcript feature and related runtime plumbing. (role: feature introducer; confidence: high; commits: c88b1e873780, 296fafd716e4, cfbf15644453; files: src/agents/embedded-agent-runner/compaction-successor-transcript.ts, src/agents/embedded-agent-runner/compact.ts, docs/concepts/compaction.md)
pashpashpash: Carried multiple merged follow-ups for successor transcript rotation behavior, labels, ordering, and review feedback in the same feature area. (role: follow-up owner; confidence: high; commits: d71ac480608b, 8cad8e6f3198, 45a108ee5f46; files: src/agents/embedded-agent-runner/compaction-successor-transcript.ts, src/agents/embedded-agent-runner/compaction-successor-transcript.test.ts)
openperf: Recently merged same-file compaction successor work for stale thinking signatures and replay correctness, making them relevant for adjacent review context. (role: recent adjacent contributor; confidence: medium; commits: a351e8b848d2; files: src/agents/embedded-agent-runner/compaction-successor-transcript.ts, src/agents/embedded-agent-runner/replay-history.ts, src/agents/embedded-agent-runner/thinking.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

vincentkoc · 2026-06-16T19:49:08Z

Maintainer verification complete. Whole-path review confirms that anchoring duplicate collection to the kept compaction branch is the best fix: it preserves the re-sent request without changing summarized-prefix or within-tail dedup behavior.

Verified:

node scripts/run-vitest.mjs src/agents/embedded-agent-runner/compaction-successor-transcript.duplicate-prompt-loss.test.ts src/agents/embedded-agent-runner/compaction-successor-transcript.test.ts src/agents/embedded-agent-runner/compaction-duplicate-user-messages.test.ts — 16 passed
git diff --check — clean
fresh structured autoreview — clean
PR CI — 136 passed, 0 pending, 0 failed

Ready to squash merge.

…otation (openclaw#93732) Duplicate user-message detection ran over the full branch, so when a prompt was re-sent within the 60s window its earlier copy in the summarized prefix and the later copy in the kept tail were both removed: the summarized copy via summarizedBranchIds and the kept copy as a duplicate. With truncateAfterCompaction enabled the prompt then vanished from the successor transcript entirely. Restrict dedup to the kept region so the first surviving copy is preserved.

Under truncateAfterCompaction, buildSuccessorEntries ran one 60s duplicate-user dedup window across both the kept pre-compaction tail and all post-compaction entries. A kept-tail prompt became the dedup first-seen and suppressed a genuine post-compaction re-issue of the same text, deleting the user instruction and orphaning its assistant reply (two consecutive assistant messages with no parent user turn), which strict providers reject on the next request. Dedup the pre-compaction kept tail and the post-compaction tail independently so the window resets at the compaction boundary. Each side still drops its own intra-window retries; a kept-tail prompt no longer reaches across the boundary. Regression of openclaw#93732, which narrowed the dedup input to keptBranchEntries but left it spanning the boundary.

openclaw-barnacle Bot added agents Agent runtime and tooling size: S proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 16, 2026

yetval force-pushed the fix/compaction-successor-duplicate-prompt-loss branch from cf41dd9 to 0ccc8f2 Compare June 16, 2026 18:12

vincentkoc self-assigned this Jun 16, 2026

vincentkoc merged commit fa65b32 into openclaw:main Jun 16, 2026
194 of 199 checks passed

github-actions Bot mentioned this pull request Jun 16, 2026

📡 Upstream Digest — 2026-06-16 22:06 UTC curtismercier/openclaw-mods#1090

Open

yetval mentioned this pull request Jun 18, 2026

fix(agents): keep post-compaction user re-issue of a kept-tail prompt during compaction rotation #94328

Merged

clawsweeper Bot mentioned this pull request Jun 18, 2026

Feishu replies disappear from webchat after compaction rotation (buildSuccessorEntries drops assistant messages) #76729

Closed

LiuwqGit mentioned this pull request Jun 19, 2026

fix(agents): preserve last assistant reply before compaction boundary (#76729) #94720

Closed

clawsweeper Bot mentioned this pull request Jul 7, 2026

fix(agents): Anthropic replay fails after boundaryless compaction #98527

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): preserve re-sent user prompt during compaction transcript rotation#93732

fix(agents): preserve re-sent user prompt during compaction transcript rotation#93732
vincentkoc merged 1 commit into
openclaw:mainfrom
yetval:fix/compaction-successor-duplicate-prompt-loss

yetval commented Jun 16, 2026

Uh oh!

clawsweeper Bot commented Jun 16, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

yetval commented Jun 16, 2026

Summary

Root cause

Fix

Why this is the right boundary

Verification

Real behavior proof

Uh oh!

clawsweeper Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 16, 2026 •

edited

Loading