fix(agents): persist embedded session JSONL transcripts (#77823) by neeravmakwana · Pull Request #77839 · openclaw/openclaw

neeravmakwana · 2026-05-05T12:45:50Z

Summary

Treat executionTrace.runner: "embedded" like CLI for post-turn session transcript persistence and the existing CLI-named embedded-aware compaction lifecycle so assistant-visible text is written to session JSONL (regression in 2026.5.3+ for webchat/Feishu-style embedded runs).

Root cause

agent-command only called persistCliTurnTranscript when runner === "cli". The PI embedded path tags runs with runner: "embedded", so user turns could be recorded elsewhere while the assistant summary line for the turn never hit the transcript writer.

Why this is safe

Reuses the existing persistCliTurnTranscript + persistTextTurnTranscript path already used for CLI; no new persistence format or trust boundary.
ACP turns still take the dedicated early-return path (persistAcpTurnTranscript); this change does not alter ACP behavior.
Compaction hook was already implemented for embedded session files (createPreparedEmbeddedPiSettingsManager, etc.); we only widen when it runs to match transcript persistence.

Security / runtime controls

Unchanged: auth, model allowlists, tool policy, channel delivery, and session store access controls. This only runs the existing transcript-append and compaction helpers after a successful embedded run completes, keyed to the same session as before.

Real behavior proof

Not run in this environment (no live gateway/webchat stack here). Validation: focused unit test exercises the agentCommand path and asserts persistCliTurnTranscript is invoked once with executionTrace.runner === "embedded" and the user body passed through.

Tests run

git diff --check
pnpm vitest run src/commands/agent.test.ts
pnpm check:changed — typecheck core failed on main as well (Cannot find module 'web-tree-sitter') in this checkout; unrelated to this diff.

Out of scope

Renaming persistCliTurnTranscript / runCliTurnCompactionLifecycle (behavior is already shared with embedded Pi).
Broader transcript model for subagents or non-embedded runners without a structured EmbeddedPiRunResult.

Fixes #77823.

AI-assisted implementation; human review recommended.

Made with Cursor

clawsweeper · 2026-05-05T12:49:17Z

Codex review: needs maintainer review before merge.

Summary
The PR widens post-turn transcript persistence and compaction to embedded runner results, adds assistant-only tail dedupe for embedded gap-fill, updates regression tests, and adds a changelog entry.

Reproducibility: yes. by source inspection: current main gates post-turn transcript persistence to runner === "cli", while the embedded Pi runner returns runner: "embedded". I did not execute a live embedded WebChat or Feishu session in this read-only review.

Real behavior proof
Override: The PR has proof: override, so the usual external-contributor live behavior proof gate is waived for this review.

Next step before merge
No ClawSweeper repair lane is needed because this open PR is the implementation path and the review found no concrete patch bug to fix automatically.

Security
Cleared: Cleared: the diff touches existing transcript persistence, tests, and changelog only, with no dependency, workflow, permission, script, package, or secret-handling changes.

Review details

Best possible solution:

Merge this PR after normal maintainer review and required checks confirm the embedded gap-fill appends assistant text exactly once while preserving Pi-owned user/tool transcript rows.

Do we have a high-confidence way to reproduce the issue?

Yes by source inspection: current main gates post-turn transcript persistence to runner === "cli", while the embedded Pi runner returns runner: "embedded". I did not execute a live embedded WebChat or Feishu session in this read-only review.

Is this the best way to solve the issue?

Yes: the PR uses the existing transcript append path, limits embedded handling to assistant gap-fill, and dedupes only when the physical transcript tail already contains equivalent assistant text. That is narrower than adding a new transcript format or changing Pi's ownership model.

What I checked:

Current main runner gate: Current main only calls persistCliTurnTranscript when result.meta.executionTrace?.runner === "cli", so embedded runner results skip this post-turn transcript mirror path. (src/agents/agent-command.ts:1199, 77ae06bfaa01)
Embedded runner metadata: The embedded Pi runner marks successful runs with executionTrace.runner: "embedded", which does not satisfy the current main CLI-only gate. (src/agents/pi-embedded-runner/run.ts:2794, 77ae06bfaa01)
Transcript ownership contract: WebChat docs say Pi owns normal session JSONL persistence and any fallback mirror must avoid duplicating a Pi-written assistant turn. Public docs: docs/web/webchat.md. (docs/web/webchat.md:52, 77ae06bfaa01)
PR embedded gate: The PR changes the persistence gate to accept cli or embedded and passes embeddedAssistantGapFill only for embedded runner results. (src/agents/agent-command.ts:1196, 0c3412387133)
PR gap-fill behavior: The PR forces embedded gap-fill to skip user prompt mirroring and dedupes assistant appends against the physical transcript tail assistant text. (src/agents/command/attempt-execution.ts:225, 0c3412387133)
Regression coverage: The PR adds tests for embedded assistant gap-fill skipping user mirroring, deduping identical assistant tails, and appending repeated assistant text after a user tail. (src/agents/command/attempt-execution.cli.test.ts:419, 0c3412387133)

Likely related people:

steipete: Recent blame/history place Peter Steinberger on the current main transcript gate area, async text-turn append work, and the WebChat transcript persistence docs that define the Pi ownership contract. (role: recent maintainer and transcript docs owner; confidence: high; commits: 18405c1acf1d, 419bcd26f079, b08220446a39; files: src/agents/agent-command.ts, src/config/sessions/transcript.ts, docs/web/webchat.md)
Ayaan Zaidi: History shows Ayaan Zaidi introduced the CLI transcript persistence path and related session metadata normalization that this PR extends to embedded runs. (role: introduced related behavior; confidence: high; commits: b8ef507cc082, 898fd0482a40, 3a3fae0eac0f; files: src/agents/agent-command.ts, src/agents/command/attempt-execution.ts)
andyylin: Prior delivery-mirror dedupe work in the session transcript area is relevant to this PR's assistant-tail duplicate prevention pattern. (role: adjacent dedupe contributor; confidence: medium; commits: e95efa437381, d842ec417924; files: src/config/sessions/transcript.ts, CHANGELOG.md)
BunsDev: Recent WebUI/session work touched the WebChat/session history surface affected when embedded assistant turns are missing from persisted JSONL. (role: adjacent WebChat/session maintainer; confidence: medium; commits: 05c9492bff0f; files: docs/web/webchat.md, src/config/sessions/transcript.ts)

Remaining risk / open question:

No live WebChat or Feishu-style embedded session was executed during this read-only review; the PR carries proof: override for that gate.
I did not run the PR's tests locally in this read-only review, so verification is from source, patch inspection, provided PR context, and the contributor's stated test commands.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 77ae06bfaa01.

neeravmakwana · 2026-05-05T13:03:02Z

Follow-up on AI review (ClawSweeper / Codex)

What changed since the prior revision

The concern was correct: blindly widening persistCliTurnTranscript to every embedded run mirrors both user + assistant while Pi already persists ordinary embedded turns via SessionManager, risking duplicate JSONL rows.

This revision narrows embedded handling:

embeddedAssistantGapFill — Passed only when executionTrace.runner === "embedded". It stops re-mirroring the user prompt (body / transcriptBody are forced empty for persistence) because Pi already owns user/tool rows.
Assistant tail dedupe — Before appending assistant text, persistTextTurnTranscript compares normalized reply text (resolveCliTranscriptReplyText) against readLatestAssistantTextFromSessionTranscript. If the latest assistant visible text already matches (whitespace-collapsed), we skip the append so we do not duplicate Pi-written final assistant turns.
CLI path unchanged — embeddedAssistantGapFill defaults false; CLI runners keep the existing full-turn mirror behavior.

Tests

attempt-execution.cli.test.ts: regression “embedded assistant gap-fill skips user mirror and dedupes identical assistant tails”.
agent.test.ts: still asserts persistCliTurnTranscript runs for embedded-marked results and now expects embeddedAssistantGapFill: true; import order cleaned up.

Still outstanding

Real behavior proof on a live gateway / WebChat or Feishu-style embedded session remains the right merge gate per CONTRIBUTING; this environment cannot supply it.

Security / controls

No change to auth, tool policy, channel routing, or session ACLs — persistence-only logic behind existing locks.

Commits: latest 87b3abc6dc on fix/embedded-session-transcript-77823.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/25387723812
Updated: 2026-05-05T16:07:38.905Z

) Run persistCliTurnTranscript and post-turn compaction for executionTrace.runner embedded, matching CLI turns so assistant text reaches session JSONL for webchat/Feishu-style runs. Co-authored-by: Cursor <[email protected]>

…penclaw#77823) Embedded runs pass embeddedAssistantGapFill so persistCliTurnTranscript skips re-appending the user prompt Pi owns and only appends assistant text when the transcript tail lacks equivalent visible assistant content. Adds CLI transcript regression coverage for gap-fill dedupe. Co-authored-by: Cursor <[email protected]>

…eeravmakwana

…eeravmakwana)

obviyus

Verified the embedded-runner transcript persistence path and the assistant-only gap-fill, including the repeated-reply case where a newer user tail must not suppress the assistant append.

Maintainer follow-up: tightened dedupe to the transcript tail, preserved the existing latest-assistant helper for /tts latest, fixed the active changelog entry, and applied the real-behavior-proof override for this maintainer-verified path.

Local gate: pnpm test src/agents/command/attempt-execution.cli.test.ts src/commands/agent.test.ts -- --reporter=verbose; OPENCLAW_LOCAL_CHECK=1 OPENCLAW_LOCAL_CHECK_MODE=throttled pnpm check:changed.

obviyus · 2026-05-05T16:05:26Z

Landed on main.

Landed source commit: 3f7d994
Merge commit: 63ce0ca

Thanks @neeravmakwana.

bryan-chx · 2026-05-05T23:23:27Z

Live verification (2026-05-06)

Local patch applied and tested on v2026.5.2. Confirmed:

Embedded assistant text now persisted — Session JSONL shows text content blocks in assistant messages where previously they were missing
No duplicate entries — Tail dedupe correctly skips re-appending when the latest JSONL entry already has equivalent assistant text
No data loss — Pure tool-call messages (no text) correctly remain without text

Patch script: ~/Documents/Workspace-naicha/脚本/patch-v2026.5.2-embedded-transcript.js

Waiting for v2026.5.5 release for official distribution.

…eeravmakwana

…eeravmakwana) * fix(agents): persist embedded runner session transcripts (openclaw#77823) Run persistCliTurnTranscript and post-turn compaction for executionTrace.runner embedded, matching CLI turns so assistant text reaches session JSONL for webchat/Feishu-style runs. Co-authored-by: Cursor <[email protected]> * fix(agents): narrow embedded transcript mirror with assistant dedupe (openclaw#77823) Embedded runs pass embeddedAssistantGapFill so persistCliTurnTranscript skips re-appending the user prompt Pi owns and only appends assistant text when the transcript tail lacks equivalent visible assistant content. Adds CLI transcript regression coverage for gap-fill dedupe. Co-authored-by: Cursor <[email protected]> * fix(agents): dedupe embedded transcript gap fill by tail * fix: persist embedded session transcripts (openclaw#77839) (thanks @neeravmakwana) --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Ayaan Zaidi <[email protected]>

…eeravmakwana

…eeravmakwana) * fix(agents): persist embedded runner session transcripts (openclaw#77823) Run persistCliTurnTranscript and post-turn compaction for executionTrace.runner embedded, matching CLI turns so assistant text reaches session JSONL for webchat/Feishu-style runs. Co-authored-by: Cursor <[email protected]> * fix(agents): narrow embedded transcript mirror with assistant dedupe (openclaw#77823) Embedded runs pass embeddedAssistantGapFill so persistCliTurnTranscript skips re-appending the user prompt Pi owns and only appends assistant text when the transcript tail lacks equivalent visible assistant content. Adds CLI transcript regression coverage for gap-fill dedupe. Co-authored-by: Cursor <[email protected]> * fix(agents): dedupe embedded transcript gap fill by tail * fix: persist embedded session transcripts (openclaw#77839) (thanks @neeravmakwana) --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Ayaan Zaidi <[email protected]>

…eeravmakwana

…eeravmakwana) * fix(agents): persist embedded runner session transcripts (openclaw#77823) Run persistCliTurnTranscript and post-turn compaction for executionTrace.runner embedded, matching CLI turns so assistant text reaches session JSONL for webchat/Feishu-style runs. * fix(agents): narrow embedded transcript mirror with assistant dedupe (openclaw#77823) Embedded runs pass embeddedAssistantGapFill so persistCliTurnTranscript skips re-appending the user prompt Pi owns and only appends assistant text when the transcript tail lacks equivalent visible assistant content. Adds CLI transcript regression coverage for gap-fill dedupe. * fix(agents): dedupe embedded transcript gap fill by tail * fix: persist embedded session transcripts (openclaw#77839) (thanks @neeravmakwana) --------- Co-authored-by: Ayaan Zaidi <[email protected]>

…eeravmakwana

…eeravmakwana) * fix(agents): persist embedded runner session transcripts (openclaw#77823) Run persistCliTurnTranscript and post-turn compaction for executionTrace.runner embedded, matching CLI turns so assistant text reaches session JSONL for webchat/Feishu-style runs. Co-authored-by: Cursor <[email protected]> * fix(agents): narrow embedded transcript mirror with assistant dedupe (openclaw#77823) Embedded runs pass embeddedAssistantGapFill so persistCliTurnTranscript skips re-appending the user prompt Pi owns and only appends assistant text when the transcript tail lacks equivalent visible assistant content. Adds CLI transcript regression coverage for gap-fill dedupe. Co-authored-by: Cursor <[email protected]> * fix(agents): dedupe embedded transcript gap fill by tail * fix: persist embedded session transcripts (openclaw#77839) (thanks @neeravmakwana) --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Ayaan Zaidi <[email protected]>

neeravmakwana mentioned this pull request May 5, 2026

[Bug]: embedded-runtime assistant replies missing from session JSONL — regression in 5.3 and 5.4 #77823

Closed

openclaw-barnacle Bot added commands Command implementations agents Agent runtime and tooling size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 5, 2026

openclaw-barnacle Bot added size: S and removed size: XS labels May 5, 2026

obviyus force-pushed the fix/embedded-session-transcript-77823 branch from 87b3abc to f001d5c Compare May 5, 2026 15:32

openclaw-barnacle Bot added size: M and removed size: S labels May 5, 2026

obviyus self-assigned this May 5, 2026

obviyus force-pushed the fix/embedded-session-transcript-77823 branch from f001d5c to 29cba27 Compare May 5, 2026 15:46

obviyus added proof: override Maintainer override for the external PR real behavior proof gate. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 5, 2026

obviyus force-pushed the fix/embedded-session-transcript-77823 branch from 29cba27 to 54ae30a Compare May 5, 2026 15:53

obviyus force-pushed the fix/embedded-session-transcript-77823 branch from 54ae30a to 0e1170f Compare May 5, 2026 15:55

obviyus force-pushed the fix/embedded-session-transcript-77823 branch from 0e1170f to 0c34123 Compare May 5, 2026 15:57

neeravmakwana and others added 4 commits May 5, 2026 21:33

fix(agents): dedupe embedded transcript gap fill by tail

e8e8107

fix: persist embedded session transcripts (openclaw#77839) (thanks @n…

3f7d994

…eeravmakwana)

obviyus force-pushed the fix/embedded-session-transcript-77823 branch from 0c34123 to 3f7d994 Compare May 5, 2026 16:04

obviyus approved these changes May 5, 2026

View reviewed changes

obviyus merged commit 63ce0ca into openclaw:main May 5, 2026
86 checks passed

obviyus mentioned this pull request May 5, 2026

Webchat persistence regression: embedded-runtime turns skip persistCliTurnTranscript causing silent assistant message drops #77518

Closed

github-actions Bot mentioned this pull request May 5, 2026

📡 Upstream Digest — 2026-05-05 17:15 UTC curtismercier/openclaw-mods#775

Open

This was referenced May 6, 2026

Webchat persistence regression still firing on 2026.5.6 — PR #77839 fix appears to have not shipped #78669

Closed

claude-cli Telegram DM context loss from missing-transcript race before transcript flush #77974

Closed

rasimme mentioned this pull request May 13, 2026

claude-cli backend skips runContextEngineMaintenance — LCM/transcript persistence broken for all CLI-routed turns #81558

Closed

This was referenced May 14, 2026

fix(agents): persist embedded-runtime assistant turns in session JSONL #77665

Closed

[Bug]: Successful assistant replies can be missing from the active transcript, causing later turns to re-answer old prompts #76990

Closed

steipete mentioned this pull request Jun 14, 2026

Tag embedded subagent gap-fill rows as delivery mirrors #88687

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): persist embedded session JSONL transcripts (#77823)#77839

fix(agents): persist embedded session JSONL transcripts (#77823)#77839
obviyus merged 4 commits into
openclaw:mainfrom
neeravmakwana:fix/embedded-session-transcript-77823

neeravmakwana commented May 5, 2026

Uh oh!

clawsweeper Bot commented May 5, 2026 •

edited

Loading

Uh oh!

neeravmakwana commented May 5, 2026 •

edited by clawsweeper Bot

Loading

Uh oh!

obviyus left a comment

Uh oh!

Uh oh!

obviyus commented May 5, 2026

Uh oh!

bryan-chx commented May 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

neeravmakwana commented May 5, 2026

Summary

Root cause

Why this is safe

Security / runtime controls

Real behavior proof

Tests run

Out of scope

Uh oh!

clawsweeper Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neeravmakwana commented May 5, 2026 • edited by clawsweeper Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Follow-up on AI review (ClawSweeper / Codex)

Uh oh!

obviyus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

obviyus commented May 5, 2026

Uh oh!

bryan-chx commented May 5, 2026

Live verification (2026-05-06)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

clawsweeper Bot commented May 5, 2026 •

edited

Loading

neeravmakwana commented May 5, 2026 •

edited by clawsweeper Bot

Loading