fix(codex/app-server): stable mirror idempotency to prevent transcript loss#77046
Conversation
|
Codex review: needs maintainer review before merge. Summary Reproducibility: yes. source-reproducible: current main keys mirrored messages by role and snapshot index while the projector can insert optional reasoning or plan assistant records before the final assistant. I did not run tests in this read-only review. Next step before merge Security Review detailsBest possible solution: Review and land this stable Codex mirror idempotency fix after normal validation, while keeping #77012 open for any remaining transcript-loss cause. Do we have a high-confidence way to reproduce the issue? Yes, source-reproducible: current main keys mirrored messages by role and snapshot index while the projector can insert optional reasoning or plan assistant records before the final assistant. I did not run tests in this read-only review. Is this the best way to solve the issue? Yes. Stable What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against cb3853587576. |
912a20d to
a74117b
Compare
a74117b to
f411534
Compare
…Unreleased (#77728) Merged via squash. Prepared head SHA: 1bd228f Co-authored-by: openperf <[email protected]> Co-authored-by: openperf <[email protected]> Reviewed-by: @openperf
…ainer-hardening * origin/main: (843 commits) docs(changelog): relocate openclaw#77046 and openclaw#77280 entries from 2026.5.3 to Unreleased (openclaw#77728) docs: reorder unreleased changelog fix: expose ollama thinking profile before activation (openclaw#77617) (thanks @yfge) fix: expose ollama thinking profile before activation test(gateway): preserve dispatch timers in waiter test(gateway): keep startup context timer live docs: document cache-friendly activity helper ci: install ffmpeg for Mantis media previews fix: avoid impossible device token rotation advice (openclaw#77688) (thanks @Conan-Scott) docs(changelog): note doctor device pairing advice fix fix(doctor): avoid impossible device token rotation advice ci: use Crabbox media previews for Mantis docs: filter maintainer-owned triage noise test: cover GitHub activity helper fix(session-file-repair): drop null-role message entries instead of preserving them (openclaw#77288) fix: prune orphan session artifacts perf: reduce GitHub activity cache misses fix: cache session list model resolution (openclaw#77650) (thanks @ragesaq) ci: embed Mantis desktop previews fix(replay-history): drop trailing stream-error placeholder before provider send (openclaw#77287) ... # Conflicts: # CHANGELOG.md
…t loss (openclaw#77046) * fix(codex/app-server): stable mirror idempotency to prevent transcript loss * Changelog: note codex/app-server transcript mirror dedupe stabilization (openclaw#77046)
…t loss (openclaw#77046) * fix(codex/app-server): stable mirror idempotency to prevent transcript loss * Changelog: note codex/app-server transcript mirror dedupe stabilization (openclaw#77046)
…rom 2026.5.3 to Unreleased (openclaw#77728) Merged via squash. Prepared head SHA: 1bd228f Co-authored-by: openperf <[email protected]> Co-authored-by: openperf <[email protected]> Reviewed-by: @openperf
…t loss (openclaw#77046) * fix(codex/app-server): stable mirror idempotency to prevent transcript loss * Changelog: note codex/app-server transcript mirror dedupe stabilization (openclaw#77046)
…rom 2026.5.3 to Unreleased (openclaw#77728) Merged via squash. Prepared head SHA: 1bd228f Co-authored-by: openperf <[email protected]> Co-authored-by: openperf <[email protected]> Reviewed-by: @openperf
…t loss (openclaw#77046) * fix(codex/app-server): stable mirror idempotency to prevent transcript loss * Changelog: note codex/app-server transcript mirror dedupe stabilization (openclaw#77046)
…rom 2026.5.3 to Unreleased (openclaw#77728) Merged via squash. Prepared head SHA: 1bd228f Co-authored-by: openperf <[email protected]> Co-authored-by: openperf <[email protected]> Reviewed-by: @openperf
…t loss (openclaw#77046) * fix(codex/app-server): stable mirror idempotency to prevent transcript loss * Changelog: note codex/app-server transcript mirror dedupe stabilization (openclaw#77046)
…rom 2026.5.3 to Unreleased (openclaw#77728) Merged via squash. Prepared head SHA: 1bd228f Co-authored-by: openperf <[email protected]> Co-authored-by: openperf <[email protected]> Reviewed-by: @openperf
Summary
Problem:
mirrorCodexAppServerTranscriptbuilds its idempotency key as`${idempotencyScope}:${message.role}:${index}`, whereindexis the position of the message inside the per-turnmessagesSnapshot. The snapshot itself is built dynamically byevent-projector.ts:188— it always starts with the user prompt, then conditionally pushes aCodex reasoningmirror, aCodex planmirror, and the final assistant reply. IfmirrorCodexAppServerTranscriptruns more than once for the same logical turn (a retried turn, or any flow that re-mirrors the just-finished turn) and the second snapshot includes a reasoning record that the first didn't, every assistant-role record after the inserted slot shifts by one index. The original assistant entry's first-call key (:assistant:1) collides with the new reasoning entry's key (also:assistant:1), so the legitimately-new reasoning record is silently dropped at the dedupe gate, and the assistant content gets re-appended under:assistant:2— the on-disk transcript ends up with a duplicate assistant entry and a missing reasoning entry. Files involved:extensions/codex/src/app-server/transcript-mirror.ts:33and the call site inextensions/codex/src/app-server/run-attempt.ts:1782.Root Cause: Positional index is not a stable identity for a logical message. The dedupe key needs to be invariant under snapshot reordering inside a fixed scope. The cross-turn dimension is already handled correctly by the existing call-site scope
`codex-app-server:${threadId}:${turnId}`— distinct turns produce distinct scopes and therefore can never collide, even when role+content match (which is what preserves repeated-message turns such as a user typing"yes"twice in one thread).Fix: A three-part change across three files:
event-projector.ts— each entry inmessagesSnapshotis now tagged withattachCodexMirrorIdentitybefore the snapshot is passed tomirrorCodexAppServerTranscript. Tags use a${turnId}:${kind}shape (:prompt,:reasoning,:plan,:assistant) so each logical message carries a stable per-turn identity that is invariant under snapshot reordering and across re-invocations.transcript-mirror.ts— the dedupe key is rebuilt: when a message carries an explicit identity tag (set byattachCodexMirrorIdentity), the key is`${scope}:${identity}`; when no tag is present (fallback for callers that did not tag), the key falls back to`${scope}:${role}:${sha256({role,content}).slice(0,16)}`. Themessagesfilter is narrowed viaMirroredAgentMessage = Extract<AgentMessage, { role: "user" | "assistant" }>.run-attempt.ts— the call-siteidempotencyScopeis shortened from`codex-app-server:${threadId}:${turnId}`to`codex-app-server:${threadId}`. Per-turn distinction is now carried by the${turnId}:${kind}identity tag on each individual entry, so the scope no longer needs to embedturnId. DroppingturnIdfrom the scope is what makes a re-emitted prior-turn entry — which still carries its original${turnId}:${kind}identity — collide with its existing on-disk key and become a true no-op.What changed:
extensions/codex/src/app-server/event-projector.ts— importattachCodexMirrorIdentity; wrap eachmessagesSnapshotpush (user prompt, reasoning mirror, plan mirror, assistant) withattachCodexMirrorIdentity(message, \${turnId}:${kind}`)`.extensions/codex/src/app-server/transcript-mirror.ts— addattachCodexMirrorIdentity(exported, forevent-projector.ts),readMirrorIdentity,fingerprintMirrorMessageContent, andbuildMirrorDedupeIdentity; replace:${index}suffix withbuildMirrorDedupeIdentity(message)(identity tag when present, content fingerprint fallback); narrow filter via theMirroredAgentMessagetype predicate.extensions/codex/src/app-server/run-attempt.ts— changeidempotencyScopefrom`codex-app-server:${params.threadId}:${params.turnId}`to`codex-app-server:${params.threadId}`. Per-turn distinctness is now provided by the${turnId}:${kind}identity tag on each entry; keepingturnIdin the scope would be redundant and would break cross-turn re-emit idempotency.extensions/codex/src/app-server/transcript-mirror.test.ts— update the three existing key-format assertions to match the identity-tag shape; adddedupes mirrored messages despite snapshot positional shifts(asserts the exact text sequence after a re-mirror that inserts a reasoning record; pre-fix code produces["hello", "hi there", "hi there"]with reasoning dropped and assistant duplicated); addkeeps repeated same-content turns distinct(verifies that two turns with identical role+content but differentturnIdsegments produce two full user+assistant pairs, not one).What did NOT change (scope boundary):
{ type, id, parentId, timestamp, message }withmessage.idempotencyKeyas a flat string. Pre-existing transcripts with the old:role:indexkeys remain valid; their old keys never collide with new identity-tag keys, which is the expected behaviour for a pre-existing entry under any dedupe scheme.appendSessionTranscriptMessage,migrateLinearTranscriptToParentLinked,selectBoundedActiveTailRecords, the session write-lock, and the gateway-injected assistant write path are untouched.mirrorCodexAppServerTranscript's parameter shape is identical. Noanytypes are introduced.Honest scope vs. #77012
This PR addresses one mechanism that produces transcript drift in the Codex app-server mirror: re-invocation of mirror for a logical turn whose snapshot has reordered between calls. The end-user-visible symptom in the linked issue is a sibling-branch on disk where a new user message shares its
parentIdwith the prior assistant entry, after which the active-tail walker orphans the prior assistant. A hermetic two-turn mirror reproducer with the production per-turn snapshot pattern does not produce that sibling-branch state on currentmain— the parent chain stays well-formed, which means a single mirror call per turn with disjoint per-turn content is not by itself the proximate cause of the on-disk shape reported there. The within-scope index collision fixed in this PR is therefore one contributing factor, not the full root cause; this PR is intentionallyRefsrather thanFixes.Reproduction
Hermetic, against current
main:Verify the new test is a real regression (it would fail without the fix):
Risk / Mitigation
turnId, so distinct turns produce distinct keys even when role+content match. Covered bykeeps repeated same-content turns distinct.(threadId, turnId)scope.JSON.stringifyfield ordering (fallback path). The fingerprint hashes{ role, content }in a fixed object literal — ordering is deterministic on every supported Node version (engines.node >= 22.14.0).runAgentHarnessBeforeMessageWriteHook, so a hook that mutates content does not destabilize the dedupe anchor on retries.pnpm tsgo:extensionsandpnpm tsgo:extensions:testare clean.Change Type (select all)
Scope (select all touched areas)
extensions/codex/src/app-server/transcript-mirror.ts,event-projector.ts,run-attempt.ts)transcript-mirror.test.ts)Linked Issue/PR
Refs #77012 — addresses the within-scope index-collision contributing factor; the broader sibling-branch symptom in that issue likely involves at least one additional path and should remain open until that path is identified and fixed.