Skip to content

Prevent Codex thread rotation from losing next-step context#94093

Merged
jalehman merged 2 commits into
openclaw:mainfrom
VACInc:fix-thread-continuity-budget
Jun 18, 2026
Merged

Prevent Codex thread rotation from losing next-step context#94093
jalehman merged 2 commits into
openclaw:mainfrom
VACInc:fix-thread-continuity-budget

Conversation

@VACInc

@VACInc VACInc commented Jun 17, 2026 •
edited by jalehman
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Clarify the user-visible failure: long-running Codex-backed channel sessions can appear to forget next steps after hours because native Codex thread rotation can replay too little prior context.
  • Scale no-context-engine Codex continuity projection from the active context budget/reserve instead of the conservative default cap.
  • Apply the same budget to both fresh native thread starts and stale-binding continuity when token pressure forces a new native thread.
  • When token pressure discards a stale native binding after a stale-resume delta was precomputed, rebuild continuity from the full mirrored OpenClaw transcript instead of reusing only the post-binding delta.
  • Bound the final projected context slice under Codex app-server turn/start text-input limits without truncating the current user request, even when prior transcript text contains projection delimiter strings.
  • Add regression coverage that preserves older handoff/next-step anchors beyond the old 24k rendered-context fallback window and across forced fresh-thread rotation.

End-user issue

A user can leave a long channel session open, come back hours later or the next day, and ask the agent to continue the work. The OpenClaw session may still be the same session, but the Codex native app-server thread may have been rotated under token pressure while the topic accumulated enough history.

Before this change, that rotation could make the next answer feel forgetful: the agent might miss the previously agreed next steps, ask the user to repeat context, or continue from only the recent tail of the conversation. This is especially visible in long-lived GPT/Codex channel sessions because the user experience looks like an idle-session reset even when no OpenClaw idle or max-age reset happened.

This PR does not make conversation memory unlimited and does not disable compaction. It makes the continuity bridge less lossy when OpenClaw has to start a fresh native Codex thread from the mirrored session history.

Root Cause

OpenClaw keeps Codex app-server conversation continuity in native Codex threads, while the mirrored OpenClaw transcript is persistence/search state. When a Codex native thread was unavailable or had to be replaced under token pressure, the no-context-engine continuity paths called projectContextEngineAssemblyForCodex without maxRenderedContextChars.

That omitted option made the projection helper fall back to its conservative 24,000 character rendered-context cap. In long-running channel topics, a token-pressure native thread rotation could therefore preserve only a small tail of the prior discussion, dropping older next-step or handoff context even though the OpenClaw session itself had not idle-reset.

There was a second loss path once stale native bindings were involved. OpenClaw correctly precomputed a small stale-resume projection containing only messages newer than the native binding because a resumed Codex thread already owns the older history. But if token pressure then discarded that native thread and started fresh, the older native-owned history was no longer available in Codex. The forced fresh start could still reuse the stale-resume delta, so the first turn on the new native thread only received post-binding context.

The app-server also rejects oversized turn/start text input before compaction can help. The final patch records the generated projected-context byte range and trims only that range after OpenClaw prompt wrappers are applied, preserving the current user request while keeping the final app-server input under Codex's text-input cap.

RCA proof from the reported private topics was redacted for PPI: the session configuration had no idle/max-age reset, while local logs showed Codex native token-limit fresh-thread rotations for the affected topic. One rotation rebuilt the turn prompt with a small prompt after a stale binding had been cleared, matching the code path above and explaining why the next answer could look like it forgot the plan after returning later.

Real behavior proof

Behavior addressed: Codex fresh native thread continuity keeps the same scaled projection budget used by the active context-engine path, forced fresh starts rebuild from mirrored session history once the native thread binding is discarded, and oversized projected context is fitted under Codex app-server turn/start text-input limits without truncating the current user request.

Real environment tested: local OpenClaw source checkout, Codex app-server Vitest shard, local formatting check, local diff whitespace check, Codex autoreview.

Exact steps or command run after this patch:

git diff --check -- extensions/codex/src/app-server/context-engine-projection.ts extensions/codex/src/app-server/context-engine-projection.test.ts extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts
pnpm exec oxfmt --check --threads=1 extensions/codex/src/app-server/context-engine-projection.ts extensions/codex/src/app-server/context-engine-projection.test.ts extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts
node scripts/run-vitest.mjs extensions/codex/src/app-server/run-attempt.test.ts extensions/codex/src/app-server/context-engine-projection.test.ts -- -t "projects bounded continuity|preserves stale-binding continuity|keeps large fresh-thread continuity|fits projected context" --reporter=verbose
.agents/skills/autoreview/scripts/autoreview --mode local --prompt "Review only the updated local PR #94093 Codex app-server continuity input-limit patch in extensions/codex/src/app-server/context-engine-projection.ts, context-engine-projection.test.ts, run-attempt.ts, and run-attempt.test.ts. Focus on whether projected context is safely bounded under Codex turn/start input limits without truncating the current user request, including full delimiter separator text inside prior transcript content, and whether the regression tests cover the original P2 finding."

Evidence after fix:

All matched files use the correct format.
[test] passed 1 Vitest shard in 5.32s
Test Files  2 passed (2)
Tests  4 passed | 115 skipped (119)
autoreview clean: no accepted/actionable findings reported
overall: patch is correct (0.86)

Observed result after fix: the fresh-thread regression keeps older continuity anchors placed before enough filler that the old default cap would truncate them, the forced-rotation regression keeps both pre-binding native-owned context and post-binding stale context when thread/resume is skipped and thread/start is used, and the new oversized-turn regression keeps the final turn/start input at or below Codex's text-input cap while preserving recent continuity and the current user request. The delimiter regression includes the full generated </conversation_context> / Current user request: separator inside old transcript content to prove quoted history cannot spoof the projected-context boundary.

What was not tested:

  • Did not rerun the full app-server test file after an earlier unfiltered local attempt ran too long and surfaced an unrelated MCP fixture failure before the targeted continuity cases. This PR keeps proof focused on the changed continuity paths.
  • Did not run a live Telegram replay; the motivating evidence came from private topics and was kept out of the PR body.
  • Did not run broad check:changed/CI locally to avoid unnecessary load for this Codex app-server continuity patch.

Verification

  • git diff --check -- extensions/codex/src/app-server/context-engine-projection.ts extensions/codex/src/app-server/context-engine-projection.test.ts extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts
  • pnpm exec oxfmt --check --threads=1 extensions/codex/src/app-server/context-engine-projection.ts extensions/codex/src/app-server/context-engine-projection.test.ts extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts
  • node scripts/run-vitest.mjs extensions/codex/src/app-server/run-attempt.test.ts extensions/codex/src/app-server/context-engine-projection.test.ts -- -t "projects bounded continuity|preserves stale-binding continuity|keeps large fresh-thread continuity|fits projected context" --reporter=verbose
  • .agents/skills/autoreview/scripts/autoreview --mode local --prompt "Review only the updated local PR #94093 Codex app-server continuity input-limit patch..."

@openclaw-barnacle openclaw-barnacle Bot added extensions: codex size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 17, 2026
@VACInc VACInc changed the title Preserve fresh-thread continuity budget Prevent Codex thread rotation from losing next-step context Jun 17, 2026
@clawsweeper

clawsweeper Bot commented Jun 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 18, 2026, 1:46 PM ET / 17:46 UTC.

Summary
The PR budget-scales Codex no-context-engine continuity projections, rebuilds full mirrored-history continuity after forced fresh-thread rotation, bounds the projected context slice under Codex turn/start text limits, and adds regression tests.

PR surface: Source +86, Tests +107. Total +193 across 4 files.

Reproducibility: yes. at source level: current main omits maxRenderedContextChars in the no-context-engine fresh and stale projection paths while the projection helper defaults missing caps to 24,000 rendered characters. I did not run a live high-window Codex session in this read-only review.

Review metrics: 2 noteworthy metrics.

  • Continuity Paths Changed: 2 budget-scaled projection paths, 1 forced-fresh rebuild path. Maintainers should review the Codex native-thread recovery invariant across fresh, stale, and forced-fresh paths before merge.
  • Codex Text Limit Mirrored: 1 upstream 1,048,576-character cap. The PR mirrors a Codex app-server protocol limit, so dependency-source proof and drift risk matter before landing.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #84084
Summary: This PR is a candidate fix for the open legacy Codex projection-budget issue, with additional overlap in stale forced-fresh continuity and turn/start input-limit handling.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦞 diamond lobster
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add a redacted live Codex app-server log, terminal transcript, or linked artifact showing forced fresh-thread continuity after the patch; redact private paths, endpoints, tokens, phone numbers, and other private details.
  • Update the PR body after adding proof so ClawSweeper re-reviews automatically; if it does not, ask a maintainer to comment @clawsweeper re-review.
  • Have a maintainer explicitly record a proof override if the private RCA and targeted tests are the accepted evidence.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR supplies targeted test and formatting output, but no after-fix real Codex app-server rotation log, terminal transcript, or linked artifact; contributor action or maintainer override is still needed. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] This PR changes how much mirrored transcript Codex receives and how projected context is trimmed after native thread rotation; a wrong range or rebuild path would show up as session continuity drift rather than a compile-time failure.
  • [P1] The proof currently consists of targeted tests, formatting, diff-check, and autoreview output only; it does not show a redacted real Codex app-server run/log/artifact exercising forced fresh-thread rotation and fitted turn/start input.

Maintainer options:

  1. Require Redacted App-Server Proof (recommended)
    Ask for a terminal transcript, runtime log, or linked artifact showing a real Codex app-server fresh-thread rotation preserving continuity and fitting turn/start input after this patch.
  2. Accept Maintainer Proof Override
    A maintainer can explicitly record that the private RCA plus targeted regression tests are enough for this session-state change.
  3. Pause Behind The Canonical Issue
    If real-path proof is unavailable, leave this PR unmerged and keep [Bug]: Codex legacy mirrored-history fallback ignores contextTokenBudget and caps high-window sessions at ~24k rendered chars #84084 as the canonical work item.

Next step before merge

  • [P1] Manual review or proof override is needed because the external PR has only targeted tests and no real Codex app-server rotation proof; no narrow code repair was identified.

Security
Cleared: The diff touches Codex runtime continuity code and tests only; I found no dependency, workflow, credential, or supply-chain regression.

Review details

Best possible solution:

Land this candidate for #84084 after real redacted app-server proof or an explicit maintainer proof override; keep the narrower overlapping PRs non-canonical unless maintainers choose a replacement.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main omits maxRenderedContextChars in the no-context-engine fresh and stale projection paths while the projection helper defaults missing caps to 24,000 rendered characters. I did not run a live high-window Codex session in this read-only review.

Is this the best way to solve the issue?

Yes for the code direction: the PR reuses the existing budget helper, handles the stale-binding forced-fresh edge, and trims only projected context to Codex's upstream input cap. The remaining blocker is real-path proof or maintainer override, not an identified code defect.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a48e5091cba7.

Label changes

Label justifications:

  • P2: This is a normal-priority Codex session-continuity bug with limited blast radius to long-running Codex app-server sessions.
  • merge-risk: 🚨 session-state: Merging changes how transcript history is projected and trimmed across Codex native thread rotation, which can affect remembered session context.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦞 diamond lobster.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR supplies targeted test and formatting output, but no after-fix real Codex app-server rotation log, terminal transcript, or linked artifact; contributor action or maintainer override is still needed. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +86, Tests +107. Total +193 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 107 21 +86
Tests 2 109 2 +107
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 216 23 +193

What I checked:

Likely related people:

  • 100yenadmin: Authored the merged PR 80761 work that added resolveCodexContextEngineProjectionMaxChars to the active context-engine projection path this PR extends to no-context-engine continuity. (role: introduced related budget-aware projection behavior; confidence: high; commits: f7ab8c26b1e7; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/context-engine-projection.ts, extensions/codex/src/app-server/context-engine-projection.test.ts)
  • jalehman: Merged PR 80761 and authored the current PR head commit that adds the Codex turn input fitting path. (role: recent area contributor and merger; confidence: high; commits: f7ab8c26b1e7, 1f3ced8f6343; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/context-engine-projection.ts)
  • steipete: Authored and merged PR 88652, which established the bounded fresh/stale Codex recovery continuity paths adjusted by this PR. (role: adjacent continuity contributor; confidence: medium; commits: 827ceb55d074; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/run-attempt.test.ts)
  • VACInc: Previously authored the merged projection truncation helper work in PR 83127 and authored the first commit on this branch, so their history is relevant beyond opening the PR. (role: recent projection contributor; confidence: medium; commits: 72e164a3fe20, 3fb9f91b8da9; files: extensions/codex/src/app-server/context-engine-projection.ts, extensions/codex/src/app-server/run-attempt.ts)
  • vincentkoc: Current shallow blame on the affected current-main lines points to a recent extension refactor, and recent Codex app-server runtime commits touch nearby thread/tool behavior. (role: recent adjacent contributor; confidence: low; commits: 1eaace70e3b4, ab1e5832d2fb, b4cdd9211957; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/context-engine-projection.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@VACInc
VACInc force-pushed the fix-thread-continuity-budget branch from b882885 to 8dad0ca Compare June 17, 2026 22:37
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 17, 2026
@jalehman jalehman self-assigned this Jun 17, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. size: M and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. size: S labels Jun 18, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 18, 2026
@jalehman
jalehman force-pushed the fix-thread-continuity-budget branch from 2383950 to 1f3ced8 Compare June 18, 2026 17:31
@jalehman
jalehman merged commit ea76a45 into openclaw:main Jun 18, 2026
158 of 159 checks passed
@jalehman

jalehman commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Merged via squash as a maintainer decision.

Thanks @VACInc!

@github-actions github-actions Bot mentioned this pull request Jun 18, 2026
Open
@clawsweeper clawsweeper Bot mentioned this pull request Jun 18, 2026
Closed
vincentkoc added a commit that referenced this pull request Jun 19, 2026
@vincentkoc
Merge branch 'main' of https://github.com/openclaw/openclaw
d8cc16a 
* 'main' of https://github.com/openclaw/openclaw: (82 commits)
  fix(e2e): validate chat tools body limit
  fix(e2e): honor chat tools body limit
  fix(e2e): validate chat tools timeout
  fix(e2e): give cleanup smoke build heap headroom
  fix(e2e): validate plugin lifecycle limits
  refactor(auto-reply): add lifecycle storage seams (#93685)
  fix: preserve pending subagent completion announces (#94349)
  fix(e2e): validate plugin log limits before setup
  fix(e2e): validate codex media timeout
  fix(e2e): validate fixture log limits
  fix(e2e): validate cleanup log limits
  fix(e2e): validate docker log limits
  fix(live): validate docker pids limits
  fix(e2e): validate docker pids limits
  test: fold channel message flows into qa e2e (#93174)
  fix(e2e): validate docker build limits
  Prevent Codex thread rotation from losing next-step context (#94093)
  fix(e2e): validate fixture cleanup interval
  fix(agents): correct claw-score validation workflow
  fix(e2e): validate log tail limits
  ...
@Nas01010101 Nas01010101 mentioned this pull request Jun 19, 2026
Closed
This was referenced Jun 19, 2026
Merged
Open
Closed
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 19, 2026
@VACInc @jalehman
Prevent Codex thread rotation from losing next-step context (openclaw…
e0fd1f5 
…#94093)

Merged via squash.

Prepared head SHA: 1f3ced8
Maintainer decision: `checks-node-core-tooling` is an unrelated baseline/tooling failure; PR-relevant CI and real behavior proof passed.

Co-authored-by: VACInc <[email protected]>
Co-authored-by: jalehman <[email protected]>
Reviewed-by: @jalehman
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 19, 2026
@vincentkoc
Merge branch 'main' of https://github.com/openclaw/openclaw
512e654 
* 'main' of https://github.com/openclaw/openclaw: (82 commits)
  fix(e2e): validate chat tools body limit
  fix(e2e): honor chat tools body limit
  fix(e2e): validate chat tools timeout
  fix(e2e): give cleanup smoke build heap headroom
  fix(e2e): validate plugin lifecycle limits
  refactor(auto-reply): add lifecycle storage seams (openclaw#93685)
  fix: preserve pending subagent completion announces (openclaw#94349)
  fix(e2e): validate plugin log limits before setup
  fix(e2e): validate codex media timeout
  fix(e2e): validate fixture log limits
  fix(e2e): validate cleanup log limits
  fix(e2e): validate docker log limits
  fix(live): validate docker pids limits
  fix(e2e): validate docker pids limits
  test: fold channel message flows into qa e2e (openclaw#93174)
  fix(e2e): validate docker build limits
  Prevent Codex thread rotation from losing next-step context (openclaw#94093)
  fix(e2e): validate fixture cleanup interval
  fix(agents): correct claw-score validation workflow
  fix(e2e): validate log tail limits
  ...
This was referenced Jun 19, 2026
Closed
Closed
@wings1029 wings1029 mentioned this pull request Jun 19, 2026
Closed
@clawsweeper clawsweeper Bot mentioned this pull request Jun 21, 2026
Open
cxbAsDev pushed a commit to cxbAsDev/openclaw that referenced this pull request Jun 23, 2026
@VACInc @jalehman @cxbAsDev
Prevent Codex thread rotation from losing next-step context (openclaw…
c8184da 
…#94093)

Merged via squash.

Prepared head SHA: 1f3ced8
Maintainer decision: `checks-node-core-tooling` is an unrelated baseline/tooling failure; PR-relevant CI and real behavior proof passed.

Co-authored-by: VACInc <[email protected]>
Co-authored-by: jalehman <[email protected]>
Reviewed-by: @jalehman
@Bartok9 Bartok9 mentioned this pull request Jun 24, 2026
Closed
@clawsweeper clawsweeper Bot mentioned this pull request Jun 24, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@jalehman jalehman

Labels

extensions: codex merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@VACInc @jalehman