Skip to content

feat(codex): bind context-engine projections to codex threads#82351

Merged
jalehman merged 4 commits into
mainfrom
josh/context-engine-bootstrap-projection
May 16, 2026
Merged

feat(codex): bind context-engine projections to codex threads#82351
jalehman merged 4 commits into
mainfrom
josh/context-engine-bootstrap-projection

Conversation

@jalehman

@jalehman jalehman commented May 15, 2026

Copy link
Copy Markdown
Contributor

What

Adds Codex app-server support for context-engine-owned thread-bootstrap projection epochs. When an owning context engine provides a bootstrap projection, Codex binds that projection metadata to the backend thread and starts a new backend thread when the context-engine epoch or compatible runtime policy changes.

Why

Lossless/context-engine compaction should own the compacted context that gets projected into Codex. Re-projecting the full context every turn wastes prompt space, while continuing the same Codex backend thread after context-engine compaction leaves Codex and the context engine out of sync.

Changes

  • Bind context-engine projection epochs
  • Start new Codex threads on epoch rollover
  • Redact/summarize projected tool payloads before Codex bootstrap
  • Route compaction pressure through owner
  • Adopt rotated successor transcript handles after owning context-engine compaction
  • Add lifecycle binding tests
  • Document Codex context-engine behavior

Testing

  • git diff --check HEAD~1..HEAD
  • CI=true pnpm test -- extensions/codex/src/app-server/compact.test.ts extensions/codex/src/app-server/run-attempt.context-engine.test.ts extensions/codex/src/app-server/context-engine-projection.test.ts

Focused local regression output:

RUN  v4.1.6
Test Files 3 passed (3)
Tests 39 passed (39)
[test] passed 1 Vitest shard in 12.02s

Real behavior proof

Behavior addressed: Lossless Claw can project Codex app-server context using thread-bootstrap epochs, reuse a matching bootstrap binding without re-projecting full context, start a fresh Codex backend thread when the runtime binding becomes incompatible, and invalidate the Codex thread binding after context-engine-owned compaction. The Clawsweeper successor-transcript compaction finding is addressed in code and covered by the focused regression test.

Real environment tested: Live OpenClaw gateway on Josh's Telegram OpenClaw topic, using the OpenAI Codex app-server runtime with Lossless Claw active as the context engine. Runtime evidence came from today's gateway log. The live checkout was branch josh/context-engine-bootstrap-projection at cca60ade0e during the manual compaction; the latest successor-handle fix commit a3f06bf160 was verified locally in the PR worktree.

Exact steps or command run after this patch: Inspected today's runtime log around Josh's manual compact at 2026-05-15 18:25 PDT and searched for Lossless/Codex context-engine compaction, projection, and binding events. Also reran focused local verification with git diff --check HEAD~1..HEAD and CI=true pnpm test -- extensions/codex/src/app-server/compact.test.ts extensions/codex/src/app-server/run-attempt.context-engine.test.ts extensions/codex/src/app-server/context-engine-projection.test.ts.

Evidence after fix: Redacted runtime log excerpts from the live gateway showing manual context-engine-owned compaction:

2026-05-15T18:25:17-07:00 [lossless-claw]
[lcm] LCM compaction leaf pass (normal): 90357 -> 72788
conversation=[redacted] summary=[redacted]

2026-05-15T18:25:24-07:00 [lossless-claw]
[lcm] LCM compaction leaf pass (normal): 72788 -> 55960
conversation=[redacted] summary=[redacted]

2026-05-15T18:25:24-07:00 [lossless-claw]
[lcm] compact: done conversation=[redacted]
session=[redacted] sessionKey=agent:main:telegram:group:[redacted]:topic:3731
ok=true compacted=true reason=compacted
tokensBefore=90357 tokensAfter=55960 duration=15672ms

2026-05-15T18:25:24-07:00 [lossless-claw]
[lcm] maintain: session queue acquired
queueKey=agent:main:telegram:group:[redacted]:topic:3731 queuedAhead=0 wait=0ms

2026-05-15T18:25:24-07:00 [agent/embedded]
completed context-engine-owned Codex app-server compaction
engineId=lossless-claw ok=true compacted=true reason=compacted
codexThreadBindingInvalidated=true

Redacted runtime log excerpt from the next live topic turn showing the invalidated binding caused a fresh thread-bootstrap projection and new Codex binding:

2026-05-15T18:25:32-07:00 [lossless-claw]
[lcm] assemble: done conversation=[redacted]
contextItems=132 summaryContextItems=4 hasSummaryItems=true
inputMessages=306 outputMessages=131 tokenBudget=258000 estimatedTokens=56296
contextProjectionMode=thread_bootstrap
contextProjectionEpoch=summary-prefix-v1:[redacted]

2026-05-15T18:25:33-07:00 [agent/embedded]
codex app-server context-engine projection decision
engineId=lossless-claw mode=thread_bootstrap
projected=true reason=missing-thread-binding
assembledMessages=131 originalHistoryMessages=306 projectedPromptChars=228176

2026-05-15T18:25:33-07:00 [agent/embedded]
codex app-server wrote context-engine thread binding
engineId=lossless-claw action=started
epoch=summary-prefix-v1:[redacted]

Earlier redacted live runtime excerpt from the same branch showing a matching bootstrap binding can be reused without re-projecting:

2026-05-15T16:12:10-07:00 [lossless-claw]
[lcm] assemble: done ... tokenBudget=258000 estimatedTokens=26227
contextProjectionMode=thread_bootstrap
contextProjectionEpoch=summary-prefix-v1:[redacted]

2026-05-15T16:12:10-07:00 [agent/embedded]
codex app-server context-engine projection decision
engineId=lossless-claw mode=thread_bootstrap
previousEpoch=summary-prefix-v1:[same redacted epoch]
projected=false reason=matching-thread-bootstrap-binding
assembledMessages=134 originalHistoryMessages=351 projectedPromptChars=107372

2026-05-15T16:12:11-07:00 [agent/embedded]
codex app-server wrote context-engine thread binding
engineId=lossless-claw action=resumed epoch=summary-prefix-v1:[same redacted epoch]

Focused local regression output for the successor-transcript compaction path:

RUN  v4.1.6
Test Files 3 passed (3)
Tests 39 passed (39)
[test] passed 1 Vitest shard in 12.02s

Observed result after fix: The live gateway logs show Lossless Claw performing context-engine compaction, Codex completing context-engine-owned app-server compaction with codexThreadBindingInvalidated=true, the next turn projecting because the thread binding was missing, and Codex writing a fresh context-engine thread binding. The earlier live logs also show Codex reusing a matching bootstrap binding without re-projecting. The focused regression verifies the Clawsweeper P2 contract surface: owning context-engine compaction sends successor sessionId/sessionFile to maintain() and clears both original and successor Codex app-server bindings.

What was not tested: The live manual compact did not return a successor transcript in the observable logs, and the deployed checkout was not yet at a3f06bf160. The successor-handle behavior is covered by the focused regression because it is the ContextEngine.compact() contract surface Clawsweeper flagged.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: codex size: XL maintainer Maintainer-authored PR labels May 15, 2026
@clawsweeper

clawsweeper Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

Summary
The branch adds an additive context-engine projection contract and wires Codex app-server thread bindings, owning-compaction handling, redacted tool-payload projection, tests, docs, and changelog around thread-bootstrap epochs.

Reproducibility: not applicable. as a feature PR. The relevant behavior is demonstrated by redacted live gateway logs in the PR body and focused regression coverage on the PR head.

Real behavior proof
Sufficient (logs): The PR body includes redacted after-fix live gateway logs showing compaction invalidation, fresh thread-bootstrap projection, binding reuse, and focused regression output for successor transcript handling.

Next step before merge
No ClawSweeper repair is needed: there are no blocking findings, and the protected maintainer label leaves this additive SDK/Codex feature in human review.

Security
Cleared: No concrete security or supply-chain regression found; the diff adds no dependencies or workflow execution paths and uses existing redaction helpers for preserved tool-result context.

Review details

Best possible solution:

Have a maintainer review and land the additive Codex/context-engine projection contract once normal CI and branch gates pass.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a feature PR. The relevant behavior is demonstrated by redacted live gateway logs in the PR body and focused regression coverage on the PR head.

Is this the best way to solve the issue?

Yes, with maintainer review: the PR uses an additive SDK field and binds projection epochs in the existing Codex thread lifecycle, which is the narrow maintainable place for persistent app-server thread coherence.

What I checked:

  • Protected PR state: The provided GitHub context shows this PR is open at head a3f06bf with labels including maintainer, extensions: codex, size: XL, and proof: sufficient; the protected label blocks cleanup close. (a3f06bf1608f)
  • Current main lacks the new SDK contract: Current main's AssembleResult ends after systemPromptAddition and has no contextProjection, ContextEngineProjection, or thread_bootstrap contract in the context-engine type surface. (src/context-engine/types.ts:6, a70c90a52bf1)
  • PR adds the projection API: The PR head adds contextProjection?: ContextEngineProjection to AssembleResult with per_turn and thread_bootstrap modes plus optional epoch/fingerprint metadata. (src/context-engine/types.ts:27, a3f06bf1608f)
  • PR binds projection decisions into Codex runs: The PR head reads thread-bootstrap projection metadata from context-engine assembly, chooses whether to project or reuse the backend thread, and preserves tool payloads only when a bootstrap projection is active. (extensions/codex/src/app-server/run-attempt.ts:682, a3f06bf1608f)
  • PR persists projection compatibility with thread bindings: The PR head passes contextEngineProjection into startOrResumeThread and includes projection mode/epoch/fingerprint in the binding compatibility check. (extensions/codex/src/app-server/thread-lifecycle.ts:91, a3f06bf1608f)
  • Owning compaction adopts successor transcript handles: The PR head delegates compaction to owning context engines, runs maintenance against returned successor sessionId/sessionFile, and clears both original and successor Codex bindings when the engine compacts. (extensions/codex/src/app-server/compact.ts:39, a3f06bf1608f)

Likely related people:

  • jalehman: Prior merged history by Josh Lehman introduced the Codex app-server context-engine lifecycle and later rotated incompatible Codex context-engine threads in the same runtime surface this PR extends. (role: feature owner; confidence: high; commits: 51186d272543, b55d9fa4660b; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/thread-lifecycle.ts, extensions/codex/src/app-server/compact.ts)
  • steipete: Recent current-main Codex app-server compaction and transcript-rotation commits define adjacent behavior around native compaction and rotated session handles. (role: recent adjacent contributor; confidence: medium; commits: 934fc6ceeb10, 65dd71d42dee; files: extensions/codex/src/app-server/compact.ts, extensions/codex/src/app-server/run-attempt.ts)
  • joshavant: Current main's most recent run-attempt.ts history includes adjacent Codex app-server notification projection work, which is near the runtime path this PR changes. (role: recent adjacent contributor; confidence: medium; commits: ea16a5e9e10c; files: extensions/codex/src/app-server/run-attempt.ts)

Remaining risk / open question:

  • I did not execute tests in this read-only sweep; the PR body reports focused local tests and live logs, but normal CI and maintainer review should still gate merge.

Codex review notes: model gpt-5.5, reasoning high; reviewed against a70c90a52bf1.

@jalehman
jalehman force-pushed the josh/context-engine-bootstrap-projection branch from 1c0ec04 to cca60ad Compare May 15, 2026 23:06
@jalehman jalehman self-assigned this May 15, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 16, 2026
@jalehman
jalehman merged commit 80ca484 into main May 16, 2026
40 of 44 checks passed
@jalehman
jalehman deleted the josh/context-engine-bootstrap-projection branch May 16, 2026 03:59
galiniliev pushed a commit to galiniliev/openclaw that referenced this pull request May 20, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
galiniliev pushed a commit to galiniliev/openclaw that referenced this pull request May 25, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
qiaokuan1992 pushed a commit to qiaokuan1992/openclaw that referenced this pull request Jun 2, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
…aw#82351)

* feat(codex): bind context-engine projections to codex threads

* fix: harden Codex context-engine projection

* fix: remove unused Codex projection helper

* fix(codex): adopt compacted context-engine transcripts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation extensions: codex maintainer Maintainer-authored PR proof: sufficient ClawSweeper judged the real behavior proof convincing. size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant