Move Codex soul context to developer instructions by pash-openai · Pull Request #84331 · openclaw/openclaw

pash-openai · 2026-05-19T21:48:19Z

Summary

Moves SOUL.md for native Codex runs out of the per-turn user input and into Codex developer instructions.
Leaves the remaining non-AGENTS workspace bootstrap files in turn reference context, and updates Codex harness docs to describe the split.
Removes the old wording that framed OpenClaw context as unable to override Codex developer instructions.

Verification

node scripts/run-vitest.mjs extensions/codex/src/app-server/run-attempt.test.ts -t "SOUL.md as Codex developer|lightweight cron"
pnpm exec oxfmt --check extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts docs/concepts/system-prompt.md docs/concepts/soul.md docs/plugins/codex-harness-reference.md docs/plugins/codex-harness-runtime.md
git diff --check

Token impact analysis

This is a meaningful win for long-running Codex-backed OpenClaw agents. Before this change, stable workspace identity context like SOUL.md and IDENTITY.md was appended to the Codex turn input every time the user sent a message. That meant a chat with an agent did not just carry one copy of the agent's soul; it accumulated another copy on every turn.

After this change, the agent still receives that identity context, but it receives it through Codex developer instructions instead of repeated per-turn user input. That is the right shape for this data: it is stable, agent-shaping context that should be present for the session, not copied into the native conversation history over and over.

I measured the before and after behavior with real OpenClaw Codex-runtime prompt assembly at the OpenClaw -> Codex app-server boundary. The test setup used a synthetic SOUL.md of 1,000 tokens and a synthetic IDENTITY.md of 500 tokens, counted with tiktoken using o200k_base.

Scenario	`turn/start.input` saved per turn	3-turn input saved	SOUL/IDENTITY duplicate context saved over 3 turns
ordinary chat	1,686	5,058	4,644
Telegram direct	1,686	5,058	4,644
Discord group	1,686	5,058	4,644
natural cron	1,689	5,067	4,650
lightweight cron command	0	0	0
lightweight heartbeat	235 input tokens	705 input tokens	0 SOUL/IDENTITY; HEARTBEAT body removed
full heartbeat	1,686	5,058	4,644

The first turn is intentionally not much smaller from the model's perspective: the agent still needs to see its soul once. The win compounds after that. In the old behavior, every subsequent turn added another full copy of the stable profile files to the conversation. In the new behavior, those files stay in the instruction surface and do not keep accumulating in turn history.

For the measured SOUL.md + IDENTITY.md setup, the stable identity block contributed about 1,548 repeated tokens each time it was appended to full turn context. That means the active duplicate-context savings scale linearly by turn:

Conversation length	Approx duplicate SOUL/IDENTITY tokens avoided on final turn	Approx cumulative duplicate token-slots avoided
10 turns	13,932	69,660
20 turns	29,412	294,120
50 turns	75,852	1,896,300

This also cleans up heartbeat behavior. HEARTBEAT.md is no longer inlined into ordinary turns, and heartbeat turns get a lightweight pointer telling the agent to read the file when it matters. So ordinary chat is no longer paying for heartbeat-specific context, and lightweight heartbeat mode avoids stuffing the heartbeat body directly into model input.

The practical result is that Codex-backed OpenClaw sessions keep the same agent identity and behavior while dramatically reducing prompt bloat in long conversations. This should improve effective context-window headroom, reduce repeated input cost, and make long-running Telegram, Discord, cron, and heartbeat sessions age much more gracefully.

clawsweeper · 2026-05-19T21:49:48Z

Codex review: needs real behavior proof before merge.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR routes SOUL.md, IDENTITY.md, TOOLS.md, and USER.md into Codex developer instructions, moves HEARTBEAT.md to heartbeat collaboration-mode pointers, updates Codex prompt snapshots/docs, and adjusts message-tool delivery guidance.

Reproducibility: yes. by source inspection. The merge result shows raw workspace profile files appended to Codex developer instructions and message_tool_only turns falling back away from the forced message tool path.

PR rating
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Summary: Not quality-ready because real behavior proof is missing and the patch has prompt-authority plus message-delivery blockers.

Rank-up moves:

Add redacted real runtime proof showing the Codex thread/start and turn/start request shape after this patch.
Resolve the raw workspace-file developer-instruction authority boundary or get explicit maintainer approval for it.
Preserve message_tool_only visible delivery through the message tool or document and prove a maintainer-approved fallback.

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Needs real behavior proof before merge: The PR body lists unit/format/diff checks but no redacted runtime logs, terminal output, request-shape artifact, screenshot, or recording showing the changed Codex app-server behavior; proof should redact private data and updating the PR body should trigger a fresh review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

Merging this would promote user-editable workspace files into Codex developer instructions, changing the trust and priority boundary for existing Codex harness users.
The message_tool_only path can lose the forced message tool and fall back to unspecified delivery guidance, which can leave intended source-channel replies in private final text.
The PR body provides unit and format checks but no redacted runtime proof of the actual Codex app-server request shape after the prompt-routing change.
The related open bridge PR may be the cleaner implementation path, so maintainers need to choose the intended SOUL/personality design before merge.

Maintainer options:

Preserve the prompt boundary (recommended)
Keep raw workspace files out of developer instructions and use a narrow bridge/config design for SOUL priority instead.
Approve developer-level workspace files
Maintainers can intentionally accept the authority change, but should require documented security and upgrade implications plus real runtime proof first.
Defer to the related bridge PR
Pause or close this PR if the related open SOUL/personality bridge PR remains the intended implementation path.

Next step before merge
The protected maintainer label plus prompt-boundary, security, message-delivery, and proof blockers require human design approval rather than an automated repair lane.

Security
Needs attention: The diff promotes user-editable workspace files into Codex developer instructions, which is a prompt authority and trust-boundary change.

Review findings

[P1] Keep raw workspace files out of developer instructions — extensions/codex/src/app-server/run-attempt.ts:4581-4585
[P1] Do not remove forced message delivery for message-tool-only turns — extensions/codex/src/app-server/run-attempt.ts:3533-3536

Review details

Best possible solution:

Keep raw workspace files as turn reference context and solve SOUL priority with a narrow Codex bridge/config design, with explicit maintainer approval and redacted runtime request-shape proof.

Do we have a high-confidence way to reproduce the issue?

Yes, by source inspection. The merge result shows raw workspace profile files appended to Codex developer instructions and message_tool_only turns falling back away from the forced message tool path.

Is this the best way to solve the issue?

No. The safer maintainable path is a narrow developer-level bridge/config decision that interprets lower-priority workspace context without promoting raw workspace files to developer policy.

Label justifications:

P2: The PR changes Codex harness prompt routing and source reply guidance with limited but real runtime blast radius.
merge-risk: 🚨 compatibility: Existing Codex harness users would see workspace files move from turn reference context into developer instructions during upgrade.
merge-risk: 🚨 message-delivery: The diff can remove the explicit message tool path for message_tool_only source replies.
merge-risk: 🚨 security-boundary: User-editable workspace files would gain developer-instruction authority over the Codex turn.
rating: 🧂 unranked krab: Current PR rating is 🧂 unranked krab because proof is 🧂 unranked krab, patch quality is 🧂 unranked krab, and Not quality-ready because real behavior proof is missing and the patch has prompt-authority plus message-delivery blockers.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists unit/format/diff checks but no redacted runtime logs, terminal output, request-shape artifact, screenshot, or recording showing the changed Codex app-server behavior; proof should redact private data and updating the PR body should trigger a fresh review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Full review comments:

[P1] Keep raw workspace files out of developer instructions — extensions/codex/src/app-server/run-attempt.ts:4581-4585
This appends SOUL.md, IDENTITY.md, TOOLS.md, and USER.md contents from the active workspace to Codex developer instructions. These files are user/workspace-editable; current main keeps them as turn reference context and explicitly says they are not developer policy, so promoting them changes the prompt authority boundary and can let workspace content outrank Codex/native runtime policy.
Confidence: 0.91
[P1] Do not remove forced message delivery for message-tool-only turns — extensions/codex/src/app-server/run-attempt.ts:3533-3536
When sourceReplyDeliveryMode is message_tool_only, final assistant text is private and visible source replies depend on the message tool. Making the forced message tool conditional on disableMessageTool and then telling the model to use an unspecified active delivery path can cause source-channel replies to disappear instead of being sent.
Confidence: 0.87

Overall correctness: patch is incorrect
Overall confidence: 0.89

Security concerns:

[medium] User-editable files gain developer authority — extensions/codex/src/app-server/run-attempt.ts:4581
SOUL.md, IDENTITY.md, TOOLS.md, and USER.md can contain workspace/user-controlled instructions; moving their raw contents into developerInstructions lets them outrank lower-priority turn context and changes the Codex/OpenClaw security boundary.
Confidence: 0.9

What I checked:

Current main keeps workspace bootstrap below developer policy: Current main renders non-AGENTS workspace files as turn input reference context and explicitly says they are project/user context, not developer policy. (extensions/codex/src/app-server/run-attempt.ts:4471, a00e7d3898cf)
PR promotes user-editable workspace files into developer instructions: The merge result appends rendered workspace developer instructions to Codex thread developerInstructions, and the renderer includes raw file contents under an instruction to internalize and follow them. (extensions/codex/src/app-server/run-attempt.ts:4581, bdaf600bdd0d)
PR weakens forced message-tool delivery: The PR makes shouldForceMessageTool return false when disableMessageTool is true, so message_tool_only can lose the explicit message tool path that current main always forces for this mode. (extensions/codex/src/app-server/run-attempt.ts:3535, bdaf600bdd0d)
Fallback wording is underspecified for private-final-text mode: The PR tells message_tool_only turns without a loaded message tool to use the active Codex delivery path, but in this mode final assistant text is not the visible source-channel delivery path. (extensions/codex/src/app-server/thread-lifecycle.ts:889, bdaf600bdd0d)
Tests assert the risky prompt authority change: The new test expects SOUL.md, IDENTITY.md, TOOLS.md, and USER.md contents to appear in Codex developerInstructions instead of turn input. (extensions/codex/src/app-server/run-attempt.test.ts:4723, bdaf600bdd0d)
History provenance for current boundary: The current workspace-context and visible-reply boundary appears to date to commit d60ab48, which introduced the Codex prompt routing and message-tool-only wording now being changed. (extensions/codex/src/app-server/run-attempt.ts:4471, d60ab485114a)

Likely related people:

Patrick Erichsen: Blame shows the current Codex workspace-context and message-tool-only prompt boundary came from d60ab48, which changed the central files touched here. (role: introduced behavior; confidence: high; commits: d60ab485114a; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/thread-lifecycle.ts)
Eva: Recent current-main work touched the Codex app-server run-attempt path for context-engine compaction behavior adjacent to this prompt-routing surface. (role: recent area contributor; confidence: medium; commits: a059309a9f9a; files: extensions/codex/src/app-server/run-attempt.ts)
100yenadmin: Co-authored recent current-main Codex app-server compaction work and authored the related open bridge PR that proposes a narrower SOUL routing path. (role: adjacent owner; confidence: medium; commits: a059309a9f9a; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/thread-lifecycle.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against a00e7d3898cf.

clawsweeper · 2026-05-19T23:18:06Z

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?

The egg game starts only after the PR passes the real-behavior proof check.
Before that, no creature, rarity, or ASCII portrait is rolled. The treat waits for real proof.
This is still just collectible flavor: proof affects review readiness, not creature quality.

* Move Codex soul context to developer instructions * Route Codex workspace context by lifetime * Refresh Codex prompt snapshots * Update prompt snapshot expectations * Fix Codex workspace context diagnostics

openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: codex size: S maintainer Maintainer-authored PR labels May 19, 2026

pash-openai force-pushed the pash/codex-soul-developer-instructions branch from 6951dbe to e6c6a56 Compare May 19, 2026 22:07

clawsweeper Bot added the merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. label May 19, 2026

Move Codex soul context to developer instructions

bd44cb9

pash-openai force-pushed the pash/codex-soul-developer-instructions branch from e6c6a56 to bd44cb9 Compare May 19, 2026 22:33

Route Codex workspace context by lifetime

f2636b0

openclaw-barnacle Bot added gateway Gateway runtime size: M and removed size: S labels May 19, 2026

Refresh Codex prompt snapshots

793fd0b

openclaw-barnacle Bot added size: L and removed size: M labels May 19, 2026

pash-openai added 2 commits May 19, 2026 16:28

Update prompt snapshot expectations

9208858

Fix Codex workspace context diagnostics

cd783d5

pash-openai merged commit e0d1a2a into main May 19, 2026
93 checks passed

pash-openai deleted the pash/codex-soul-developer-instructions branch May 19, 2026 23:47

Haderach-Ram mentioned this pull request May 20, 2026

Ecosystem Digest — 2026-05-20 Haderach-Ram/openclaw-radar#13

Open

github-actions Bot mentioned this pull request May 20, 2026

📡 Upstream Digest — 2026-05-20 02:34 UTC curtismercier/openclaw-mods#901

Open

clawsweeper Bot mentioned this pull request May 20, 2026

[codex] make workspace prompt routing selectable #83612

Closed

pashpashpash mentioned this pull request May 27, 2026

fix(codex): ensure codex subagent bootstrap parity with pi subagents - only inject AGENTS.md and TOOLS.md #85811

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Move Codex soul context to developer instructions#84331

Move Codex soul context to developer instructions#84331
pash-openai merged 5 commits into
mainfrom
pash/codex-soul-developer-instructions

pash-openai commented May 19, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 19, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

pash-openai commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Token impact analysis

Uh oh!

clawsweeper Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented May 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pash-openai commented May 19, 2026 •

edited

Loading

clawsweeper Bot commented May 19, 2026 •

edited

Loading