GPT-5.4 runtime completion rollup by 100yenadmin · Pull Request #65217 · openclaw/openclaw

100yenadmin · 2026-04-12T06:55:55Z

Summary

This is the runtime completion rollup for the GPT-5.4 / Codex parity program.

It supersedes #64679 and keeps the runtime closeout in one readable merge unit.

What This Fixes From The Original GPT-5.4 Prompt

The original runtime complaint was that GPT-5.4 could make a good plan, stop, and ask for permission again on the next turn.

This rollup finishes the runtime side of that problem:

unconfigured GPT-5-family openai / openai-codex runs now auto-activate the strict-agentic contract
the planning-only retry guard uses the same GPT-5-family matcher as activation, so those paths cannot drift apart
the strict-agentic blocked exit now emits explicit replay/liveness metadata instead of looking like a vague terminal reply
executionContract: "default" still opts out cleanly

No new public runtime API is introduced here.

Verification

Branch-owned runtime coverage passed locally:

CI=1 pnpm exec vitest run \
  src/agents/execution-contract.test.ts \
  src/agents/pi-embedded-runner/run.incomplete-turn.test.ts \
  src/agents/openclaw-tools.update-plan.test.ts

On the merged runtime+proof integration branch, the new repo-instruction followthrough scenario also passed on the real QA harness:

pnpm openclaw qa suite \
  --provider-mode mock-openai \
  --model openai/gpt-5.4 \
  --alt-model openai/gpt-5.4-alt \
  --scenario instruction-followthrough-repo-contract \
  --output-dir .artifacts/qa-rollup/instruction-mock

That scenario passed without needing any extra runtime nudge beyond the behavior in this rollup.

Dependency Note

This PR is the runtime half of the 2-PR closeout. It pairs with the parity proof rollup PR, which supersedes the remaining qa-lab/docs proof slices.

…liveness Closes two hard blockers on the GPT-5.4 parity completion gate: 1) Criterion 1 (no stalls after planning) is universal, but the pre-existing strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5 openai / openai-codex users who never set `agents.defaults.embeddedPi.executionContract` still got only 1 planning-only retry and then fell through to the normal completion path with the plan-only text, i.e. they still stalled. Introduce `resolveEffectiveExecutionContract(...)` in src/agents/execution-contract.ts. Behavior: - supported provider/model (openai or openai-codex + gpt-5-family) AND explicit "strict-agentic" or unspecified → "strict-agentic" - supported provider/model AND explicit "default" → "default" (opt-out) - unsupported provider/model → "default" regardless of explicit value `isStrictAgenticExecutionContractActive` now delegates to the effective resolver so the 2-retry + blocked-state treatment applies by default to every GPT-5 openai/codex run. Explicit opt-out still works for users who intentionally want the pre-parity-program behavior. 2) Criterion 4 (replay/liveness failures are explicit, not silent disappearance) is violated by the strict-agentic blocked exit itself. Every other terminal return path in src/agents/pi-embedded-runner/run.ts sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`, but the strict-agentic exit at run.ts:1615 falls through without them. Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a `setTerminalLifecycleMeta` call so downstream observers (lifecycle log, ACP bridge, telemetry) see the same explicit terminal state they see on every other exit branch. Regressions added: - `auto-enables update_plan for unconfigured GPT-5 openai runs` - `respects explicit default contract opt-out on GPT-5 runs` - `does not auto-enable update_plan for non-openai providers even when unconfigured` - `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit` - `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state` - `respects explicit default contract opt-out on GPT-5 openai runs` Local validation: - pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts 122/122 passing. Refs #64227

Triages all three loop-6 review comments on PR #64679: 1. Copilot: 'The strict-agentic blocked exit returns an error payload (isError: true) but sets livenessState to "abandoned". Elsewhere in the runner/lifecycle flow, error terminal states are treated as "blocked".' Verified: every other hardcoded error terminal branch in run.ts (role ordering at 1152, image size at 1206, schema error at 1244, compaction timeout at 1128, aborted-with-no-payloads at 606) uses livenessState: "blocked". Match that convention at the strict-agentic blocked exit at 1634. Updated the 'emits explicit replayInvalid + abandoned liveness state' regression test to assert the new "blocked" value and renamed the assertion commentary. 2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says explicit "strict-agentic" in config always resolves to "strict-agentic", but the implementation collapses to "default" whenever the provider/mode is unsupported.' Rewrite the JSDoc to explicitly document the unsupported-provider collapse as the lead case (strict-agentic is a GPT-5-family openai/openai-codex-only runtime contract) before listing the supported-lane behavior matrix. No code change; this is a docstring-only clarification. 3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says to prefer sonnet-4.6 for Anthropic test constants.' Swap claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating fixtures that assert non-openai providers don't auto-enable the planning tool. Behavior unchanged; model constant now matches repo testing guidance. Local validation: - pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts 29/29 passing. Refs #64227

… blocked state Addresses loop-7 Copilot finding on PR #64679: loop 6 changed the assertion to livenessState === 'blocked' to match the rest of the hard-error terminal branches in run.ts, but the test title still said 'abandoned liveness state', which made failures and test output misleading. Rename the test title to match the asserted value. No code change beyond the it(...) title. Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts (19/19 pass). Refs #64227

…ariant GPT-5 model ids

…rollup/gpt54-runtime-completion

openclaw-barnacle · 2026-04-12T06:56:24Z

Closing this PR because the author has more than 10 active PRs in this repo. Please reduce the active PR queue and reopen or resubmit once it is back under the limit. You can close your own PRs to get back under the limit.

100yenadmin · 2026-04-12T06:56:24Z

Current verification on this rollup head:

branch-owned runtime suite passed: 44/44
the merged runtime+proof integration branch passed the new instruction-followthrough-repo-contract scenario on the real QA harness
that scenario passing without any extra runtime patch is the important signal here: the H behavior is enough to satisfy the original AGENT.md / SOUL.md followthrough proof when paired with the proof rollup

This PR supersedes #64679 and is intended to be the single runtime closeout PR for the GPT-5.4 work.

100yenadmin · 2026-04-12T06:58:08Z

GitHub auto-closed this PR because the author temporarily exceeded the repo's active-PR limit.

A fresh replacement PR from the same branch is now open at #65219.

Copilot

Pull request overview

This PR completes the runtime-side rollup for GPT-5.4 / Codex parity by making strict-agentic behavior the default for GPT‑5-family OpenAI runs (unless explicitly opted out), aligning GPT‑5 family detection across retry guards, and ensuring the strict-agentic blocked exit emits explicit liveness/replay metadata.

Changes:

Add an execution-contract resolver that auto-activates strict-agentic for unconfigured GPT‑5-family openai / openai-codex runs while preserving executionContract: "default" opt-out.
Reuse the same provider+model matcher for the planning-only retry guard (including provider-prefixed model IDs).
Emit replayInvalid + livenessState: "blocked" metadata on the strict-agentic blocked exit and add regression tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/agents/pi-embedded-runner/run/incomplete-turn.ts	Uses shared strict-agentic provider/model matcher for planning-only retry guard.
src/agents/pi-embedded-runner/run.ts	Adds explicit `replayInvalid` + `livenessState` and terminal lifecycle meta on strict-agentic blocked exit.
src/agents/pi-embedded-runner/run.incomplete-turn.test.ts	Adds coverage for blocked-exit metadata and auto-activation/opt-out behavior.
src/agents/openclaw-tools.update-plan.test.ts	Updates tests to reflect auto-enable behavior for GPT‑5 OpenAI runs and opt-out handling.
src/agents/execution-contract.ts	Introduces `resolveEffectiveExecutionContract` + shared GPT‑5-family matcher (including `provider/` and `provider:` prefixes).
src/agents/execution-contract.test.ts	Adds focused unit tests for execution-contract resolution and GPT‑5-family detection.

greptile-apps · 2026-04-12T07:03:39Z

Greptile Summary

This PR completes the runtime half of the GPT-5.4 parity closeout: unconfigured openai/openai-codex + gpt-5-family runs now auto-activate the strict-agentic contract, the planning-only retry guard is unified with the same isStrictAgenticSupportedProviderModel matcher to prevent drift, and the strict-agentic blocked exit now emits explicit livenessState/replayInvalid metadata. All changes are well-tested and the logic is sound. The only minor note is that new Anthropic model constants in the tests (claude-opus-4-6, claude-sonnet-4-6) don't follow the sonnet-4.6 format preferred in CLAUDE.md.

Confidence Score: 5/5

Safe to merge — no logic or correctness issues; only minor test naming convention gaps.

All P0/P1 concerns reviewed and none found. The regex, prefix-stripping, config resolution, retry-limit logic, and blocked-exit metadata are all correct and well-tested. Both findings are P2: a test model naming style guide gap and a clarification question about downstream replay-consumer behavior (which appears intentional per in-code comments).

No files require special attention.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: src/agents/execution-contract.test.ts
Line: 176-178

Comment:
**Preferred Anthropic model constant not used**

`CLAUDE.md` says to prefer `sonnet-4.6` for Anthropic model constants in tests. This file (and `openclaw-tools.update-plan.test.ts`) uses `claude-opus-4-6` / `claude-sonnet-4-6` (hyphen-separated, `claude-` prefixed) in tests added by this PR. These are new tests, so the "prefer" directive applies directly.

```suggestion
        modelId: "sonnet-4.6",
```

**Context Used:** CLAUDE.md ([source](https://app.greptile.com/review/custom-context?memory=fd949e91-5c3a-4ab5-90a1-cbe184fd6ce8))

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
Line: 143

Comment:
**`replayInvalid: false` on a strict-agentic blocked exit may mislead replay consumers**

At the strict-agentic blocked exit, `replayInvalid` is `false` because the plan-only transcript had no side-effect-producing tool calls. The comment in `run.ts` explains this is intentional. However, any consumer that automatically re-enqueues runs where `replayInvalid === false` would see `isError: true` + `livenessState: "blocked"` + `replayInvalid: false` and could enter a replay loop that hits the same blocked exit repeatedly.

Is there a downstream consumer that gates replays on `replayInvalid` alone, or do all such consumers also check `livenessState` / `isError` before retrying?

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "Merge branch 'fix/strict-agentic-default..." | Re-trigger Greptile}

greptile-apps · 2026-04-12T07:03:42Z

+          modelId: "claude-opus-4-6",
+        }),
+      ).toBe("default");


Preferred Anthropic model constant not used

CLAUDE.md says to prefer sonnet-4.6 for Anthropic model constants in tests. This file (and openclaw-tools.update-plan.test.ts) uses claude-opus-4-6 / claude-sonnet-4-6 (hyphen-separated, claude- prefixed) in tests added by this PR. These are new tests, so the "prefer" directive applies directly.

Suggested change

modelId: "claude-opus-4-6",

}),

).toBe("default");

modelId: "sonnet-4.6",

Context Used: CLAUDE.md (source)

Prompt To Fix With AI

This is a comment left during a code review. Path: src/agents/execution-contract.test.ts Line: 176-178 Comment: **Preferred Anthropic model constant not used** `CLAUDE.md` says to prefer `sonnet-4.6` for Anthropic model constants in tests. This file (and `openclaw-tools.update-plan.test.ts`) uses `claude-opus-4-6` / `claude-sonnet-4-6` (hyphen-separated, `claude-` prefixed) in tests added by this PR. These are new tests, so the "prefer" directive applies directly. ```suggestion modelId: "sonnet-4.6", ``` **Context Used:** CLAUDE.md ([source](https://app.greptile.com/review/custom-context?memory=fd949e91-5c3a-4ab5-90a1-cbe184fd6ce8)) How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

greptile-apps · 2026-04-12T07:03:43Z

+      },
+    ]);
+    expect(result.meta.livenessState).toBe("blocked");
+    expect(result.meta.replayInvalid).toBe(false);


replayInvalid: false on a strict-agentic blocked exit may mislead replay consumers

At the strict-agentic blocked exit, replayInvalid is false because the plan-only transcript had no side-effect-producing tool calls. The comment in run.ts explains this is intentional. However, any consumer that automatically re-enqueues runs where replayInvalid === false would see isError: true + livenessState: "blocked" + replayInvalid: false and could enter a replay loop that hits the same blocked exit repeatedly.

Is there a downstream consumer that gates replays on replayInvalid alone, or do all such consumers also check livenessState / isError before retrying?

Prompt To Fix With AI

This is a comment left during a code review. Path: src/agents/pi-embedded-runner/run.incomplete-turn.test.ts Line: 143 Comment: **`replayInvalid: false` on a strict-agentic blocked exit may mislead replay consumers** At the strict-agentic blocked exit, `replayInvalid` is `false` because the plan-only transcript had no side-effect-producing tool calls. The comment in `run.ts` explains this is intentional. However, any consumer that automatically re-enqueues runs where `replayInvalid === false` would see `isError: true` + `livenessState: "blocked"` + `replayInvalid: false` and could enter a replay loop that hits the same blocked exit repeatedly. Is there a downstream consumer that gates replays on `replayInvalid` alone, or do all such consumers also check `livenessState` / `isError` before retrying? How can I resolve this? If you propose a fix, please make it concise.

Eva added 6 commits April 12, 2026 02:48

agents: widen strict-agentic auto-activation to handle prefixed and v…

1c0111f

…ariant GPT-5 model ids

Align strict-agentic retry matching

cb35893

Merge branch 'fix/strict-agentic-default-plus-blocked-liveness' into …

4b94e9c

…rollup/gpt54-runtime-completion

Copilot AI review requested due to automatic review settings April 12, 2026 06:55

openclaw-barnacle Bot added agents Agent runtime and tooling size: L r: too-many-prs Auto-close: author has more than twenty active PRs. labels Apr 12, 2026

openclaw-barnacle Bot closed this Apr 12, 2026

100yenadmin mentioned this pull request Apr 12, 2026

GPT-5.4 / Codex agentic runtime parity in OpenClaw #64227

Closed

Copilot started reviewing on behalf of 100yenadmin April 12, 2026 06:56 View session

100yenadmin mentioned this pull request Apr 12, 2026

agents: (GPT 5.4 Parity vs. Opus Agentic) auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness #64679

Closed

Copilot AI reviewed Apr 12, 2026

View reviewed changes

greptile-apps Bot reviewed Apr 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

GPT-5.4 runtime completion rollup#65217

GPT-5.4 runtime completion rollup#65217
100yenadmin wants to merge 6 commits into
openclaw:mainfrom
electricsheephq:rollup/gpt54-runtime-completion

100yenadmin commented Apr 12, 2026

Uh oh!

openclaw-barnacle Bot commented Apr 12, 2026

Uh oh!

100yenadmin commented Apr 12, 2026

Uh oh!

100yenadmin commented Apr 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

greptile-apps Bot commented Apr 12, 2026

Uh oh!

greptile-apps Bot Apr 12, 2026

Uh oh!

greptile-apps Bot Apr 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

100yenadmin commented Apr 12, 2026

Summary

What This Fixes From The Original GPT-5.4 Prompt

Verification

Dependency Note

Uh oh!

openclaw-barnacle Bot commented Apr 12, 2026

Uh oh!

100yenadmin commented Apr 12, 2026

Uh oh!

100yenadmin commented Apr 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

greptile-apps Bot commented Apr 12, 2026

Greptile Summary

Confidence Score: 5/5

Uh oh!

greptile-apps Bot Apr 12, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Apr 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants