agents: (GPT 5.4 Parity vs. Opus Agentic) auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness#64679
Conversation
Greptile SummaryThis PR closes two runtime contract gaps in the GPT-5.4 parity gate: (1) Confidence Score: 5/5Safe to merge — changes are well-scoped, correctly implemented, and covered by new regressions. All remaining findings are P2 (one minor test model-constant style issue per CLAUDE.md conventions). The core logic in execution-contract.ts and run.ts is correct, backwards-compatible, and the behavioral changes are intentional and explicitly documented. No files require special attention. Prompt To Fix All With AIThis is a comment left during a code review.
Path: src/agents/openclaw-tools.update-plan.test.ts
Line: 98
Comment:
**Non-preferred Anthropic model constant**
`CLAUDE.md` says to prefer `sonnet-4.6` for Anthropic model constants in tests, and to update such constants whenever the file is touched. This test was newly introduced in this PR, so it should use the preferred constant.
```suggestion
modelId: "sonnet-4.6",
```
**Context Used:** CLAUDE.md ([source](https://app.greptile.com/review/custom-context?memory=fd949e91-5c3a-4ab5-90a1-cbe184fd6ce8))
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "agents: auto-activate strict-agentic for..." | Re-trigger Greptile |
There was a problem hiding this comment.
Pull request overview
This PR closes two runtime-contract gaps in the GPT‑5 parity work by (1) auto-activating the strict-agentic execution contract for supported GPT‑5 OpenAI/Codex runs when executionContract is unspecified (while preserving explicit "default" opt-out) and (2) ensuring the strict-agentic “planning-only retries exhausted” terminal exit emits replay/liveness metadata so observers don’t see a silent terminal transition.
Changes:
- Add
resolveEffectiveExecutionContract(...)and route strict-agentic activation checks through it to auto-enable strict-agentic for supported GPT‑5 OpenAI/Codex runs when unconfigured. - Thread
replayInvalid+livenessStatethrough the strict-agentic blocked-exit return path andattempt.setTerminalLifecycleMeta?.(...). - Add/expand regression tests covering auto-activation behavior, opt-out behavior, and terminal replay/liveness emission.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/agents/pi-embedded-runner/run.ts | Adds replay/liveness terminal metadata emission for the strict-agentic blocked exit and types import for liveness state. |
| src/agents/pi-embedded-runner/run.incomplete-turn.test.ts | Adds regressions for strict-agentic blocked-exit liveness + GPT‑5 auto-activation and explicit default opt-out behavior. |
| src/agents/openclaw-tools.update-plan.test.ts | Updates gating tests to reflect strict-agentic auto-activation for unconfigured GPT‑5 OpenAI/Codex runs and explicit default opt-out. |
| src/agents/execution-contract.ts | Introduces an effective execution-contract resolver and updates strict-agentic activation logic to use it. |
41748de to
3913e9d
Compare
3913e9d to
f811b4d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f811b4d773
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
7b7a7da to
3df6f0b
Compare
…liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs #64227
Triages all three loop-6 review comments on PR #64679: 1. Copilot: 'The strict-agentic blocked exit returns an error payload (isError: true) but sets livenessState to "abandoned". Elsewhere in the runner/lifecycle flow, error terminal states are treated as "blocked".' Verified: every other hardcoded error terminal branch in run.ts (role ordering at 1152, image size at 1206, schema error at 1244, compaction timeout at 1128, aborted-with-no-payloads at 606) uses livenessState: "blocked". Match that convention at the strict-agentic blocked exit at 1634. Updated the 'emits explicit replayInvalid + abandoned liveness state' regression test to assert the new "blocked" value and renamed the assertion commentary. 2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says explicit "strict-agentic" in config always resolves to "strict-agentic", but the implementation collapses to "default" whenever the provider/mode is unsupported.' Rewrite the JSDoc to explicitly document the unsupported-provider collapse as the lead case (strict-agentic is a GPT-5-family openai/openai-codex-only runtime contract) before listing the supported-lane behavior matrix. No code change; this is a docstring-only clarification. 3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says to prefer sonnet-4.6 for Anthropic test constants.' Swap claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating fixtures that assert non-openai providers don't auto-enable the planning tool. Behavior unchanged; model constant now matches repo testing guidance. Local validation: - pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts 29/29 passing. Refs #64227
… blocked state Addresses loop-7 Copilot finding on PR #64679: loop 6 changed the assertion to livenessState === 'blocked' to match the rest of the hard-error terminal branches in run.ts, but the test title still said 'abandoned liveness state', which made failures and test output misleading. Rename the test title to match the asserted value. No code change beyond the it(...) title. Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts (19/19 pass). Refs #64227
…ariant GPT-5 model ids
3df6f0b to
cb35893
Compare
|
Current head
I also reran |
|
Current head: Maintainer update after the fresh local verification pass:
I do not see remaining branch-owned review work here. If GitHub is still red, the remaining failures are the shared mainline type/check noise outside this diff, not this PR’s runtime scope. |
|
@steipete remaining GPT 5.4 fixes are ready for review. Built and completed live harness test (see PR for it + you can test as well if you like or use in future for tuning for models to openclaw by provider/model). Passes 10/10 parity scenario test side by side vs Opus 4.6 in Openclaw. |
|
Superseded by #65217. The runtime changes, tests, and current verification were carried forward into the runtime completion rollup so maintainers only need to review one runtime closeout PR. Closing this preserves the discussion history without splitting review attention. |
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs #64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR #64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs #64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR #64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs #64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs openclaw#64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR openclaw#64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs openclaw#64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR openclaw#64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs openclaw#64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs openclaw#64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR openclaw#64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs openclaw#64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR openclaw#64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs openclaw#64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs openclaw#64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR openclaw#64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs openclaw#64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR openclaw#64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs openclaw#64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs openclaw#64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR openclaw#64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs openclaw#64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR openclaw#64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs openclaw#64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs openclaw#64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR openclaw#64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs openclaw#64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR openclaw#64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs openclaw#64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
* agents: auto-activate strict-agentic for GPT-5 and emit blocked-exit liveness
Closes two hard blockers on the GPT-5.4 parity completion gate:
1) Criterion 1 (no stalls after planning) is universal, but the pre-existing
strict-agentic execution contract was opt-in only. Out-of-the-box GPT-5
openai / openai-codex users who never set
`agents.defaults.embeddedPi.executionContract` still got only 1
planning-only retry and then fell through to the normal completion path
with the plan-only text, i.e. they still stalled.
Introduce `resolveEffectiveExecutionContract(...)` in
src/agents/execution-contract.ts. Behavior:
- supported provider/model (openai or openai-codex + gpt-5-family) AND
explicit "strict-agentic" or unspecified → "strict-agentic"
- supported provider/model AND explicit "default" → "default" (opt-out)
- unsupported provider/model → "default" regardless of explicit value
`isStrictAgenticExecutionContractActive` now delegates to the effective
resolver so the 2-retry + blocked-state treatment applies by default to
every GPT-5 openai/codex run. Explicit opt-out still works for users who
intentionally want the pre-parity-program behavior.
2) Criterion 4 (replay/liveness failures are explicit, not silent
disappearance) is violated by the strict-agentic blocked exit itself.
Every other terminal return path in src/agents/pi-embedded-runner/run.ts
sets `replayInvalid` + `livenessState` via `setTerminalLifecycleMeta`,
but the strict-agentic exit at run.ts:1615 falls through without them.
Add explicit `livenessState: "abandoned"` + `replayInvalid` (via the
shared `resolveReplayInvalidForAttempt` helper) to that exit, plus a
`setTerminalLifecycleMeta` call so downstream observers (lifecycle log,
ACP bridge, telemetry) see the same explicit terminal state they see on
every other exit branch.
Regressions added:
- `auto-enables update_plan for unconfigured GPT-5 openai runs`
- `respects explicit default contract opt-out on GPT-5 runs`
- `does not auto-enable update_plan for non-openai providers even when unconfigured`
- `emits explicit replayInvalid + abandoned liveness state at the strict-agentic blocked exit`
- `auto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked state`
- `respects explicit default contract opt-out on GPT-5 openai runs`
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts src/agents/pi-embedded-runner.buildembeddedsandboxinfo.test.ts src/agents/system-prompt.test.ts src/agents/openclaw-tools.sessions.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.test.ts
122/122 passing.
Refs openclaw#64227
* agents: address loop-6 review comments on strict-agentic contract
Triages all three loop-6 review comments on PR openclaw#64679:
1. Copilot: 'The strict-agentic blocked exit returns an error payload
(isError: true) but sets livenessState to "abandoned". Elsewhere in
the runner/lifecycle flow, error terminal states are treated as
"blocked".' Verified: every other hardcoded error terminal branch in
run.ts (role ordering at 1152, image size at 1206, schema error at
1244, compaction timeout at 1128, aborted-with-no-payloads at 606)
uses livenessState: "blocked". Match that convention at the
strict-agentic blocked exit at 1634. Updated the 'emits explicit
replayInvalid + abandoned liveness state' regression test to assert
the new "blocked" value and renamed the assertion commentary.
2. Copilot: 'The JSDoc for resolveEffectiveExecutionContract says
explicit "strict-agentic" in config always resolves to
"strict-agentic", but the implementation collapses to "default"
whenever the provider/mode is unsupported.' Rewrite the JSDoc to
explicitly document the unsupported-provider collapse as the lead
case (strict-agentic is a GPT-5-family openai/openai-codex-only
runtime contract) before listing the supported-lane behavior matrix.
No code change; this is a docstring-only clarification.
3. Greptile P2: 'Non-preferred Anthropic model constant. CLAUDE.md says
to prefer sonnet-4.6 for Anthropic test constants.' Swap
claude-opus-4-6 → claude-sonnet-4-6 in the two update_plan gating
fixtures that assert non-openai providers don't auto-enable the
planning tool. Behavior unchanged; model constant now matches repo
testing guidance.
Local validation:
- pnpm test src/agents/openclaw-tools.update-plan.test.ts src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
29/29 passing.
Refs openclaw#64227
* test: rename strict-agentic blocked-exit liveness regression to match blocked state
Addresses loop-7 Copilot finding on PR openclaw#64679: loop 6 changed the
assertion to livenessState === 'blocked' to match the rest of the
hard-error terminal branches in run.ts, but the test title still said
'abandoned liveness state', which made failures and test output
misleading. Rename the test title to match the asserted value. No
code change beyond the it(...) title.
Validation: pnpm test src/agents/pi-embedded-runner/run.incomplete-turn.test.ts
(19/19 pass).
Refs openclaw#64227
* agents: widen strict-agentic auto-activation to handle prefixed and variant GPT-5 model ids
* Align strict-agentic retry matching
* runtime: harden strict-agentic model matching
---------
Co-authored-by: Eva <[email protected]>
Summary
Two runtime fixes that close the remaining holes on completion-gate criteria 1 and 4 from #64227. Both are places where the first-wave parity work (PRs A #64241 and C #64300) did the right thing under an opt-in strict-agentic contract but left the default path short of the bar.
openaiandopenai-codexusers get the 2-retry + blocked-state treatment automatically. ExplicitexecutionContract: "default"still opts out.run.tsnow setsreplayInvalid+livenessState+ callssetTerminalLifecycleMeta.Stacked conceptually on PR A (#64241, merged) and PR C (#64300, merged); refines both without re-opening them. Part of #64227.
What changed
1. Default-contract stall blocker
src/agents/execution-contract.tsgets a newresolveEffectiveExecutionContract(params):"strict-agentic""strict-agentic""default""default"(opt-out)"strict-agentic"← new"default""strict-agentic""default"(inert)isStrictAgenticExecutionContractActivedelegates to the resolver, so every caller — theupdate_planauto-enable inopenclaw-tools.registration.ts, the 2-retry cap inpi-embedded-runner/run.ts, the blocked-state branch in the same file — picks up the new semantics in lockstep.Before: a user on
openai/gpt-5.4withoutexecutionContractset got 1 retry, then the code fell through to the normal success path and returned the plan-only assistant text as a completion. Criterion 1 was violated on the default install.After: the same user gets the strict-agentic 2-retry cap and then
STRICT_AGENTIC_BLOCKED_TEXTwithisError: true. Criterion 1 is satisfied for the universal statement in #64227.2. Blocked-exit liveness
src/agents/pi-embedded-runner/run.tsalready threadsreplayInvalid+livenessStatethrough every other early-return branch inrunEmbeddedPiAgent, plus aattempt.setTerminalLifecycleMeta?.(...)call and the returnedmetaobject. The strict-agentic blocked exit added in PR A #64241 didn't do any of that, so it was silent disappearance at exactly the point where the parity program said blocked state must be loudest.This PR adds the terminal-state threading to that exit:
livenessState: "blocked". TheEmbeddedRunLivenessStateunion is"working" | "paused" | "blocked" | "abandoned"."blocked"is the accurate label here because the run is waiting on an explicit act-now decision from the contract, not because the user walked away (which is what"abandoned"means in the same contract).replayInvalid, delegated to the sharedresolveReplayInvalidForAttempt(null)helper. The plan-only transcript is typically replay-safe even though the run is terminal, so the helper returnsfalse, but routing through it keeps the replay-metadata edge cases consistent with the rest of the file.attempt.setTerminalLifecycleMeta?.(...), using the same call pattern as every other terminal branch.Regressions added
auto-enables update_plan for unconfigured GPT-5 openai runsrespects explicit default contract opt-out on GPT-5 runsdoes not auto-enable update_plan for non-openai providers even when unconfiguredemits explicit replayInvalid + blocked liveness state at the strict-agentic blocked exitauto-activates strict-agentic for unconfigured GPT-5 openai runs and surfaces the blocked staterespects explicit default contract opt-out on GPT-5 openai runsValidation
→ 122/122 passing. Rebased on green
main(HEAD545490c592at push time).Non-goals
EmbeddedRunLivenessStateenum or add new statesgpt-5model-prefix gate