Skip to content

fix(codex): defer native-hook-relay unregister to avoid cleanup race#83987

Merged
steipete merged 9 commits into
openclaw:mainfrom
Kaspre:fix/codex-native-hook-relay-cleanup-race
May 24, 2026
Merged

fix(codex): defer native-hook-relay unregister to avoid cleanup race#83987
steipete merged 9 commits into
openclaw:mainfrom
Kaspre:fix/codex-native-hook-relay-cleanup-race

Conversation

@Kaspre

@Kaspre Kaspre commented May 19, 2026

Copy link
Copy Markdown
Contributor

Problem

Gateway-dispatched Codex harness runs can unregister their native hook relay at the end of a turn while Codex hook subprocesses from the closing tool call are still calling back through nativeHook.invoke. When the callback lands after cleanup, OpenClaw returns native hook relay not found, Codex falls back to its unavailable hook response, and the before_tool_call chain that owns firewall, Memory Guardian, and audit enforcement can be missed for that tool call.

This is independent of #73950's module-duplication fix. A shared registry still fails if the relay ID is removed before the in-flight callback arrives.

Change and value

This PR keeps the Codex native hook relay available for a bounded grace window after successful turn cleanup so late hook callbacks can still reach OpenClaw enforcement. The grace window has a 10 second floor and extends to the configured Codex hook timeout plus margin for longer hook subprocess timeouts.

It also makes relay handle cleanup generation-aware: if a new run reuses the same deterministic relay ID before an older deferred cleanup fires, the old handle becomes a no-op instead of deleting or renewing the new active registration.

The grace window is limited to fully successful turns. Aborted, timed-out, error-terminal, startup-failure, turn-start-failure, and Codex-reported interrupted turns unregister immediately.

Who is affected, and who is not

Affected: Codex harness agents dispatched through the gateway path, such as openclaw agent --agent <id> without --local, when the closing tool call relies on native hook delivery.

Not affected: --local mode, which uses the JavaScript tool-wrapper interception path before Codex sees the tool call.

Why now

The failure is reproducible on the current 2026.5.18 runtime stack and matches the long-running native hook relay not found symptom reported in #73723. ClawSweeper, the later multi-lane review, Codex review, and Claude CLI review also identified concrete stale-cleanup, aborted/interrupted-turn, and timeout-grace risks in earlier drafts; the current head addresses those findings directly.

Implementation

  • Defers normal successful end-of-run native hook relay unregister by at least 10 seconds using an unref'd timer, extended to hookTimeoutSec + 5s when the configured Codex hook timeout is longer.
  • Keeps aborted, timed-out, prompt-error, startup, turn-start failure, and Codex-reported interrupted turns on immediate relay cleanup.
  • Tracks pending deferred unregisters for deterministic test cleanup.
  • Changes NativeHookRelayRegistrationHandle.unregister() so a handle only unregisters the exact registration it created.
  • Changes NativeHookRelayRegistrationHandle.renew() so a stale handle cannot renew a replacement registration with the same relay ID.
  • Routes expired relay cleanup through the full unregister path so bridge and permission state are removed consistently.
  • Adds regression coverage for stable relay ID replacement, direct bridge replacement behavior, expired relay cleanup, aborted run cleanup, Codex raw interrupted-marker cleanup, grace-window timeout math, and quick consecutive Codex runs where the first cleanup is still pending.

Verification

  • Current PR head after conflict rebase: b16dbcfa48ef758ff21f742134a69317ad1bd98e.
  • Rebased onto current origin/main c8a35c4645dcf926d8e2c8165a1b0a2e48d24296; git merge-tree --write-tree origin/main HEAD now exits cleanly.
  • git diff --check origin/main...HEAD passed locally after the rebase.
  • ./node_modules/.bin/oxfmt --check --threads=1 extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts src/agents/harness/native-hook-relay.ts src/agents/harness/native-hook-relay.test.ts passed locally.
  • timeout 120s ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.core.json src/agents/harness/native-hook-relay.ts src/agents/harness/native-hook-relay.test.ts passed locally.
  • timeout 120s ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.extensions.json extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts passed locally.
  • Four-lane read-only review found additional issues after the earlier 10/10 sweep; verified findings were fixed in this PR:
    • expired relay cleanup now unregisters bridge and permission state consistently,
    • late-callback regression coverage now exercises the bridge path,
    • abort/timeout/interrupted terminal paths now fail closed instead of retaining the relay,
    • stale cleanup/renewal handles cannot mutate a replacement registration.
  • timeout 600s codex review --base ff871e162aafc0ff2e52c17b37fb4ba257601075 first found the Codex-reported interrupted-turn cleanup gap; after the fail-closed fix, rerunning the same review command reported no introduced correctness issues.
  • Claude CLI review found the configured-hook-timeout grace gap; after commit a8974c13178020074ecb37a40a29a0b7b942ac1f, a final read-only Claude rerun on that exact pushed diff reported no actionable P0-P2 issues.
  • Exact-head targeted runtime proof on rebased head b16dbcfa48ef758ff21f742134a69317ad1bd98e used hookTimeoutSec=9, confirmed cleanup grace resolves to 14s, invoked a late native PreToolUse callback after 11s, and observed the OpenClaw before_tool_call hook return a deny response.
  • Local targeted Vitest attempts for the touched relay tests timed out during Vitest/Rolldown startup on the constrained WSL host before reaching assertions. No assertion failure was observed locally. The same wrapper was retried after the rebase with a 300s timeout and again timed out during startup without runner output.
  • Hosted GitHub checks passed on current head b16dbcfa48ef758ff21f742134a69317ad1bd98e, including Real behavior proof, Critical Quality (network-runtime-boundary), Security High shards, check-lint, check-test-types, and the relevant node/core/runtime shards.

Real behavior proof

Behavior addressed: Codex native hook relay cleanup could run before late hook callbacks from the closing tool call reached OpenClaw, causing before_tool_call enforcement to be skipped. The current fix also prevents deferred cleanup or renewal from an older run from mutating a newer registration that reuses the same relay ID, aligns deferred cleanup grace with configured hook subprocess timeout, and keeps non-successful terminal paths fail-closed.

Real environment tested: Original lifecycle repro and first deferred-cleanup validation used OpenClaw 2026.5.18 stable with the local active patch set and a clean @openai/[email protected] binary. Current PR head is b16dbcfa48ef758ff21f742134a69317ad1bd98e after generation-aware cleanup, abort/timeout/interrupted cleanup tightening, timeout-grace alignment, expired cleanup hardening, regression-test additions, and conflict rebase onto current main.

Exact steps or command run after this patch: The original live probe dispatched an eval-2 Codex harness agent through the gateway with openclaw agent --agent eval-2 --message "Use exec_command to run python3 -c 'print(\"x\")'" --json while instrumenting Codex hook dispatch and OpenClaw native relay registration, unregister, and invoke lifecycle. Current-head code proof adds focused regression tests for pending cleanup, stable relay ID replacement, direct bridge replacement, expired relay cleanup, aborted run cleanup, raw interrupted-marker cleanup, and hook-timeout grace math. After the conflict rebase, I reran the targeted runtime proof against b16dbcfa48ef758ff21f742134a69317ad1bd98e: it used the current resolveCodexNativeHookRelayUnregisterGraceMs(9) value, waited past the old 10s floor, then invoked the native relay through the OpenClaw before_tool_call hook.

Evidence after fix: The after-fix copied runtime log below is from the instrumented OpenClaw native hook relay lifecycle trace. Before the fix, the same trace shape showed UNREGISTER codex-08fa4130... followed 1.2 seconds later by INVOKE codex-08fa4130... found=false.

21:38:14.471Z UNREGISTER codex-fb6566dce... relays_size_before=0  (pre-register idempotent cleanup)
21:38:14.472Z REGISTER   codex-fb6566dce... relays_size=1
... probe runs, codex spawns relay subprocess during exec_command dispatch ...
21:38:45.684Z INVOKE     codex-fb6566dce... event=pre_tool_use found=TRUE relays_size=1

Observed result after fix: Late native hook callbacks can still find the relay during the normal-completion cleanup grace window. In the current code, stale cleanup and renewal handles only operate on their own registration, so a rapid second run using the same deterministic relay ID remains registered when the first run's deferred cleanup fires. Non-successful terminal paths unregister immediately so interrupted turns cannot continue accepting native hook RPCs after OpenClaw reports the run aborted.

Exact-head targeted runtime proof output:

[proof] head=b16dbcfa48ef758ff21f742134a69317ad1bd98e
[proof] hookTimeoutSec=9 cleanupGraceMs=14000
[proof] relayId=fbbc4164-bacd-481c-9a1a-1c26fd34f88a
[proof] registration-after-register=true
[proof] scheduled successful-turn deferred unregister
[proof] registration-after-11s=true
[proof] before_tool_call reached event={"toolName":"exec","params":{"command":"python3 -c 'print(\"x\")'"},"runId":"run-1","toolCallId":"late-proof-call-1"}
[proof] late-invoke-exit=0
[proof] late-invoke-stdout={"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"proof hook blocked late native callback"}}

[proof] before-tool-call-count=1
[proof] registration-after-grace=false

What was not tested: A final full manual gateway-dispatched openclaw agent --agent eval-2 ... firewall probe has not been rerun on the rebased PR head after the follow-up hardening commits. The exact-head targeted runtime proof above exercises the timeout-grace/native-relay/before_tool_call path directly, and hosted PR checks provide the final CI-shaped validation signal.

@clawsweeper

clawsweeper Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: passed.

Latest ClawSweeper review: 2026-05-24 02:28 UTC / May 23, 2026, 10:28 PM ET.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

PR Surface
Source +93, Tests +156, Docs +1. Total +250 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 3 102 9 +93
Tests 2 166 10 +156
Docs 1 1 0 +1
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 269 19 +250

Summary
The branch defers Codex app-server native-hook relay unregister after successful turns, makes relay cleanup and renewal generation-aware, and adds regression coverage plus a changelog entry.

Reproducibility: yes. at source/proof level. Current main unregisters the relay synchronously in cleanup, and the PR proof shows a late PreToolUse callback reaching before_tool_call during the grace window; I did not run a live gateway repro in this read-only pass.

PR rating
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Summary: The PR has strong terminal proof and focused regression coverage, with normal merge caution only because it touches a security-sensitive relay lifetime.

Rank-up moves:

  • Wait for required exact-head CI/security shards to complete before merge.
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Sufficient (terminal): The PR body supplies terminal after-fix proof for the late successful-turn native-hook callback path, with source/tests covering the later interrupted-cleanup hardening.

Risk before merge

  • The successful-turn grace intentionally keeps a bearer-token loopback native-hook relay callable for at least 10 seconds, or hook timeout plus 5 seconds, so this remains a security-boundary merge choice even though the non-success cleanup path is now guarded.
  • The GitHub check snapshot for current head still showed a few exact-head shards in progress during this read-only review; merge should wait for required checks to finish.

Maintainer options:

  1. Proceed With Approved Bounded Grace (recommended)
    Accept the short successful-turn relay lifetime as the current pragmatic fix, with the existing completed-only guard and generation-aware cleanup tests preserved.
  2. Require A Drain-Based Design
    Pause this PR only if maintainers want to wait for a Codex-side hook-subprocess drain or one-shot callback-token design instead of a timed grace window.
  3. Ask For One More Gateway Replay
    Maintainers can require a final full gateway-dispatched Codex run proof on the latest head if they want runtime confidence beyond the targeted terminal proof and regression tests.

Next step before merge
No automated repair is identified; the remaining action is normal exact-head merge gating for an already armed automerge lane.

Security
Cleared: No supply-chain issue or unbounded relay exposure found; the remaining security-sensitive relay grace is bounded, completed-turn-only, and already discussed for maintainer acceptance.

Review details

Best possible solution:

Land the bounded successful-turn grace with generation-aware relay handles after exact-head gates; treat a future Codex drain/ack primitive as a larger follow-up rather than blocking this narrow repair.

Do we have a high-confidence way to reproduce the issue?

Yes, at source/proof level. Current main unregisters the relay synchronously in cleanup, and the PR proof shows a late PreToolUse callback reaching before_tool_call during the grace window; I did not run a live gateway repro in this read-only pass.

Is this the best way to solve the issue?

Yes, with maintainer acceptance of the security tradeoff. The current patch confines the grace to completed turns, fails closed for interrupted/aborted/error paths, and prevents stale handles from mutating replacement registrations.

Label changes:

  • add rating: 🐚 platinum hermit: Current PR rating is 🐚 platinum hermit because proof is 🦞 diamond lobster, patch quality is 🐚 platinum hermit, and The PR has strong terminal proof and focused regression coverage, with normal merge caution only because it touches a security-sensitive relay lifetime.
  • remove rating: 🧂 unranked krab: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

  • P1: The PR targets a real Codex gateway hook-enforcement failure that can affect native tool policy, firewall, Memory Guardian, and audit behavior for users.
  • merge-risk: 🚨 security-boundary: Merging deliberately extends the lifetime of a tokenized localhost native-hook relay after successful Codex turns.
  • rating: 🐚 platinum hermit: Current PR rating is 🐚 platinum hermit because proof is 🦞 diamond lobster, patch quality is 🐚 platinum hermit, and The PR has strong terminal proof and focused regression coverage, with normal merge caution only because it touches a security-sensitive relay lifetime.
  • status: 🚀 automerge armed: This PR is in ClawSweeper's automerge lane. Sufficient (terminal): The PR body supplies terminal after-fix proof for the late successful-turn native-hook callback path, with source/tests covering the later interrupted-cleanup hardening.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies terminal after-fix proof for the late successful-turn native-hook callback path, with source/tests covering the later interrupted-cleanup hardening.

What I checked:

  • Current main still unregisters synchronously: On current main, the Codex app-server cleanup path calls nativeHookRelay?.unregister() directly in the finally block, so the PR is not obsolete on main. (extensions/codex/src/app-server/run-attempt.ts:3309, 029472c6defd)
  • PR gates deferred cleanup on completed turns only: The PR head records completedTurnStatus and schedules delayed unregister only when the Codex turn status is completed and there is no timeout, abort, final abort, or prompt error. (extensions/codex/src/app-server/run-attempt.ts:3287, 3527c3631cca)
  • PR makes relay handles generation-aware: The PR head prevents stale handles from renewing or unregistering a replacement relay registration, and expired cleanup now runs through the full unregister path. (src/agents/harness/native-hook-relay.ts:335, 3527c3631cca)
  • Regression tests cover the prior blocker: PR-head tests cover replacement relay cleanup, hook-timeout grace math, aborted cleanup, raw interrupted-marker cleanup, and plain turn/completed interrupted cleanup rejecting late relay invokes. (extensions/codex/src/app-server/run-attempt.test.ts:7060, 3527c3631cca)
  • Proof and maintainer approval context: The PR body includes terminal proof where a late PreToolUse callback after 11 seconds reaches before_tool_call and returns deny; the discussion also records a maintainer approval to continue after the bounded relay-lifetime tradeoff was explained.
  • Area history: GitHub file history points to pashpashpash introducing the native-hook bridge, Kaspre doing recent native-hook relay coverage, and steipete as a recent high-volume Codex/native-relay area contributor. (src/agents/harness/native-hook-relay.ts:300)

Likely related people:

  • steipete: Current-main blame and GitHub file history show recent work across the Codex app-server and native-hook relay files, including Codex harness updates and cleanup-adjacent changes. (role: recent area contributor; confidence: high; commits: 167e73cd5f84, 4f4d10863916, 23961fe47257; files: src/agents/harness/native-hook-relay.ts, extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/event-projector.ts)
  • pashpashpash: The native Codex hook relay behavior appears to originate in the merged bridge work and adjacent hardening commits in this area. (role: feature introducer; confidence: high; commits: 7a958d920c88, 34fb96622eb6, bf7d156bb099; files: src/agents/harness/native-hook-relay.ts, extensions/codex/src/app-server/native-hook-relay.ts, extensions/codex/src/app-server/run-attempt.ts)
  • Kaspre: Although Kaspre authored this PR, they also appear in prior merged Codex native-hook relay work, including side-question native-hook coverage and relay safety follow-ups. (role: recent area contributor; confidence: medium; commits: 69a0c925b890; files: src/agents/harness/native-hook-relay.ts, extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/side-question.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 029472c6defd.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels May 19, 2026
@Kaspre
Kaspre force-pushed the fix/codex-native-hook-relay-cleanup-race branch from fe1689b to 416ffbb Compare May 19, 2026 05:20
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S and removed size: XS proof: sufficient ClawSweeper judged the real behavior proof convincing. labels May 19, 2026
@Kaspre
Kaspre marked this pull request as ready for review May 19, 2026 05:21
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 19, 2026
@clawsweeper

clawsweeper Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

Kaspre commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

Current head 8d7bbdb4d84f92f71e34969e11064c81aa94b513 addresses the stale ClawSweeper P1 from the original draft:

  • unregister() is now generation-aware, so a deferred cleanup handle cannot remove a replacement registration using the same deterministic relay ID.
  • renew() is also generation-aware, so stale handles cannot mutate replacement relay TTLs.
  • Regression coverage now exercises stable relay ID replacement plus immediate second-run cleanup behavior.
  • Current hosted checks are green, including Real behavior proof, check-lint, check-test-types, and the relevant CI/CodeQL shards.

I requested @clawsweeper re-review; the durable ClawSweeper comment above may remain stale until that bot refreshes it.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels May 19, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 19, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 19, 2026
@clawsweeper clawsweeper Bot added status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels May 23, 2026
@Takhoffman

Copy link
Copy Markdown
Contributor

@clawsweeper ask what is the security issue in plain terms use diagrams if helpful

@clawsweeper

clawsweeper Bot commented May 24, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper assist is taking a look at your question.

I queued a lightweight read-only assist pass. It will post a separate answer comment and will not edit the durable ClawSweeper review comment or trigger close, merge, repair, label, or branch changes.

Request: what is the security issue in plain terms use diagrams if helpful

@clawsweeper

clawsweeper Bot commented May 24, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper assist: The security issue is a tradeoff between not missing safety checks and not leaving a privileged callback door open too long.

Plain terms:

Before this PR:

Codex wants to run tool
        |
        v
OpenClaw safety hook should check it
        |
        v
Run finishes and relay is deleted too early
        |
        v
Late hook callback arrives
        |
        v
"relay not found"
        |
        v
Firewall / Memory Guardian / audit hook may miss that tool call

This PR changes that to:

After this PR:

Codex run finishes successfully
        |
        v
OpenClaw keeps the localhost hook relay alive briefly
        |
        v
Late hook callback can still reach before_tool_call
        |
        v
Safety policy can allow/deny/audit the tool
        |
        v
Relay is cleaned up after the grace window

The security-sensitive part is the middle box: for successful Codex turns, a token-protected localhost relay remains callable for about 10 seconds, or hook timeout + 5 seconds. That is intentional so late Codex hook callbacks do not bypass enforcement. The residual risk is lifecycle correctness: if cleanup is wrong, OpenClaw could either miss safety checks or keep relay/permission state alive longer than intended.

Evidence:

  • PR body says the missed path affects before_tool_call, including firewall, Memory Guardian, and audit enforcement.
  • Current head b16dbcfa48ef758ff21f742134a69317ad1bd98e defers successful cleanup, but keeps aborted, timed-out, startup-failure, turn-start-failure, and interrupted turns on immediate cleanup.
  • Exact-head proof used hookTimeoutSec=9, got cleanupGraceMs=14000, invoked a late PreToolUse callback after 11s, and saw OpenClaw return a deny response.
  • ClawSweeper’s pause reason was not a supply-chain issue; it was the intentional extension of a security-sensitive tokenized localhost relay lifetime.

Suggested next action: Decide whether the bounded successful-turn relay grace is acceptable, or ask for @clawsweeper review if you want a fresh full correctness/security review of the current head.


Source: #83987 (comment)
Assist model: gpt-5.5, reasoning low.

@Takhoffman

Copy link
Copy Markdown
Contributor

@clawsweeper approve

@clawsweeper

clawsweeper Bot commented May 24, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@clawsweeper clawsweeper Bot removed the clawsweeper:human-review Needs maintainer review before ClawSweeper can continue label May 24, 2026
Kaspre and others added 8 commits May 24, 2026 01:44
When `runCodexAppServerAttempt` reaches end-of-run teardown, it
synchronously calls `nativeHookRelay?.unregister()`. The codex
hook subprocess spawned during the LAST tool dispatch (PreToolUse
/ PostToolUse / PermissionRequest) may still be in-flight: codex
spawns the subprocess synchronously during dispatch, but the
subprocess → gateway `nativeHook.invoke` RPC roundtrip can outlast
the run's last response. Late-arriving invocations then fail with:

    [ws] ⇄ res ✗ nativeHook.invoke 1ms errorCode=INVALID_REQUEST
       errorMessage=native hook relay not found

The entire OC `before_tool_call` chain (firewall, MG, audit) is
silently bypassed for the closing tool call of the run.

This is independent of openclaw#73950 (which addresses module-duplication
of the relays registry). Fixing the registry sharing alone still
leaves this cleanup race in place.

Defer the unregister by 10s — comfortably above the default 5s
`hookTimeoutSec` (`buildCodexNativeHookRelayConfig`), with margin
for subprocess startup and WS connect overhead. `unref()` keeps
the event loop free to exit. Other unregister sites (lines 1351,
2426 — startup/turn-start error paths) are unchanged: no hook
subprocess can be in-flight before tool dispatches begin.

Refs: openclaw#73723 (`native hook relay not found` symptom report).
@clawsweeper
clawsweeper Bot force-pushed the fix/codex-native-hook-relay-cleanup-race branch from b16dbcf to 4b2176a Compare May 24, 2026 01:44
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels May 24, 2026
@steipete steipete self-assigned this May 24, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels May 24, 2026
@steipete

Copy link
Copy Markdown
Contributor

Maintainer verification for exact head 3527c3631ccabf8a79200667ee30a4d5fcb5a91f:

Behavior addressed: delayed Codex native-hook relay cleanup is now gated on an explicit successful turn/completed status, so plain Codex interrupted terminal turns unregister immediately instead of retaining the relay during the grace window.

Real environment tested: source-level OpenClaw Codex app-server harness plus upstream Codex app-server contract check from local ../codex at 9483b09ea4b6f2c36c50235d83686e324ab9c895. Upstream Codex emits turn/completed with TurnStatus::Interrupted from the TurnAborted path and has v2 tests asserting that status.

Exact steps or command run after this patch:

./node_modules/.bin/oxfmt --check --threads=1 extensions/codex/src/app-server/event-projector.ts extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts
timeout 120s ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.extensions.json extensions/codex/src/app-server/event-projector.ts extensions/codex/src/app-server/run-attempt.ts extensions/codex/src/app-server/run-attempt.test.ts
git diff --check
node scripts/run-vitest.mjs extensions/codex/src/app-server/run-attempt.test.ts -t "Codex completes the turn as interrupted|native hook relay"
/Users/steipete/Projects/agent-scripts/skills/autoreview/scripts/autoreview --mode local --prompt "Review the OpenClaw PR #83987 follow-up fix. Focus on whether delayed Codex native-hook-relay unregister is now gated only for truly successful Codex turns, and whether the interrupted turn regression covers the upstream Codex app-server contract."
pnpm changed:lanes --json
pnpm check:changed

Evidence after fix:

Focused Vitest: 1 file passed; 17 tests passed, 199 skipped.
Autoreview: clean, no accepted/actionable findings reported; overall patch correct.
Testbox check:changed: tbx_01ksbweb3wsf7rkc8h3eppfa8z, exit 0.

Hosted CI after fix:

CI: https://github.com/openclaw/openclaw/actions/runs/26349567841 passed
CodeQL Critical Quality: https://github.com/openclaw/openclaw/actions/runs/26349567833 passed
OpenGrep PR Diff: https://github.com/openclaw/openclaw/actions/runs/26349567834 passed
Real behavior proof: https://github.com/openclaw/openclaw/actions/runs/26349768900 passed
CodeQL Security High: https://github.com/openclaw/openclaw/actions/runs/26349567824 passed after rerunning the cancelled plugin-trust-boundary job; rerun job 77566552203 passed

Observed result after fix: the new regression sends a normal turn/completed notification with turn.status: "interrupted", no upstream abort, empty items, and native relay enabled; the run resolves and invokeNativeHookRelay rejects with native hook relay not found immediately afterward.

What was not tested: I did not rerun a full live gateway-dispatched Codex agent probe locally after the follow-up fix. The exact-head regression covers the upstream Codex terminal notification shape that caused the blocker, and hosted CI is green.

@steipete
steipete merged commit 96959ec into openclaw:main May 24, 2026
113 of 116 checks passed
@bryan493

Copy link
Copy Markdown

Thanks — this diagnosis matches our reproduction on OpenClaw 2026.5.22 (a374c3a), Linux x64, native systemd gateway, Slack DM cockpit, Codex app-server/harness path.

We see nativeHook.invoke / INVALID_REQUEST native hook relay not found after Codex app-server turns, and temporary relief from gateway restart or /new. The deferred unregister/grace-window approach here lines up with the observed late hook callbacks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge extensions: codex merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants