-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Codex PreToolUse native hook relay spawns CPU-bound openclaw-hooks processes and stalls gateway RPC #91009
Copy link
Copy link
Open
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Summary
On OpenClaw
2026.6.1with the bundled@openclaw/codexintegration, Codex app-server tool calls can spawn multiple short-livedopenclaw-hooks/openclaw hooks relay --provider codex --event pre_tool_useprocesses. In the observed case these processes consumed ~100%+ CPU each, the gateway event loop became degraded, and local gateway RPC calls (health/status) timed out after 10s even thoughopenclaw-gateway.servicestayed alive and the agent session completed successfully.This looks like a core Codex native hook relay issue rather than a Telegram/systemd/disk/RAM problem.
Environment
2026.6.1 (2e08f0f)@openclaw/codex:2026.6.122.22.2node --max-old-space-size=12288 .../openclaw/dist/index.js gateway --port 18789codex app-server --listen stdio://)tools.loopDetection.enabled=trueopenclaw hooks list --jsonshowed only bundled/internal hooks (boot-md,bootstrap-extra-files,command-logger,compaction-notifier,session-memory)before_tool_callhooks and no trusted tool policies were configuredObserved behavior
During an agent turn with multiple tool calls:
openclaw gateway call health --jsonandopenclaw gateway call status --jsonfailed with:The gateway service did not crash. The active agent session later ended with
status=success,timedOut=false.pssnapshots during the degraded window showed several concurrent hook relay processes:One parent command was:
healthcall succeeded but reported degraded event loop:{ "ok": true, "durationMs": 32, "eventLoop": { "degraded": true, "reasons": ["event_loop_delay", "event_loop_utilization", "cpu"], "delayP99Ms": 3347.1, "utilization": 1, "cpuCoreRatio": 1.133 } }Code path that looks suspicious
In the installed bundle,
@openclaw/codex/dist/harness.jshard-enables native hook relay for both attempts and side questions:The Codex native hook relay config maps
pre_tool_useto CodexPreToolUseand registers a synchronous command hook. The installed bundle also appears to special-casepre_tool_useso it can still install aPreToolUsehook even whenshouldRelayEvent(event)is false/no-op.In the gateway bundle,
nativeHookRelayEventHasLocalWork(registration, "pre_tool_use")returns true when either abefore_tool_callpolicy exists or loop detection may run. Withtools.loopDetection.enabled=true, this makesPreToolUserelay active even without custom hooks or trusted policies.The resulting command path is a separate CLI process per hook invocation:
That per-tool-call process fan-out appears to be enough to starve the gateway/event loop under normal Codex tool bursts.
Expected behavior
health,status) should stay responsive during Codex tool-call bursts.PreToolUsepath should not spawn a heavyweight CLI process per tool call.PreToolUseshould only be registered when it has real local work, or it should be handled in-process/resident rather than through repeated CLI relay processes.Local workaround tested
As a temporary production hotfix, I changed the active project-scoped
@openclaw/codex/dist/harness.jsfrom:to:
Then restarted
openclaw-gateway.service.After the workaround:
durationMs=31ms,eventLoop.degraded=false,delayP99Ms≈21msbashtool calls with 0 failuresopenclaw-hooksprocesses appearednativeHook.invoke, and 0 hook relay entriesThis workaround is intentionally blunt and likely disables useful Codex native hook functionality. It is not a proper upstream fix.
Suggested fix direction
PreToolUsewhenshouldRelayEvent("pre_tool_use")is false and no real before-tool-call/trusted policy work exists.openclaw hooks relayfor every tool call.codex.nativeHookRelay.enabled/codex.nativeHookRelay.events/codex.nativeHookRelay.preToolUse.Related issues I found
These look related but not the exact same failure mode:
openclaw hooks relaymentioned