Skip to content

Codex harness app-server shared client is evicted across agents because cache key includes agentDir #79495

Description

@maxschachere

Summary

In v2026.5.7, the Codex harness app-server shared client appears to be a single global singleton, but its start-options cache key includes agentDir. In multi-agent OpenClaw setups, different agents use different agent dirs, so switching between agents can clear the existing shared app-server client instead of reusing a warm per-agent client.

This can cause repeated app-server startup/reconnect/logging overhead under normal multi-agent workloads, and is especially expensive on smaller ARM hosts such as Raspberry Pi.

Relevant code paths

  • extensions/codex/src/app-server/shared-client.ts
    • getSharedCodexAppServerClient(...) stores one global state.client, state.promise, and state.key.
    • When the computed key changes, it calls clearSharedCodexAppServerClient().
  • extensions/codex/src/app-server/config.ts
    • codexAppServerStartOptionsKey(...) includes agentDir in the key.

That means agents with different agentDir values, for example main, dev, ops, and analyst, can evict each other's app-server client.

Observed impact

On a Raspberry Pi Docker install with multiple OpenClaw agents and scheduled/background sessions:

  • Codex harness mode produced many *.jsonl.codex-app-server.json binding files across agents.
  • CPU usage stayed materially higher while Codex harness was active.
  • After switching the same agents back to the native/Pi runtime, no new Codex app-server binding files were created and CPU dropped sharply.
  • Example local measurements from the same host/workload:
    • Before switching away from Codex harness: avg CPU around 23.7%, peak around 78.1%.
    • After switching to native/Pi runtime and allowing the gateway to settle: avg CPU around 5.9%, peak around 25.1%.
  • Codex app-server SQLite logs also grew large across agent dirs, especially background agents.

This does not look like a Docker memory/swap issue. It looks like the current shared-client design does not scale well when multiple OpenClaw agents use the Codex app-server harness concurrently or interleaved.

Expected behavior

The Codex app-server client should be reused per compatible start-options key instead of using one global singleton that is cleared on every key mismatch.

Suggested fix

Replace the singleton shared client with a keyed pool/map, roughly:

const clients = new Map<string, SharedCodexAppServerClient>();

Then reuse the warm client for the matching codexAppServerStartOptionsKey, with idle TTL cleanup to avoid leaking clients.

I would not simply remove agentDir from the key, since it may be needed to preserve per-agent HOME/CODEX_HOME/auth/session isolation. A keyed pool seems safer: keep agentDir in the key, but stop different agent dirs from evicting each other.

Secondary related issue

Dynamic tool fingerprints also appear to be stored as full normalized JSON strings rather than compact hashes. On the same host, dynamic tool fingerprints averaged about 9.5 KB and reached about 37 KB per binding file. That is probably separate from the singleton eviction issue, but it adds extra filesystem/log churn and makes compatibility checks heavier than necessary.

A compact stable hash plus lightweight metadata would likely be enough for durable compatibility checks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions