Skip to content

Commit 8b78eef

Browse files
authored
Process-exclusive elastic pooling for injected StructuredOutput runs (#293)
* fix: process-exclusive elastic pooling for injected StructuredOutput runs (#292) Replace the per-connection FIFO that serialized every injected-tool schema run (introduced in #56 for OpenCode's process-global, name-keyed MCP registry) with space-based isolation: selection synchronously reserves a process with no other injected run, spawning past the configured pool size when all are reserved, and the reservation is released only after the owning session.release() completes. Pi, OpenCode, and custom injecting backends now overlap uniformly — parallel(N) schema runs execute N-wide — while the #56 leak stays impossible: no two injected runs ever share a process. Non-injected runs keep idle/grow-to-size/least-loaded multiplexing. Surplus processes stay warm for a 30s keep-alive, then reap back to the steady-state size; dispose()/forceKill() cover elastic and mid-reap connections. The #56 serialization regression tests are retargeted to overlap + distinct-process + distinct-capture assertions across the fake, Pi, and OpenCode backends, and the live e2e now expects three elastic subprocesses for injected backends. Implemented-by: Codex (gpt-5.6-sol, xhigh) * chore: dependency-gate maintenance — claude-agent-acp 0.62.0, codex-acp 1.6.12 Bump @agentclientprotocol/claude-agent-acp 0.61.0 -> 0.62.0, which wraps the current @anthropic-ai/claude-agent-sdk 0.3.219; the root pnpm override pinning 0.3.218 is now stale and removed so the adapter's own exact pin wins. Sync the codex-acp fork with upstream (models-availability e2e fix, test-only), fork auto-released as @automatalabs/codex-acp 1.6.12; bump the exact pin. Welded doc citations updated in docs/api.md, docs/design-notes.md, and the acp-auth-spec §3.3 heading per the docs-drift suite. * test(live-e2e): reroute opencode leg to deepseek-v4-flash With process-exclusive injected pooling (#292) the gate's three opencode schema runs fire concurrently, and kimi-k3 reliably drops one of the three under burst (observed twice at the pre-push gate; the pi leg's identical 3-wide OpenRouter fan-out on another model passes consistently, ruling out runner-side contention). Reroute the leg's default model per owner direction; AGENTPRISM_OPENCODE_E2E_MODEL still overrides. * fix: per-spawn OPENCODE_DB isolation for concurrent opencode processes Concurrent OpenCode instances in the same project share the cwd-keyed sqlite database (WAL) and interfere across sessions — upstream anomalyco/opencode#31307. Process-exclusive injected pooling (#292) made overlapping opencode processes routine, surfacing the interference as a deterministic mid-run "ACP connection closed" on one of three concurrent gate legs (full-suite 3/4 failure rate, model-independent: reproduced on kimi-k3 and deepseek-v4-flash; solo legs and pi's identical 3-wide fan-out always passed). spawnConfig() now assigns each spawned server its own OPENCODE_DB under the OS tmpdir; auth.json is separate and unaffected, and an explicitly exported OPENCODE_DB wins. Tradeoff documented in-line: cross-process session/load|resume falls back to the runner's fresh-session path. Full live 4-leg suite green with the isolation in place. * fix: full per-spawn XDG isolation for opencode servers, not just OPENCODE_DB Research into anomalyco/opencode's source and issue tracker showed the shared surface is wider than the sqlite db: concurrent instances also share one snapshot git index, the log, tool-output, and auth.json, and 1.18.5 still serializes all statements behind an untimed in-process semaphore, so db contention freezes a whole process (ACP stdout pump included). Isolating OPENCODE_DB alone is documented as insufficient (#33321). Each spawned server now gets fresh XDG data/state/cache trees with auth.json/mcp-auth.json seeded in and autoupdate disabled; XDG_CONFIG_HOME stays shared so user config and providers apply, and an explicit OPENCODE_DB passes through. Validated: 9/9 backend unit tests, hermetic suite green, full live 4-leg e2e green with the opencode leg overlapping three isolated processes.
1 parent ea9fee8 commit 8b78eef

20 files changed

Lines changed: 701 additions & 219 deletions

‎.changeset/acp-deps-july-24.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
"@automatalabs/acp-agents": patch
3+
---
4+
5+
Dependency-gate maintenance: bump @agentclientprotocol/claude-agent-acp to 0.62.0 (wraps the
6+
current @anthropic-ai/claude-agent-sdk 0.3.219, retiring the root pnpm override) and
7+
@automatalabs/codex-acp to 1.6.12 (fork resynced with upstream's models-availability e2e fix).
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
"@automatalabs/acp-agents": patch
3+
---
4+
5+
Isolate every spawned `opencode acp` process behind fresh per-spawn XDG data/state/cache trees
6+
with the user's credentials seeded in (and autoupdate disabled for the child). Concurrent
7+
OpenCode instances share the sqlite database, snapshot git index, log, and auth.json, and
8+
interfere across sessions (anomalyco/opencode#31307, #29395, #21215, #38366, #37059) — observed
9+
as mid-run "ACP connection closed" once process-exclusive injected pooling overlapped opencode
10+
processes. Isolating only OPENCODE_DB is insufficient (#33321). User config (XDG_CONFIG_HOME)
11+
stays shared; an explicitly exported OPENCODE_DB passes through. Cross-process session reattach
12+
for opencode now falls back to the runner's fresh-session path.

‎.changeset/swift-lions-pool.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"@automatalabs/acp-agents": patch
3+
---
4+
5+
Replace per-connection serialization for injected StructuredOutput runs with process-exclusive
6+
elastic pooling, including idle surplus reaping and full disposal coverage.

‎docs/api.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1085,7 +1085,7 @@ quarantines the pooled child from new work, and disposes it after existing sibli
10851085
This policy is identical for every built-in and custom ACP backend. A `child_cleanup_error` returned
10861086
by close remains observable through the runner's normal error path.
10871087

1088-
**Structured output channels.** Claude and Codex keep their agent-specific schema channels authoritative. Pi, OpenCode, and opted-in custom ACP backends use the client-hosted MCP path: when `RunOptions.schema` is set and initialize advertises HTTP MCP support, the runner appends a client-hosted HTTP MCP server to `session/new.mcpServers`. The injected server is named `structured_output` (or `structured_output_2`, etc. on name collision), runs on `127.0.0.1` with an unguessable token path, and exposes `StructuredOutput`; Pi shows the namespaced alias `mcp__structured_output__StructuredOutput`. Its input schema is the requested JSON Schema and a valid call captures the result. Injected-tool schema runs are serialized per pooled connection. The common prompt-embedded schema plus validated final-text ladder remains the fallback when capture is absent or invalid. User-provided `mcpServers` are preserved and are not part of the resume hash.
1088+
**Structured output channels.** Claude and Codex keep their agent-specific schema channels authoritative. Pi, OpenCode, and opted-in custom ACP backends use the client-hosted MCP path: when `RunOptions.schema` is set and initialize advertises HTTP MCP support, the runner appends a client-hosted HTTP MCP server to `session/new.mcpServers`. The injected server is named `structured_output` (or `structured_output_2`, etc. on name collision), runs on `127.0.0.1` with an unguessable token path, and exposes `StructuredOutput`; Pi shows the namespaced alias `mcp__structured_output__StructuredOutput`. Its input schema is the requested JSON Schema and a valid call captures the result. Each injected-tool schema run reserves a pooled process exclusively from other injected runs; when every process is reserved, the pool grows elastically past `size`, then keeps surplus idle processes warm briefly before shrinking back to that steady-state size. The reservation remains held through `session.release()`, so two injected runs never share one process. Non-injected runs retain ordinary idle/grow-to-size/least-loaded multiplexing and may co-locate with an injected run. The common prompt-embedded schema plus validated final-text ladder remains the fallback when capture is absent or invalid. User-provided `mcpServers` are preserved and are not part of the resume hash.
10891089

10901090
**Model specs**: after the engine's existing precedence resolves one effective string, the runner splits it on the first `/`. If the first segment, ASCII-case-insensitively, is `claude`, `codex`, `opencode`, `pi`, or a registered custom backend name, that harness is selected and exactly one segment is stripped; a custom registration wins on collision. A registered harness name by itself is backend-only and issues no model `session/set_config_option`, preserving the harness default. Any other first segment sends the entire authored string unchanged to `AGENTPRISM_DEFAULT_BACKEND` (historical default `claude`), so `anthropic/…`, `openai/…`, bare `opus`, and bare `gpt-…` are not aliases. When an id remains it is the exact `configId:"model"` value: no case folding, normalization, catalog matching, bracket parsing, sibling effort/Fast option driving, retry, echo verification, or fallback. Brackets, dots, and provider-style prefixes are ordinary id characters. For pi, `pi/<provider>/<model-id>` therefore sends `<provider>/<model-id>` verbatim. Harness rejection follows the existing agent-error path; `onModelFallback` remains source-compatible but model resolution does not emit it. Live-catalog-verified examples: `claude/opus[1m]`, `codex/gpt-5.6-sol`, `opencode/zai/glm-5.2`; use backend-only `claude`, `codex`, `opencode`, or `pi` when the model is configured in the harness.
10911091

@@ -1170,8 +1170,8 @@ await runner.logout({ model: "codex" });
11701170

11711171
Installed adapter status from the bundled dists:
11721172

1173-
- `@agentclientprotocol/claude-agent-acp@0.61.0`: advertises `auth.logout`, implements `logout`, and implements `authenticate` for its gateway auth methods; terminal login methods are advertised only when the client advertises terminal auth support. As of 0.60.0 it advertises `providers` and implements `providers/list`, `providers/set`, and `providers/disable` for a single provider `providerId` `"main"` supporting `apiType` `anthropic`, `bedrock`, and `vertex`; `providers/set` rejects any other `providerId`/`apiType` with invalid-params, and `vertex` additionally requires `_meta.claudeCode.vertex.{projectId,region}` (recorded as durable routing config and replayed on every reconstructed `providers/set`, so pooled connections re-route correctly). `providers/disable` is idempotent.
1174-
- `@automatalabs/[email protected].11`: advertises `auth.logout`, implements `authenticate` (`api-key`, `chat-gpt`, and `gateway` when gateway support is advertised), and implements `logout`. As of 1.6.0 (upstream sync) it also advertises `providers` and implements `providers/list`, `providers/set`, and `providers/disable` for its single client-configurable custom gateway provider: `providerId` `"custom-gateway"`, `supported: ["openai"]`, `required: false`, `current` carrying only the non-secret `{ apiType, baseUrl }` (never headers) and `null` while unconfigured; `providers/set` rejects any other `providerId`/`apiType` with invalid-params and `providers/disable` is idempotent. Its separate reasoning-effort options remain agent-owned configuration; model-spec brackets are never interpreted by this client.
1173+
- `@agentclientprotocol/claude-agent-acp@0.62.0`: advertises `auth.logout`, implements `logout`, and implements `authenticate` for its gateway auth methods; terminal login methods are advertised only when the client advertises terminal auth support. As of 0.60.0 it advertises `providers` and implements `providers/list`, `providers/set`, and `providers/disable` for a single provider `providerId` `"main"` supporting `apiType` `anthropic`, `bedrock`, and `vertex`; `providers/set` rejects any other `providerId`/`apiType` with invalid-params, and `vertex` additionally requires `_meta.claudeCode.vertex.{projectId,region}` (recorded as durable routing config and replayed on every reconstructed `providers/set`, so pooled connections re-route correctly). `providers/disable` is idempotent.
1174+
- `@automatalabs/[email protected].12`: advertises `auth.logout`, implements `authenticate` (`api-key`, `chat-gpt`, and `gateway` when gateway support is advertised), and implements `logout`. As of 1.6.0 (upstream sync) it also advertises `providers` and implements `providers/list`, `providers/set`, and `providers/disable` for its single client-configurable custom gateway provider: `providerId` `"custom-gateway"`, `supported: ["openai"]`, `required: false`, `current` carrying only the non-secret `{ apiType, baseUrl }` (never headers) and `null` while unconfigured; `providers/set` rejects any other `providerId`/`apiType` with invalid-params and `providers/disable` is idempotent. Its separate reasoning-effort options remain agent-owned configuration; model-spec brackets are never interpreted by this client.
11751175
- Host-resolved OpenCode (`opencode-ai` 1.17.14 in the verified profile): advertises the `opencode-login` terminal-style method when the client advertises terminal auth, acknowledges `authenticate`, and relies on its provider credential store; it does not advertise logout. The credential-gated live suite verifies the installed executable because OpenCode is not bundled.
11761176
- `@automatalabs/pi-acp`: unconditionally advertises `anthropic-api-key`, `openai-api-key`, `gemini-api-key`, `xai-api-key`, `openrouter-api-key`, and `pi-stored-credentials`. The first five are `env_var` methods; the stored-credentials method is a bare `agent` method backed by `~/.pi/agent/auth.json`. Authentication is ambient/no-op, while a known model with missing credentials rejects with ACP `-32000`.
11771177

@@ -1242,7 +1242,7 @@ await chat.prompt("Revise section 3 — the user wants X.");
12421242

12431243
Set `agent(prompt, { keepSession: true })` in the script (or `RunOptions.keepSession` on direct `run()` calls) when you intend to re-open: it skips the release-time best-effort `session/close`, guaranteeing the agent-persisted session is untouched. Without it the record is still surfaced, and the four first-class agents keep closed sessions loadable — but `keepSession` is the explicit, agent-agnostic contract. Check `reopen.load`/`reopen.resume` before offering re-attach and optional `reopen.fork` before offering a fork in UI: an agent that persists nothing advertises none of them, and its sessions are reachable only while held open (`openSession`). The fork flag is optional so records written before this field existed remain valid.
12441244

1245-
Lifecycle methods are capability-gated after initialize. In particular, `forkSession()` requires `sessionCapabilities.fork`; when absent it throws a non-recoverable `WorkflowError` naming the backend and `session/fork` before any fork request is sent. The installed `@agentclientprotocol/claude-agent-acp@0.61.0` advertises `loadSession: true` plus `sessionCapabilities` for list/delete/resume/close/fork (fork verified live: the forked session carries the source conversation's context). `@automatalabs/[email protected].11` advertises `loadSession: true` plus list/delete/resume/close — no fork yet. OpenCode advertises load/list/resume/close/fork (also verified live). `@automatalabs/pi-acp` advertises load plus list/resume/close/fork and deliberately omits delete; unsupported lifecycle methods still fail through the same gate.
1245+
Lifecycle methods are capability-gated after initialize. In particular, `forkSession()` requires `sessionCapabilities.fork`; when absent it throws a non-recoverable `WorkflowError` naming the backend and `session/fork` before any fork request is sent. The installed `@agentclientprotocol/claude-agent-acp@0.62.0` advertises `loadSession: true` plus `sessionCapabilities` for list/delete/resume/close/fork (fork verified live: the forked session carries the source conversation's context). `@automatalabs/[email protected].12` advertises `loadSession: true` plus list/delete/resume/close — no fork yet. OpenCode advertises load/list/resume/close/fork (also verified live). `@automatalabs/pi-acp` advertises load plus list/resume/close/fork and deliberately omits delete; unsupported lifecycle methods still fail through the same gate.
12461246

12471247
### Capabilities
12481248

@@ -1276,7 +1276,7 @@ Two gates run before any prompt tokens are spent:
12761276
- The agent must advertise `agentCapabilities.mcpCapabilities.acp === true`; otherwise the ACP server config fails with non-recoverable `SCRIPT_VALIDATION_ERROR`.
12771277
- The runner must have a complete `clientHandlers.mcp`; declaring `{ type: "acp" }` without a handler is also a non-recoverable config error.
12781278

1279-
Installed backend status verified from the packaged dists: `@agentclientprotocol/claude-agent-acp@0.61.0` advertises `http`/`sse` MCP support but no `acp`, `@automatalabs/[email protected].11` advertises `mcpCapabilities: { acp: false, http: true, sse: false }` and rejects ACP MCP config internally, OpenCode advertises HTTP/SSE MCP support, and `@automatalabs/pi-acp` serves stdio, Streamable HTTP, and SSE while advertising `{ http:true, sse:true }`. Pi also consumes the stable MCP base protocol plus sampling, roots, and form/URL elicitation; client-hosted `acp` remains runner-owned.
1279+
Installed backend status verified from the packaged dists: `@agentclientprotocol/claude-agent-acp@0.62.0` advertises `http`/`sse` MCP support but no `acp`, `@automatalabs/[email protected].12` advertises `mcpCapabilities: { acp: false, http: true, sse: false }` and rejects ACP MCP config internally, OpenCode advertises HTTP/SSE MCP support, and `@automatalabs/pi-acp` serves stdio, Streamable HTTP, and SSE while advertising `{ http:true, sse:true }`. Pi also consumes the stable MCP base protocol plus sampling, roots, and form/URL elicitation; client-hosted `acp` remains runner-owned.
12801280

12811281
---
12821282

0 commit comments

Comments
 (0)