Skip to content

Commit 4070464

Browse files
authored
Merge pull request #998 from lidge-jun/codex/260803-integration-switches
feat(codex): a write substrate that survives being interrupted
2 parents 1f07c00 + 9bf41ad commit 4070464

160 files changed

Lines changed: 35601 additions & 571 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 118 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -1,81 +1,130 @@
1-
# Codex and Claude Desktop toggles
1+
# Client integration switches: Codex, Claude Desktop, and the memory they need
22

3-
Split out of `260803_integrations_toggle_all` after its fourth audit
4-
(`../260803_integrations_toggle_all/007_audit_synthesis_r4.md`). That unit keeps
5-
Claude Code and Grok, whose toggles need no durable operation state. These two
6-
need one, and it is the shared dependency four audit rounds kept circling.
3+
Split out of `260803_integrations_toggle_all` after its fourth audit. That unit
4+
shipped Claude Code and Grok. This one was scoped around a durable
5+
operation-state engine — and a research cycle against the real code says that
6+
engine solves the wrong problem.
77

8-
## Why these two are together and separate
8+
> **Re-planned 260803 after four research passes.** The rollback engine
9+
> (`010_operation_state.md`, never written) is **dropped**. Evidence:
10+
> `001`-`004`. The replacement is smaller, and it also fixes a defect in the
11+
> toggle we shipped hours ago.
912
10-
| | Codex | Claude Desktop |
11-
|---|---|---|
12-
| Artifacts | `config.toml`, `opencodex.config.toml`, model catalog, resume history (SQLite) | `<id>.json`, `.bak`, `_meta.json`, four fields in our config |
13-
| Owner | us, mostly | **Claude Desktop** — we edit another app's registry |
14-
| Prior state recoverable? | from its own journal, when hashes still match | only from `_meta.json` bytes; `appliedId` was never recorded |
15-
| Removal code exists? | yes, `restoreNativeCodex` | **no** |
13+
## The requirement in the owner's words
1614

17-
Neither can be undone by re-running its enable path, which is what makes Claude
18-
Code and Grok cheap. Codex's enable is not the inverse of its disable — the
19-
journal fallback means a disable may strip fragments rather than restore bytes,
20-
and a later enable writes today's catalog, not yesterday's arrangement. Desktop
21-
cannot re-derive which profile the user had selected at all.
15+
> "스위치를 꺼도 프록시는 살아있어야 돼. 코덱스 말고 다른 것만 켜고 싶을 수도 있잖아."
2216
23-
So both need an operation record that outlives the request, and that record is
24-
the first phase.
17+
Turning a client off must leave the proxy running and serving every other
18+
client. That is two obligations: the mutation must not stop the proxy, and the
19+
OFF must survive a restart.
20+
21+
## What the research changed
22+
23+
| Prior claim | What the code says |
24+
|---|---|
25+
| Codex needs a durable operation-state engine | `ocx restore` already restores native Codex **without stopping the proxy** (`src/cli/help.ts:18`), and `ocx restore back` is the enable direction. Both exist. (`001`) |
26+
| Desktop removal is impossible to do safely | Anthropic documents that a selected config without a valid `inferenceProvider` launches **standard mode**. We aim at that instead of guessing. (`002`) |
27+
| The missing piece is crash recovery | The missing piece is **desired state**. Only Claude Code has one; Grok's shipped toggle is silently re-enabled by the next `ocx start`. (`003`) |
28+
29+
The fourth research doc (`004`) is an unrelated live defect found while looking:
30+
one out-of-enum modality value makes a client reject its entire config. It joins
31+
this unit as an independent work-phase (LOOP-UNIT-CHAIN-01).
2532

2633
## Read first
2734

28-
From the parent unit, all still authoritative:
35+
In this unit, all four written this cycle:
36+
37+
- `001_native_restore_thesis.md` — Codex restore, its asymmetries, the history lock
38+
- `002_desktop_standard_mode.md` — the official standard-mode contract, and why `default` is not the restore verb
39+
- `003_durable_desired_state.md` — desired vs observed, the Grok regression, the schema
40+
- `004_export_modality_poisoning.md` — the gjc/Pi enum defect
2941

30-
- `001_removal_path_inventory.md` — what each client's disable actually costs
31-
- `003`, `004`, `005`, `007` — the four audit syntheses; `007` is why this unit
32-
exists
33-
- `002_consequence_dialog_ux.md` — dialog direction; the Codex and Desktop copy
34-
lives there
42+
From the parent unit, still authoritative: `001_removal_path_inventory.md`
43+
(what each disable costs) and `002_consequence_dialog_ux.md` (dialog direction).
44+
Its `007_audit_synthesis_r4.md` is the reason this unit exists, but its central
45+
conclusion is now **superseded** by `001`-`003` here.
3546

3647
## Phases
3748

38-
| Phase | Doc | Deliverable |
39-
|---|---|---|
40-
| WP1 | `010_operation_state.md` | The durable operation record: a versioned discriminated journal entry, prepare/commit, restart reconciliation, a field-scoped config writer |
41-
| WP2 | `020_codex_toggle.md` | Codex disable/enable on top of it |
42-
| WP3 | `030_desktop_toggle.md` | Desktop removal + rollback, `appliedProfileId` schema work |
43-
44-
WP2 and WP3 are **parallel siblings**, not a sequence: Desktop does not depend on
45-
Codex (audit r4 #11). Both depend on WP1 and nothing else.
46-
47-
## What WP1 must deliver
48-
49-
Named by audit round 4, findings #1, #2, #5, #3:
50-
51-
1. **A versioned discriminated journal entry.** Today's `JournalEntry`
52-
(`src/integrations/journal.ts:34-51`) is file-shaped: one `configPath`, one
53-
`SnapshotRef`, one `resultFingerprint`. A routing description and three
54-
library members do not fit it. Needs `file-v1 | native-state-v1 |
55-
desktop-hybrid-v1` with validated serialized state, per-member fingerprints,
56-
and retention behavior.
57-
2. **Prepare/commit with restart reconciliation.** A crash between the mutation
58-
and the append leaves no undo state at all. Idempotent re-apply does not fix
59-
this — it helps once a state exists, and none was recorded. The prepared
60-
record must be durable before the mutation and resolved after.
61-
3. **A field-scoped config writer.** `saveConfigPreservingClaudeCode` persists
62-
the whole live object and its own docstring says a `providers` hand edit is
63-
clobbered (`src/config.ts:2132-2135`); worse, when disk and caller both
64-
changed `claudeCode`, the caller's stale subtree wins, so one toggle field can
65-
clobber a concurrent Desktop-profile edit. Desktop's four fields need a write
66-
that reloads from disk and touches only named paths.
67-
68-
## Carried-forward findings
69-
70-
Everything in `005` §Carried forward plus round 4's, in particular: Codex's
71-
pre-state must capture the effective injection mode and history policy, not just
72-
a routing kind (r4 #4); `injectCodexConfig` takes a concrete `catalogPath`, so
73-
"selector" needs a resolver (r4 #4); Desktop's ordering is dangling-pointer-safe
74-
but not transactionally crash-safe and must say so (r4 #6); auto-apply must be
75-
suppressed while an operation is prepared (r4 #6).
76-
77-
## Status
78-
79-
Not started. `020` and `030` carry their pre-split content and are **stale**
80-
against the operation-state design WP1 has not written yet; their next P
81-
re-verifies them against it before any build.
49+
**Re-sliced after audit round 2** (`006_audit_synthesis_r2.md`). The first map
50+
sliced along a schema — one ten-key `clientIntegrations` map — which forced every
51+
phase to touch every client's write path. Two audit rounds widened rather than
52+
narrowed. This map slices along **ownership boundaries**: one client family per
53+
phase, each independently auditable.
54+
55+
| Phase | Doc | Deliverable | Audit state |
56+
|---|---|---|---|
57+
| WP2 | `010_modality_boundary.md` | The client-dialect modality filter | **clean through two rounds** |
58+
| WP3 | `020_api_keys_row.md` | API keys out of the card grid into their own row | never blocking |
59+
| WP4 | `030_desired_state.md` → re-scoped | Desired state for **Codex only**, plus its gates, single-flight, ownership preflight and CLI semantics | re-scope pending |
60+
| WP5 | `040_codex_toggle.md` | The Codex switch on WP4's flag | rewrite pending |
61+
| WP6 | *(new doc)* | Grok's desired state, reusing the shape WP4 proved | not started |
62+
| WP7 | `050_desktop_toggle.md` | The Desktop switch via documented standard mode | deferred behind WP5 |
63+
64+
The two pending docs were renamed so the decade order matches the phase order:
65+
reading the unit lexicographically now gives the build order, which is the whole
66+
point of the numbering convention.
67+
68+
WP1 was the research cycle plus this roadmap, twice audited.
69+
70+
WP2 and WP3 depend on nothing here and on each other not at all — they ship
71+
first, alone, because each changes one thing at one boundary. That property is
72+
the only thing that survived both audit rounds intact.
73+
74+
Three deliberate exclusions, each an accepted audit finding rather than a
75+
convenience:
76+
77+
- **Claude Code's ingress gates are not touched at all.** Round 1 #1 established
78+
that `claudeCode.enabled` is the documented kill switch for `/v1/messages`;
79+
the honest conclusion is not to route it through a new helper in this unit.
80+
- **The six file clients get no desired-state flag here.** Round 2 #4 (existing
81+
explicit OFF choices are not migrated) and #5 (a mutating GET) both belong to a
82+
phase that does not exist yet, and inventing it under audit pressure is what
83+
produced round 2's new findings.
84+
- **Desktop is behind Codex, not beside it.** Round 2 #3 found no coherent rule
85+
when a foreign profile is selected. The goal explicitly permits an evidenced
86+
deferral; this is one.
87+
88+
## Scope boundary
89+
90+
IN: `src/clients/config-export.ts`, `src/types.ts`, `src/config.ts`,
91+
`src/codex/sync.ts`, `src/grok/sync.ts`, `src/cli/index.ts`,
92+
`src/cli/opencode.ts`, `src/claude/desktop-3p.ts`,
93+
`src/server/management/native-integration-routes.ts`,
94+
`src/server/management/agent-settings-routes.ts`, `src/server/management-api.ts`,
95+
`gui/src/pages/integrations/*`, `gui/src/styles-integrations.css`,
96+
`gui/src/i18n/*`, `tests/`, `gui/tests/`.
97+
98+
OUT: releases, publishing, deploys, tags; starring the repository; rewriting the
99+
six-client file machinery in `src/integrations/`; `docs-site` restructuring;
100+
recording the previous `appliedId` (deferred, `002` §Residual).
101+
102+
## Criteria
103+
104+
- C1 — gjc loads our emitted config with no schema error, proven from the real
105+
file; Pi's identical exposure is closed in the same change.
106+
- C2 — Codex stays disabled across a proxy restart, an `ocx ensure`, and a
107+
`POST /api/sync`.
108+
- C3 — an upgrading user with no `clientIntegrations` key sees no behavior change.
109+
- C4 — disabling Codex never stops the proxy and never closes `/v1/responses`.
110+
- C5 — Codex toggles both directions from the overview with the proxy running.
111+
- C6 — a Codex disable blocked by the held history DB is an explained refusal
112+
naming the cause, never a raw 500 and never a false green.
113+
- C7 — startup convergence never touches state owned by a service running from a
114+
different `OPENCODEX_HOME` (round 2 #2, blocking).
115+
- C8 — API keys render as a row above the grid, observed rendered.
116+
- C9 — typecheck, full test, gui lint, gui test, privacy scan all green.
117+
- C10 — Desktop ships with a proven restore path OR is deferred with recorded
118+
evidence. Both outcomes close this criterion.
119+
120+
## Risk register
121+
122+
| Risk | Mitigation |
123+
|---|---|
124+
| A gate silently unplugs a working client on upgrade | Absent key means ON, everywhere, with a test for the absent-config case (`003`) |
125+
| Gating a safety path | Explicit do-not-gate list: journal repair, ownership checks, owned teardown, shared transports (`003`) |
126+
| Desktop pointed at a missing file | Never delete `appliedId`, never leave it dangling, never pick an entry by the name `Default` — this machine's `Default` is already dangling (`002`) |
127+
| The modality fix erases valid internal metadata | Filter at the client-dialect boundary only; management and CLI keep carrying `audio` verbatim (`004`) |
128+
| A green suite hides the real failure | 91 tests pass today beside a config gjc refuses to load. Every criterion names a live artifact, not a unit test |
129+
| A repair introduces a worse defect than the one it fixes | Round 2 produced exactly this. One client family per phase, re-audited before the next starts — never a cross-client rewrite under audit pressure |
130+
| Convergence tears down a foreign home's state | Every native remover runs `assertNativeTeardownOwned` before removal, and reconciliation runs only after service-ownership resolution (round 2 #2) |
Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# The native-restore thesis, tested against the code
2+
3+
Research doc. No diffs here — the implementation designs live in the decade
4+
docs. This one records what the code actually does, because the previous
5+
unit's plan was built on a claim the code does not support.
6+
7+
## The claim under test
8+
9+
`000_plan.md` asserts that Codex and Claude Desktop both need a durable
10+
operation-state engine — a versioned discriminated journal entry, prepare/commit
11+
with restart reconciliation, and a field-scoped config writer — before either
12+
can get a toggle. The owner's counter-thesis: the proxy keeps RUNNING while a
13+
client is switched back to its native path, so each client only needs to be
14+
returned to a path that already exists, not replayed from a recorded snapshot.
15+
16+
For Codex the counter-thesis is correct, and the evidence is not subtle.
17+
18+
## Codex: the toggle pair already ships as a CLI
19+
20+
`ocx restore` is documented in `src/cli/help.ts:18-20` as:
21+
22+
> Restore native Codex config without stopping the proxy; `restore back`
23+
> re-points codex at the running proxy.
24+
25+
That is the toggle, both directions, with the proxy up. `src/cli/index.ts:770`
26+
calls `restoreNativeCodex()` with no lifecycle operation anywhere near it, and
27+
`src/cli/index.ts:756` implements the enable direction as `syncModelsToCodex(live.port)`
28+
against a proxy it first proves is live via `findLiveProxy()` (`:751`).
29+
30+
Stronger still: `POST /api/stop` (`src/server/management-api.ts:181`) calls
31+
`restoreNativeCodex()` FIRST and only then schedules the drain and exit. The
32+
restore therefore already executes while the listener is serving. Whether the
33+
proxy later stops is irrelevant to the restore itself.
34+
35+
The service-stop path does verify the listener is gone before restoring
36+
(`src/service.ts:2571`), but that check enforces the requested outcome "service
37+
stopped" so success is not claimed while a supervisor respawns the process. It
38+
is not a precondition inside `restoreNativeCodex()`.
39+
40+
**Conclusion:** no durable operation-state engine is required to make Codex
41+
switchable. The disable path exists, the enable path exists, and both are
42+
proxy-agnostic.
43+
44+
## What restore actually touches, and where it is not symmetric
45+
46+
Four state groups, when no external `model_provider` owns the config:
47+
48+
| Group | Restore behavior | Evidence |
49+
|---|---|---|
50+
| `config.toml` + `opencodex.config.toml` | Byte-exact from the journal when the injected hash still matches; otherwise strip owned fragments | `src/codex/journal.ts:109`, `src/codex/inject.ts:770` |
51+
| Injection journal | Deleted on a complete restore, retained on a partial one | `src/codex/journal.ts:133` |
52+
| Model catalog | Pristine backup + post-sync native additions, or drop slash-qualified routed rows keeping native ones | `src/codex/catalog/sync.ts:572-590` |
53+
| Resume history | May update `state_5.sqlite`, patch/append rollout JSONL, consume the backup manifest | `src/codex/history-provider.ts:413,656,691` |
54+
55+
When an external provider such as `custom` owns `model_provider`, restore removes
56+
only the stale journal and deliberately leaves everything else alone
57+
(`src/codex/inject.ts:765`). That is a pre-existing courtesy to a user who moved
58+
off us by hand, and the toggle must preserve it rather than "fixing" it.
59+
60+
Three asymmetries matter for the toggle's honesty:
61+
62+
1. **Enable is `syncModelsToCodex()`, not `injectCodexConfig()`.** Injection
63+
selects and writes a catalog path but does not build the routed rows;
64+
`src/codex/sync.ts:83-110` refreshes the catalog and then injects, and that
65+
is what `ocx restore back` uses. A toggle wired to bare injection would
66+
re-point Codex at a catalog that no longer lists the routed models.
67+
2. **A post-injection root model selection is destroyed, not restored.** If the
68+
user edited config after we injected, the hash no longer matches, the
69+
fallback strip runs, and a root `model = "provider/slug"` line is removed
70+
(`src/codex/inject.ts:315,700`). Re-enabling has no record of that selection.
71+
The dialog copy must not promise to put it back.
72+
3. **Resume history is reversible but not byte-identical.** Restore patches line
73+
one when safe and appends a `session_meta`
74+
(`src/codex/history-provider.ts:81,444`); re-enabling appends another
75+
provider change rather than deleting that history.
76+
77+
## The history lock, and why the current return shape is not enough
78+
79+
The write path mirrors Codex's five-second SQLite busy timeout and retries twice
80+
with a 500 ms delay (`src/codex/history-provider.ts:25,526`). A Codex app or IDE
81+
holding the WAL writer lock is exactly what makes it fail.
82+
83+
At the low-level boundary the failure is structured:
84+
85+
```ts
86+
return withHistoryRetry(...) ?? { rows: 0, files: 0, failed: true };
87+
```
88+
89+
Recoverable busy/lock/permission failures return `failed: true`; corruption and
90+
programming errors throw (`src/codex/history-provider.ts:511,536,577`).
91+
92+
But `restoreNativeCodex()` discards that structure. `history.failed` only edits
93+
the message string, and `success` stays `cfg.success`
94+
(`src/codex/inject.ts:787,794`). A config restore that succeeded while the
95+
history stayed locked returns roughly:
96+
97+
```
98+
{ success: true, message: "... history could NOT be restored ..." }
99+
```
100+
101+
So a GUI that trusts `success` reports a clean disable while routed threads stay
102+
tagged `opencodex` and remain invisible in the native app. And `failed: true`
103+
itself conflates lock contention with `EPERM`/`EACCES`.
104+
105+
**Design consequence for `020`:** the toggle route must not consume
106+
`restoreNativeCodex()`'s boolean. It needs the structured per-artifact outcome —
107+
config, catalog, history — with the history failure carrying a classified reason,
108+
so the card can render "disabled, but your routed threads are still hidden
109+
because the Codex app is holding the history database" instead of a green check
110+
or a raw 500. Parsing the message string is not acceptable.
111+
112+
## The real hazard is not the socket
113+
114+
The sibling explorer's central finding, and the one that reshapes this unit:
115+
**there is no persisted per-client desired state.** After a disable, the next
116+
proxy startup, `ocx ensure`, `ocx sync`, a provider/model mutation, or
117+
`POST /api/sync` can silently re-inject Codex (`src/cli/index.ts:318,365`,
118+
`src/server/management/config-routes.ts:261`).
119+
120+
That is the actual engineering work this unit needs — and it is a much smaller
121+
thing than the operation-state engine `000_plan.md` specified. A durable
122+
desired-state flag per client, defaulting ON so no existing setup dies on
123+
upgrade, consulted by every automatic apply path. It is a switch's memory, not a
124+
rollback engine.
125+
126+
`030` and the durable-state design pick this up; `010` is retired as specified
127+
because the crash-recovery machinery it described is not what the evidence asks
128+
for.
129+
130+
## Observable consequences of a Codex disable
131+
132+
What the consequence dialog must actually name:
133+
134+
- plain `codex` returns to its native provider path (`src/codex/inject.ts:688`)
135+
- routed `provider/model` catalog rows disappear; native rows survive
136+
(`src/codex/catalog/sync.ts:578,590`)
137+
- the generated `opencodex` profile is removed or replaced, and managed native
138+
subagent defaults are restored or stripped (`src/codex/inject.ts:703`)
139+
- previously routed threads are retagged to their native provider — or stay
140+
hidden if the history DB was locked (`src/codex/history-provider.ts:677`)
141+
- the proxy keeps serving Claude, Grok, the exported file clients, and direct API
142+
callers, because restore contains no lifecycle operation at all
143+
144+
That last line is the whole point of the feature.

0 commit comments

Comments
 (0)