Skip to content

Commit ddcd310

Browse files
t3dotggclaude
andauthored
chore: docs, dev scripts and CI catch up with orchestration V2 (#15041)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
1 parent c5a929e commit ddcd310

29 files changed

Lines changed: 305 additions & 417 deletions

‎.agents/skills/test-t3-app/references/sqlite-fixtures.md‎

Lines changed: 7 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ Load this reference only when inspecting or seeding local T3 state directly.
44

55
## Select the correct database
66

7-
When `--base-dir` or `--home-dir` is explicit, runtime state lives under `<base-dir>/userdata` and the database path is `<base-dir>/userdata/state.sqlite`. The `<base-dir>/dev` state directory is only the fallback for an implicit development home, preventing an ordinary `vp run dev` from touching production state.
7+
When `--base-dir` or `--home-dir` is explicit, runtime state lives under `<base-dir>/userdata` and the database path is `<base-dir>/userdata/statev2.sqlite`. The `<base-dir>/dev` state directory is only the fallback for an implicit development home, preventing an ordinary `vp run dev` from touching production state.
8+
9+
The server copies the V1 `state.sqlite` into `statev2.sqlite` only when `statev2.sqlite` is missing. After the first start, edits to `state.sqlite` change nothing.
810

911
Start the target runtime once before seeding so all migrations have run. Use an isolated base directory. Stop the server before writes to avoid racing application state or an active projection.
1012

@@ -23,7 +25,7 @@ Inspect current columns before writing a fixture:
2325
```bash
2426
node apps/server/scripts/t3-sqlite-state.ts query \
2527
--base-dir <base-dir> \
26-
--sql "PRAGMA table_info(projection_threads)"
28+
--sql "PRAGMA table_info(orchestration_v2_projection_threads)"
2729
```
2830

2931
Apply a SQL fixture from a file:
@@ -38,21 +40,10 @@ Use one statement per invocation for both `query` and `exec`; the helper wraps w
3840

3941
## Seed projection data carefully
4042

41-
The web UI primarily reads these projection tables:
42-
43-
- `projection_projects`
44-
- `projection_threads`
45-
- `projection_thread_messages`
46-
- `projection_thread_activities`
47-
- `projection_thread_sessions`
48-
- `projection_turns`
49-
- `projection_pending_approvals`
50-
- `projection_thread_proposed_plans`
51-
52-
Inspect `PRAGMA table_info(<table>)` and the current migrations under `apps/server/src/persistence/Migrations/` before constructing inserts. Keep identifiers unique, timestamps as ISO strings, JSON columns valid, and related project/thread/turn IDs consistent.
43+
Clients read projects from `projection_projects` and everything else from the `orchestration_v2_projection_*` tables: threads, runs, messages, turn items, runtime requests, plans, and provider sessions. The older `projection_thread*` tables hold V1 history, which the server imports once per thread at startup; later edits there do not reach the UI.
5344

54-
For a substantial current example, inspect `seedDatabase` in `scripts/mobile-showcase-environment.ts`. Adapt its column set to the target database instead of assuming copied SQL remains current.
45+
Most V2 rows carry a `payload_json` that must decode against the schemas in `packages/contracts/src/orchestrationV2.ts`. The safest start is a row the app wrote itself: create a thread through the UI, copy its rows, and edit them. Keep identifiers unique, timestamps as ISO strings, and related project, thread, and run IDs consistent.
5546

56-
Direct projection writes are appropriate for ephemeral visual states, edge-case counts, long titles, activity lists, and similar UI fixtures. They do not create a coherent orchestration event history. Do not modify `orchestration_events` unless the test specifically exercises projector internals, and do not use direct projection writes to claim backend business behavior works.
47+
Direct projection writes are appropriate for ephemeral visual states, edge-case counts, long titles, long timelines, and similar UI fixtures. They do not create a coherent event history. Leave the event log (`orchestration_events`) unchanged, and do not use direct projection writes to claim backend business behavior works.
5748

5849
Use the app's commands or APIs for behavior tests. Use `node apps/server/src/bin.ts auth ...` for auth state rather than editing `auth_pairing_links` or `auth_sessions`.

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -487,6 +487,7 @@ jobs:
487487
test,
488488
test_web,
489489
test_server,
490+
transfer-report,
490491
rust,
491492
mobile_native_changes,
492493
mobile_native_static_analysis,

‎AGENTS.md‎

Lines changed: 6 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -88,17 +88,8 @@ The most common defect in this repo is a change that works on the path you teste
8888

8989
An empty database is a bad test. Seed your worktree's `.t3` with a copy of real data instead of pointing at live state:
9090

91-
- Copy from `~/.t3/userdata` (the developer's real data, the most realistic test set) or `~/.t3/dev`. Worktree state lives at `<worktree>/.t3/userdata`.
92-
- Snapshot the database with `VACUUM INTO`, which is safe even while a server has the source open and yields one consistent file:
93-
94-
```bash
95-
mkdir -p .t3/userdata
96-
rm -f .t3/userdata/state.sqlite* # VACUUM INTO refuses to overwrite
97-
bun -e "new (require('bun:sqlite').Database)(process.env.HOME + '/.t3/userdata/state.sqlite', { readonly: true }).run(\"VACUUM INTO '.t3/userdata/state.sqlite'\")"
98-
```
99-
100-
A plain `cp` is only safe when no server has the source open, and must bring the `-wal` and `-shm` siblings along. A live file copy is a corrupt copy.
101-
91+
- Run `vp run migrate-dev-db` with your dev server stopped. It rebuilds `<worktree>/.t3/userdata/statev2.sqlite` from a read-only snapshot of `~/.t3/userdata/statev2.sqlite`, the developer's real data. It keeps recent projects and their stopped threads, and drops scheduled tasks, pending work, and auth sessions, so your dev server never runs the developer's agents. Raise `--projects` and `--threads-per-project` for more data.
92+
- Refresh `statev2.sqlite`, not `state.sqlite`. The server copies the V1 `state.sqlite` only when `statev2.sqlite` is missing.
10293
- Bring `secrets` and `settings.json` only if the flow under test needs them.
10394
- Copy in, never symlink. Data flows one way: into your sandbox, never back out.
10495

@@ -108,7 +99,7 @@ An empty database is a bad test. Seed your worktree's `.t3` with a copy of real
10899
- Test meaningful logic or observable behavior. Do not render components to static markup to assert props or attributes, or add tests that merely assert callback wiring or mirror the implementation.
109100
- **Do not run repo-wide checks.** No `vp check`, no `vp run -r test`, no `vp run -r typecheck` unless I ask. CI owns the full suite.
110101
- Backend behavior changes ship with focused tests for that behavior.
111-
- The server is event-sourced and its async flows emit typed receipts. Wait on receipts and worker drains, never on sleeps or polling. A test that needs a timeout to pass is wrong.
102+
- The server is event-sourced, and side effects run after the command commits. In tests, drain the effect worker (`OrchestrationEffectWorkerV2.drain`) or await the specific persisted event or `Deferred` that marks the milestone. Never wait on sleeps or polling. A test that needs a timeout to pass is wrong.
112103
- Upon request, user-visible frontend changes should get one integrated pass in a real client: `test-t3-app` for web, `test-t3-mobile` for mobile. The primary agent does this once after integrating. Subagents do not launch their own dev servers. Ask permission before doing computer use or spinning up browsers.
113104

114105
For authorized mobile verification, a missing or outdated native client is a build step, not a blocker. Run `node scripts/mobile-native-client.ts ensure <ios|android> <device-id>` on the simulator host before starting Metro. It checks the local Expo fingerprint and builds/installs when needed. See `test-t3-mobile` for the full workflow.
@@ -120,7 +111,7 @@ For authorized mobile verification, a missing or outdated native client is a bui
120111
- Body: the problem in a sentence or two, then how you fixed it. End with the model and harness that did the work.
121112
- UI changes need before/after images. Motion or timing needs a short video.
122113
- Upload PR evidence to GitHub. Never commit PR-only screenshots or assets such as `.github/pr-assets/`.
123-
- One concern per PR. If the description says "also", split it.
114+
- One request is one PR. Split it only when the maintainer asks. Outside contributions follow the stricter [one problem per PR](CONTRIBUTING.md#one-problem) rule.
124115
- When babysitting: poll checks and comments newer than the last push, verify each bot finding against the source, fix real ones, dismiss false positives with a written reason. Stay quiet when nothing is new. Stop when the bots are green on the latest commit.
125116

126117
## Documentation
@@ -143,9 +134,9 @@ Most code changes do not need an internal documentation change. Agents can read
143134

144135
## How it works
145136

146-
Clients send typed WebSocket requests. The server turns them into _commands_, a pure _decider_ turns commands into persisted _events_, and a _projector_ derives the read model the UI renders. Provider CLIs run as subprocesses; per-provider _adapters_ translate their native protocols into orchestration events. Side effects run in queue-backed _reactors_ that emit _receipts_ when milestones land. Each turn ends with a _checkpoint_, a hidden git ref, so the app can diff and restore.
137+
Clients send typed WebSocket requests. The server turns them into _commands_. The _orchestrator_ (`apps/server/src/orchestration-v2/Orchestrator.ts`) serializes commands and decides _events_ without doing any I/O. The _event sink_ commits those events, the _projections_ the UI reads, the _command receipt_, and _outbox_ effects in one transaction. The _effect worker_ then runs the effects, such as starting a provider turn or capturing a checkpoint, and feeds results back as commands. Provider CLIs run as subprocesses; per-provider _adapters_ translate their native protocols into orchestration events. Each turn ends with a _checkpoint_, a hidden git ref, so the app can diff and restore.
147138

148-
Full glossary with file links: `docs/internals/glossary.md`
139+
Architecture and its constraints: `docs/internals/overview.md`. Glossary: `docs/internals/glossary.md`
149140

150141
## Where code lives
151142

‎CONTRIBUTING.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ Explain their relationship when it is not obvious. An adjacent cleanup, refactor
7070
its own PR unless it is necessary to solve the same problem. A large diff alone does not establish that
7171
the PR contains unrelated work.
7272

73+
This rule is for outside contributions. Maintainers, the logins in
74+
[.github/TRIAGE_EXEMPTIONS.td](.github/TRIAGE_EXEMPTIONS.td), may batch related fixes in one PR.
75+
7376
Follow the [documentation rules](AGENTS.md#documentation). Keep internal docs for decisions and
7477
hard-to-discover constraints. Update user guides when how to use a feature changes; skip descriptions
7578
of obvious controls and cosmetic changes.

‎apps/desktop/src/backend/DesktopBackendConfiguration.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -644,7 +644,7 @@ const resolveWslStartConfig = Effect.fn("desktop.backendConfiguration.resolveWsl
644644
// The packaged sidecar is a Windows executable and cannot run inside the
645645
// Linux WSL backend. Keep the field absent instead of passing an unusable
646646
// `/mnt/.../*.exe` path; WSL resource telemetry is reported unavailable.
647-
// See docs/architecture/resource-telemetry.md.
647+
// See docs/internals/resource-telemetry.md.
648648
...buildObservabilityFragment(input.observabilitySettings),
649649
};
650650

‎apps/server/scripts/migrate-dev-db.test.ts‎

Lines changed: 66 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -14,48 +14,78 @@ const withDatabase = <A, E>(
1414
effect: Effect.Effect<A, E, SqlClient.SqlClient>,
1515
) => effect.pipe(Effect.provide(NodeSqliteClient.layer({ filename: databasePath })));
1616

17-
/** A migrated source db with one thread per lifecycle state. Only
18-
* `stopped-thread` qualifies for the clone. */
17+
/** A migrated source db with one V2 thread per lifecycle state. Only
18+
* `stopped-thread` and its fork qualify for the clone. */
1919
const createFixtureSource = Effect.fn("createMigrateDevDbFixtureSource")(function* (
2020
baseDir: string,
2121
) {
2222
const fs = yield* FileSystem.FileSystem;
2323
const path = yield* Path.Path;
2424
const stateDir = path.join(baseDir, "userdata");
25-
const databasePath = path.join(stateDir, "state.sqlite");
25+
const databasePath = path.join(stateDir, "statev2.sqlite");
2626
yield* fs.makeDirectory(stateDir, { recursive: true });
2727
yield* withDatabase(
2828
databasePath,
2929
Effect.gen(function* () {
3030
const sql = yield* SqlClient.SqlClient;
3131
yield* runMigrations();
32-
// The real shared db carries this column from a branch build without a
33-
// matching migration; reproduce that drift so the filter is exercised.
34-
yield* sql`ALTER TABLE projection_threads ADD COLUMN monitor_json TEXT`;
3532

3633
yield* sql`INSERT INTO projection_projects
3734
(project_id, title, workspace_root, scripts_json, created_at, updated_at, deleted_at)
3835
VALUES
3936
('project-kept', 'Kept', '/tmp/kept', '[]', '2026-08-01', '2026-08-01', NULL),
4037
('project-deleted', 'Deleted', '/tmp/deleted', '[]', '2026-08-01', '2026-08-02', '2026-08-02')`;
4138

39+
const forkPayload =
40+
'{"lineage":{"parentThreadId":"stopped-thread","relationshipToParent":"fork","rootThreadId":"stopped-thread"}}';
41+
const subagentPayload =
42+
'{"lineage":{"parentThreadId":"subagent-parent","relationshipToParent":"subagent","rootThreadId":"subagent-parent"},"forkedFrom":{"type":"node","nodeId":"node-1"}}';
43+
// Excluded threads are newer than the kept family, so only the filters
44+
// can keep them out of a one-family-per-project clone.
4245
const threads = [
43-
["stopped-thread", "project-kept", "stopped", null, null],
44-
["running-thread", "project-kept", "running", null, null],
45-
["settled-thread", "project-kept", "stopped", "2026-08-01", null],
46-
["monitored-thread", "project-kept", "stopped", null, '{"kind":"pr"}'],
47-
["deleted-project-thread", "project-deleted", "stopped", null, null],
46+
["stopped-thread", "project-kept", "completed", "{}", "2026-08-01"],
47+
["fork-thread", "project-kept", "completed", forkPayload, "2026-08-02"],
48+
["running-thread", "project-kept", "running", "{}", "2026-08-05"],
49+
["settled-thread", "project-kept", "completed", '{"settledAt":"2026-08-01"}', "2026-08-05"],
50+
[
51+
"limit-thread",
52+
"project-kept",
53+
"completed",
54+
'{"limitRecovery":{"autoResume":true}}',
55+
"2026-08-05",
56+
],
57+
// Its result never reached the parent, so startup would deliver it.
58+
["subagent-parent", "project-kept", "completed", "{}", "2026-08-05"],
59+
["subagent-child", "project-kept", "completed", subagentPayload, "2026-08-05"],
60+
["deleted-project-thread", "project-deleted", "completed", "{}", "2026-08-05"],
4861
] as const;
49-
for (const [threadId, projectId, status, settledAt, monitorJson] of threads) {
50-
yield* sql`INSERT INTO projection_threads
51-
(thread_id, project_id, title, created_at, updated_at, settled_at, monitor_json)
52-
VALUES (${threadId}, ${projectId}, ${threadId}, '2026-08-01', '2026-08-01', ${settledAt}, ${monitorJson})`;
53-
yield* sql`INSERT INTO projection_thread_sessions (thread_id, status, updated_at)
54-
VALUES (${threadId}, ${status}, '2026-08-01')`;
62+
for (const [threadId, projectId, runStatus, payload, updatedAt] of threads) {
63+
yield* sql`INSERT INTO orchestration_v2_projection_threads
64+
(thread_id, project_id, title, default_provider, runtime_mode, interaction_mode, created_at, updated_at, payload_json)
65+
VALUES (${threadId}, ${projectId}, ${threadId}, 'codex', 'full-access', 'default', '2026-08-01', ${updatedAt}, ${payload})`;
66+
yield* sql`INSERT INTO orchestration_v2_projection_runs
67+
(run_id, thread_id, ordinal, provider, status, requested_at, payload_json)
68+
VALUES (${`run-${threadId}`}, ${threadId}, 1, 'codex', ${runStatus}, '2026-08-01', '{}')`;
5569
yield* sql`INSERT INTO orchestration_events
5670
(event_id, aggregate_kind, stream_id, stream_version, event_type, occurred_at, actor_kind, payload_json, metadata_json)
5771
VALUES (${`event-${threadId}`}, 'thread', ${threadId}, 0, 'thread.created', '2026-08-01', 'user', '{}', '{}')`;
5872
}
73+
// A provider session shared by two threads names its latest writer.
74+
yield* sql`INSERT INTO orchestration_v2_projection_provider_sessions
75+
(provider_session_id, thread_id, provider, status, updated_at, payload_json)
76+
VALUES ('session-shared', 'running-thread', 'codex', 'stopped', '2026-08-01', '{}')`;
77+
yield* sql`INSERT INTO orchestration_v2_projection_provider_session_bindings
78+
(provider_session_id, thread_id)
79+
VALUES ('session-shared', 'running-thread'), ('session-shared', 'stopped-thread')`;
80+
yield* sql`INSERT INTO orchestration_v2_projection_context_transfers
81+
(context_transfer_id, source_thread_id, target_thread_id, type, status, updated_at, payload_json)
82+
VALUES ('transfer-1', 'settled-thread', 'stopped-thread', 'provider_handoff', 'completed', '2026-08-01', '{}')`;
83+
yield* sql`INSERT INTO scheduled_tasks
84+
(task_id, title, prompt, enabled, schedule_json, project_id, workspace_strategy_json,
85+
model_selection_json, runtime_mode, interaction_mode, created_by, creation_source,
86+
created_at, updated_at, last_run_status, run_count)
87+
VALUES ('task-1', 'Nightly', 'Run it', 1, '{}', 'project-kept', '{}', '{}',
88+
'full-access', 'default', 'user', 'user', '2026-08-01', '2026-08-01', 'never', 0)`;
5989
yield* sql`INSERT INTO auth_sessions (session_id, subject, scopes, method, issued_at, expires_at)
6090
VALUES ('session-1', 'user', '[]', 'pairing', '2026-08-01', '2027-08-01')`;
6191
}),
@@ -64,7 +94,7 @@ const createFixtureSource = Effect.fn("createMigrateDevDbFixtureSource")(functio
6494
});
6595

6696
it.layer(NodeServices.layer)("migrate-dev-db", (it) => {
67-
it.effect("keeps only stopped threads from live projects and clears auth state", () =>
97+
it.effect("keeps stopped thread families from live projects and clears pending work", () =>
6898
Effect.gen(function* () {
6999
const fs = yield* FileSystem.FileSystem;
70100
const path = yield* Path.Path;
@@ -73,7 +103,7 @@ it.layer(NodeServices.layer)("migrate-dev-db", (it) => {
73103
const source = yield* createFixtureSource(sourceDir);
74104

75105
const result = yield* runMigrateDevDb(
76-
{ baseDir: destDir, source, projects: 5, threadsPerProject: 10 },
106+
{ baseDir: destDir, source, projects: 5, threadsPerProject: 1 },
77107
{ sharedHome: sourceDir },
78108
);
79109

@@ -83,23 +113,32 @@ it.layer(NodeServices.layer)("migrate-dev-db", (it) => {
83113
Effect.gen(function* () {
84114
const sql = yield* SqlClient.SqlClient;
85115
const threads = yield* sql<{ thread_id: string }>`
86-
SELECT thread_id FROM projection_threads ORDER BY thread_id`;
116+
SELECT thread_id FROM orchestration_v2_projection_threads ORDER BY thread_id`;
87117
const events = yield* sql<{ stream_id: string }>`
88-
SELECT stream_id FROM orchestration_events`;
89-
const [auth] = yield* sql<{ count: number }>`
90-
SELECT COUNT(*) AS count FROM auth_sessions`;
91-
return { threads, events, authCount: auth?.count ?? 0 };
118+
SELECT stream_id FROM orchestration_events ORDER BY stream_id`;
119+
const sessions = yield* sql<{ provider_session_id: string }>`
120+
SELECT provider_session_id FROM orchestration_v2_projection_provider_sessions`;
121+
const [leftovers] = yield* sql<{ auth: number; tasks: number; transfers: number }>`
122+
SELECT
123+
(SELECT COUNT(*) FROM auth_sessions) AS auth,
124+
(SELECT COUNT(*) FROM scheduled_tasks) AS tasks,
125+
(SELECT COUNT(*) FROM orchestration_v2_projection_context_transfers) AS transfers`;
126+
return { threads, events, sessions, leftovers };
92127
}),
93128
);
94129
assert.deepStrictEqual(
95130
kept.threads.map((row) => row.thread_id),
96-
["stopped-thread"],
131+
["fork-thread", "stopped-thread"],
97132
);
98133
assert.deepStrictEqual(
99134
kept.events.map((row) => row.stream_id),
100-
["stopped-thread"],
135+
["fork-thread", "stopped-thread"],
136+
);
137+
assert.deepStrictEqual(
138+
kept.sessions.map((row) => row.provider_session_id),
139+
["session-shared"],
101140
);
102-
assert.equal(kept.authCount, 0);
141+
assert.deepStrictEqual(kept.leftovers, { auth: 0, tasks: 0, transfers: 0 });
103142
}),
104143
);
105144

0 commit comments

Comments
 (0)