|
| 1 | +# Spec 1:1 Cutover Plan |
| 2 | + |
| 3 | +Goal: Align the orchestration model to `SPEC.md` 1:1 and remove legacy persistence/application cruft. |
| 4 | + |
| 5 | +Execution mode for this plan: |
| 6 | +- Hard cutover only. Existing DB and migration history are disposable. |
| 7 | +- Intermediate steps are allowed to break runtime, tests, typecheck, and lint. |
| 8 | +- We optimize for small, reviewable work units, not continuous app operability. |
| 9 | +- Only the final gate requires everything to run cleanly. |
| 10 | + |
| 11 | +## 1. Freeze SPEC contract as source of truth |
| 12 | +Work units: |
| 13 | +- Create `.plans/spec-contract-matrix.md` with one row per requirement in `SPEC.md` sections `7.1`-`7.4`. |
| 14 | +- Add exact SQL-level requirements per row: table, column, type, nullability, PK/unique, index, and invariants. |
| 15 | +- Add app-level requirements per row: writer path, reader path, and owning module. |
| 16 | +- Mark each row with status labels: `required`, `implemented`, `to-replace`, `delete`. |
| 17 | +- Identify any ambiguous spec lines and record a concrete interpretation in the matrix. |
| 18 | + |
| 19 | +Deliverables: |
| 20 | +- Complete matrix file with no unclassified rows. |
| 21 | +- Single source checklist used by all later steps. |
| 22 | + |
| 23 | +Breakage allowed: |
| 24 | +- No code changes required yet. |
| 25 | + |
| 26 | +Exit criteria: |
| 27 | +- Every requirement in `7.1`-`7.4` has exactly one matrix row. |
| 28 | + |
| 29 | +## 2. Hard cutover migrations (replace current migration set) |
| 30 | +Work units: |
| 31 | +- Delete the current legacy migration files and rewrite migration loader ordering. |
| 32 | +- Create `001_orchestration_events.ts` with full envelope columns and required event indexes. |
| 33 | +- Create `002_orchestration_command_receipts.ts` with PK + lookup indexes. |
| 34 | +- Create `003_checkpoint_diff_blobs.ts` with uniqueness on `(thread_id, from_turn_count, to_turn_count)`. |
| 35 | +- Create `004_provider_session_runtime.ts` with PK and runtime lookup indexes. |
| 36 | +- Create `005_projections.ts` with all projection tables: |
| 37 | + - `projection_projects` |
| 38 | + - `projection_threads` |
| 39 | + - `projection_thread_messages` |
| 40 | + - `projection_thread_activities` |
| 41 | + - `projection_thread_sessions` |
| 42 | + - `projection_thread_turns` |
| 43 | + - `projection_checkpoints` |
| 44 | + - `projection_pending_approvals` |
| 45 | + - `projection_state` |
| 46 | +- Add all required indexes/constraints in `005_projections.ts`. |
| 47 | +- Ensure old tables (`projects`, `provider_checkpoints`, `provider_sessions`) are not recreated. |
| 48 | + |
| 49 | +Deliverables: |
| 50 | +- New 5-file migration chain. |
| 51 | +- Updated migration loader references only new migrations. |
| 52 | + |
| 53 | +Breakage allowed: |
| 54 | +- Repositories/services can be temporarily broken due to removed old tables. |
| 55 | + |
| 56 | +Exit criteria: |
| 57 | +- Fresh DB initializes with only canonical tables plus migration bookkeeping. |
| 58 | + |
| 59 | +## 3. Align persistence row/request schemas to DB 1:1 |
| 60 | +Work units: |
| 61 | +- Define row schemas for each canonical table (contracts or persistence layer module). |
| 62 | +- Define request schemas for every insert/update/query operation touching canonical tables. |
| 63 | +- Remove or deprecate row/request schemas tied to deleted legacy tables. |
| 64 | +- Normalize enum and null semantics to match contracts exactly. |
| 65 | +- Ensure SQL aliases map 1:1 to schema field names (no implicit shape transforms). |
| 66 | + |
| 67 | +Deliverables: |
| 68 | +- Canonical row/request schemas committed. |
| 69 | +- Zero references to legacy row schemas in active code paths. |
| 70 | + |
| 71 | +Breakage allowed: |
| 72 | +- Runtime can still fail while query layers are being rewired. |
| 73 | + |
| 74 | +Exit criteria: |
| 75 | +- Every canonical table used in code has a typed row schema and typed request schema. |
| 76 | + |
| 77 | +## 4. Rewrite event store for full persisted envelope |
| 78 | +Work units: |
| 79 | +- Refactor append path to write full envelope fields: |
| 80 | + - `event_id`, `aggregate_kind`, `stream_id`, `stream_version`, `event_type`, `occurred_at`, `command_id`, `causation_event_id`, `correlation_id`, `actor_kind`, `payload_json`, `metadata_json` |
| 81 | +- Implement stream version assignment/checking per aggregate stream. |
| 82 | +- Refactor read/replay path to decode payload and metadata from JSON and return `OrchestrationEvent` consistently. |
| 83 | +- Remove assumptions from old minimal schema (`aggregate_id`, missing metadata/actor). |
| 84 | +- Add explicit SQL ordering guarantees for replay (`ORDER BY sequence ASC`). |
| 85 | + |
| 86 | +Deliverables: |
| 87 | +- Event store append/replay fully aligned with canonical envelope. |
| 88 | + |
| 89 | +Breakage allowed: |
| 90 | +- Command dispatch flow can be partially broken until receipts/projectors are updated. |
| 91 | + |
| 92 | +Exit criteria: |
| 93 | +- Event store no longer depends on legacy event table shape. |
| 94 | + |
| 95 | +## 5. Add command receipt idempotency |
| 96 | +Work units: |
| 97 | +- Introduce persistence access layer for `orchestration_command_receipts`. |
| 98 | +- In command dispatch flow, check existing receipt by `commandId` before append. |
| 99 | +- On first execution, persist accepted receipt with `resultSequence`. |
| 100 | +- On domain rejection, persist rejected receipt with error payload. |
| 101 | +- On duplicate command, return prior result from receipt without re-appending event. |
| 102 | +- Ensure receipt write and event append ordering is deterministic. |
| 103 | + |
| 104 | +Deliverables: |
| 105 | +- Dispatch path with idempotency behavior wired through receipts. |
| 106 | + |
| 107 | +Breakage allowed: |
| 108 | +- Snapshot/read model may still be inconsistent until projectors are fully wired. |
| 109 | + |
| 110 | +Exit criteria: |
| 111 | +- Duplicate command IDs no longer create duplicate events. |
| 112 | + |
| 113 | +## 6. Build DB-backed projection pipeline |
| 114 | +Work units: |
| 115 | +- Create projector runner that consumes events and applies table-specific projections. |
| 116 | +- Implement projector handlers for each projection table. |
| 117 | +- For each handler, update target row(s) and `projection_state.last_applied_sequence` in the same transaction. |
| 118 | +- Define projector names used in `projection_state` and make them stable constants. |
| 119 | +- Add replay bootstrap from event store to bring projections up to latest sequence on startup. |
| 120 | +- Add safe resume logic from projector `last_applied_sequence`. |
| 121 | + |
| 122 | +Deliverables: |
| 123 | +- Persistent projector pipeline writing all `projection_*` tables. |
| 124 | + |
| 125 | +Breakage allowed: |
| 126 | +- Web/API layer may still read old in-memory model until step 7. |
| 127 | + |
| 128 | +Exit criteria: |
| 129 | +- Events drive projection rows in DB; projection state advances transactionally. |
| 130 | + |
| 131 | +## 7. Move RPC reads to projections and diff blobs |
| 132 | +Work units: |
| 133 | +- Implement snapshot query service reading only projection tables. |
| 134 | +- Build thread hydration from projection rows: messages, activities, checkpoints, session. |
| 135 | +- Compute `snapshotSequence` as the minimum required projector sequence from `projection_state`. |
| 136 | +- Implement `getTurnDiff` query backed by `checkpoint_diff_blobs` only. |
| 137 | +- Remove or bypass in-memory snapshot construction for RPC responses. |
| 138 | +- Validate replay handoff contract: snapshot sequence -> replay from `fromSequenceExclusive`. |
| 139 | + |
| 140 | +Deliverables: |
| 141 | +- `orchestration.getSnapshot` and `orchestration.getTurnDiff` served from DB projections/blob store. |
| 142 | + |
| 143 | +Breakage allowed: |
| 144 | +- Provider runtime persistence may still be partially legacy until step 8. |
| 145 | + |
| 146 | +Exit criteria: |
| 147 | +- No orchestration read RPC depends on legacy tables or in-memory-only state. |
| 148 | + |
| 149 | +## 8. Migrate provider runtime persistence to canonical table |
| 150 | +Work units: |
| 151 | +- Create repository/service for `provider_session_runtime`. |
| 152 | +- Update adapter/session manager to persist runtime/resume cursor in new table. |
| 153 | +- Ensure domain-visible session state still flows through orchestration events to `projection_thread_sessions`. |
| 154 | +- Remove writes to legacy provider session tables. |
| 155 | +- Verify restart/resume path reads runtime state from canonical table only. |
| 156 | + |
| 157 | +Deliverables: |
| 158 | +- Provider runtime state entirely backed by `provider_session_runtime`. |
| 159 | + |
| 160 | +Breakage allowed: |
| 161 | +- Some legacy interfaces may still exist but should be disconnected. |
| 162 | + |
| 163 | +Exit criteria: |
| 164 | +- Runtime restore no longer reads/writes legacy provider session persistence. |
| 165 | + |
| 166 | +## 9. Remove old cruft aggressively |
| 167 | +Work units: |
| 168 | +- Delete legacy repositories/services that map to removed tables. |
| 169 | +- Remove dead migration imports and obsolete persistence service interfaces. |
| 170 | +- Remove compatibility code paths that translate legacy row shapes. |
| 171 | +- Remove unused contracts/types linked to deprecated persistence model. |
| 172 | +- Update internal docs/comments to reference canonical projection/event model only. |
| 173 | + |
| 174 | +Deliverables: |
| 175 | +- Legacy persistence and translation layers removed from active codebase. |
| 176 | + |
| 177 | +Breakage allowed: |
| 178 | +- Temporary compile failures acceptable while deletion/refactor is in progress. |
| 179 | + |
| 180 | +Exit criteria: |
| 181 | +- No production code path references deleted legacy tables/services. |
| 182 | + |
| 183 | +## 10. Final verification gate (first point where green is required) |
| 184 | +Work units: |
| 185 | +- Add migration tests that assert canonical tables, columns, constraints, and indexes. |
| 186 | +- Add event store tests for envelope persistence, metadata, actor kind, and replay. |
| 187 | +- Add receipt idempotency tests for accept/reject/duplicate paths. |
| 188 | +- Add projector tests for transactional row updates + `projection_state` updates. |
| 189 | +- Add snapshot tests verifying projection-sourced output and `snapshotSequence` semantics. |
| 190 | +- Add turn diff tests verifying `checkpoint_diff_blobs` source of truth. |
| 191 | +- Add provider runtime tests for persist + restart + resume behavior. |
| 192 | +- Run project lint/typecheck/tests and fix failures. |
| 193 | + |
| 194 | +Deliverables: |
| 195 | +- Green checks with canonical schema + persistence model in place. |
| 196 | + |
| 197 | +Breakage allowed: |
| 198 | +- None at end of step. |
| 199 | + |
| 200 | +Exit criteria: |
| 201 | +- SPEC `7.1`-`7.4` requirements satisfied and validated by tests. |
0 commit comments