Skip to content

[Bug]: migrate-dev-db silently empties every thread table when --source is a V1 state.sqlite #19018

Closed

Description

@renepf

Area: apps/server
Impact: Major degradation or frequent failure — the command exits 0 and reports success, so the data loss is only visible by counting rows afterwards.

I searched existing issues (migrate-dev-db, V1 state.sqlite, empty dev database) and found no duplicate. #15017 is a different defect in the same area.

Steps to reproduce

# 1. A V1 source: a database from before the V2 thread tables.
sqlite3 "file:$HOME/.t3/userdata/state.sqlite?mode=ro" \
  "select count(*) from projection_threads;"        # 105
sqlite3 "file:$HOME/.t3/userdata/state.sqlite?mode=ro" \
  "select count(*) from sqlite_master where name='orchestration_v2_projection_threads';"   # 0

# 2. From a linked worktree, with the dev server stopped:
vp run migrate-dev-db --source "$HOME/.t3/userdata/state.sqlite" \
  --projects 5 --threads-per-project 10

# 3. Count what arrived:
sqlite3 <worktree>/.t3/userdata/statev2.sqlite \
  "select count(*) from projection_threads;"        # 0

A synthetic source works too: build one with runMigrations({ toMigrationInclusive: 54 }) and insert one row into projection_threads before running the command.

Expected behavior

Either the V1 thread data survives into the dev database, or the command fails loudly. The script's own documentation promises the first: apps/server/scripts/migrate-dev-db.ts:276-277 says "A source from before the V2 thread tables has no visibility to filter on, so it is copied whole", and :544-545 says migrations run before the prune so "a source older than this checkout would otherwise crash the prune queries". Both read as deliberate V1 support.

Actual behavior

The command exits 0, prints Dev database ready: … (844 KB) and lists every kept project with 0 threads. Every thread table is empty.

Copying visible threads from /Users/.../.t3/userdata/state.sqlite (read-only)...
Running migrations on the snapshot...
INFO (#4): Migrations ran successfully {
  migrations: [ '55_OrchestrationV2', ..., '61_PullRequestReviewGuides' ] }
Pruning to 5 projects, 10 stopped threads each...
Compacting into .../.t3/userdata/statev2.sqlite...

Dev database ready: .../.t3/userdata/statev2.sqlite (844 KB)
  t3code: 0 threads
  ops: 0 threads
  processes: 0 threads
  zoho-api: 0 threads
  website: 0 threads
  10 orchestration events kept

Measured, source against result:

table V1 source result
projection_threads 105 0
projection_thread_messages 9672 0
projection_thread_activities 2657 0
projection_turns 83 0
projection_thread_pull_requests 9 0
projection_thread_sessions 6 0
orchestration_events where aggregate_kind='thread' 15615 0
orchestration_events where aggregate_kind='project' 13 10

Cause

The copy phase does the right thing: a source without orchestration_v2_projection_threads sets filtered = false (:290) and is copied whole. Migration 55_OrchestrationV2 then populates part of the V2 world — ApplicationEventSource.ts:82 writes orchestration_events from orchestration_v2_events (:63), :165 and :210 write project events from projection_projects (:139), :251 writes orchestration_command_receipts — which is why the result still carries 5 projects and 10 project events.

What no migration does is import V1 threads. That is a server boot step: serverRuntimeStartup.ts:507-509 runs legacyV1ThreadImporter.reconcileShells, which reads FROM projection_threads (orchestration-v2/legacy/LegacyV1ThreadImporter.ts:575, second site :666). migrate-dev-db never calls it, so after the migrate phase orchestration_v2_projection_threads is empty.

pruneSnapshot then builds thread_families, stopped_families and kept_threads from that one empty table (:358-399) and sweeps every table carrying thread_id (:402-408, :412-425), plus every thread event (:428-432), with WHERE thread_id NOT IN (SELECT thread_id FROM kept_threads). With a V1 source the keep set is necessarily empty, so the sweep deletes exactly the rows the copy phase went out of its way to bring along.

The comment at :402-404 names the trap without seeing it: "Every V2 table and every V1 table the lazy importer reads is keyed by thread_id, so one sweep covers both." The sweep covers both. The keep set is V2-only.

Why the test suite does not catch it

apps/server/scripts/migrate-dev-db.test.ts:185-202, it.effect("upgrades a source from before the V2 thread tables"), builds its source with runMigrations({ toMigrationInclusive: 54 }) and inserts no rows at all, then asserts only assert.include(result.executedMigrations, "55_OrchestrationV2"). It proves the migrate phase runs and nothing about what survives it. It is green because the fixture had nothing to lose. Adding a single projection_threads row to that fixture and asserting it survives would fail today.

Environment

  • macOS 26.3.2 (build 25D2150, Darwin 25.3.0), Apple Silicon
  • Node 24.13.1 (package.json engines ^24.13.1)
  • pingdotgg/t3code at 7b6d907574, run from a linked git worktree
  • Source: a real ~/.t3/userdata/state.sqlite, 53 MB, 105 V1 threads, no statev2.sqlite alongside it

Suggested fixes, for maintainers to judge

  1. A guard, cheapest and smallest. A new error class in the row at :50-150 that aborts when orchestration_v2_projection_threads is empty while the source carried projection_threads rows. It turns silent loss into a clear refusal. The check has to hang off the source, not the result, or it would also abort on a legitimately empty database.
  2. Make the V1 path real. Run the headless equivalent of the boot import inside the script — persistence/initializeV2Database.ts:20-24 plus orchestration-v2/legacy/LegacyV1ThreadImporter.ts:575 — between the migrate and prune phases. This creates a second entry point into the V1-to-V2 transition that you would have to maintain, so it is your architectural call, not something I would send as a PR unasked.

In either case the vacuous test above should gain a data row.

Activity

  1. juliusmarminge commented on Oct 21, 2026

    @juliusmarminge
    Member

    Reproduced on Linux/Node 24.21.0 at 70e5e384da. Created a migration-54 source with one project and one legacy thread, then ran vp run migrate-dev-db --source /tmp/triage-19018/source/state.sqlite --projects 5 --threads-per-project 10 from a linked worktree with no dev server running. The command exited 0 and printed Repro: 0 threads; SQLite counts were source 1, destination 0. No matching open duplicate found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions