Skip to content

Commit 7738d71

Browse files
committed
fix(archive): preserve recovery data and reject invalid dates
🤖 Co-authored by GPT-6 in Codex via T3 Code
1 parent 56d8a3d commit 7738d71

8 files changed

Lines changed: 146 additions & 66 deletions

File tree

‎BRANCH_DETAILS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Archived conversations use cold storage instead of retaining full hot projection
55
- `state.sqlite` keeps archived thread shells, pull-request relation rows needed to discover linked conversations, and command receipts required for retry idempotency. Conversation events, messages, activities, turns, checkpoints, plans, session/runtime rows, and content attachments are written as bounded gzip-compressed chunks in the separate `archive.sqlite` database, then removed from hot storage. Command receipts remain hot until permanent deletion so retrying an accepted archive command returns its original sequence. `projection_thread_pull_requests` remains hot with the shell, including its persisted PR and stack snapshots; archive does not rewrite those rows, and permanent deletion removes them. Pin state remains lightweight shell metadata and survives archive and unarchive.
66
- `apps/server/scripts/t3-sqlite-state.ts` is the supported isolated-database inspection path. It targets hot `state.sqlite` by default and accepts `--database archive` for cold manifest and chunk queries, with both paths derived from the shared server configuration. Archive and restore behavior should use application commands rather than direct archive-bundle writes.
77
- Content attachments are part of the archive bundle and return on unarchive. Attachment collection follows exact ids persisted in thread messages, while retry cleanup may reuse exact attachment chunk filenames already recorded in the cold bundle; normalized thread-name collisions cannot claim another thread's files. Provider diagnostic logs and terminal history logs are deliberately destructive on archive: they are deleted, never copied into the bundle, and never restored. Provider-log cleanup matches only the exact thread log and its numeric rotations so similarly prefixed thread ids are not affected.
8-
- Cold restore preserves binary SQL values, validates attachment entry names, and atomically replaces attachment files before marking SQL rows restored. It keeps compressed chunks authoritative on failure, pages chunk reads to bound memory, rejects unknown tables/chunk kinds, and intersects archived row columns with the current schema so older bundles remain recoverable after compatible migrations.
8+
- Cold restore preserves binary SQL values, rejects unsafe attachment entry names with a retryable failure, and atomically replaces attachment files before marking SQL rows restored. A malformed attachment leaves the cold bundle authoritative until repaired and retried. Restore pages chunk reads to bound memory, rejects unknown tables/chunk kinds, and intersects archived row columns with the current schema so older bundles remain recoverable after compatible migrations.
99
- Permanent thread deletion removes the shell, event stream, command receipts, every thread-owned projection/runtime/checkpoint row, attachments, terminal history, provider logs and rotations, and any cold-archive manifest/chunks. Exact attachment ownership metadata remains available until external attachment and provider-log cleanup succeeds, after which the SQL and cold-chunk rows are removed; cross-thread plan references are cleared rather than leaving dangling ids, and the durable cleanup-queue entry is retained until filesystem cleanup and free-page reclamation succeed so interrupted deletes retry safely. Startup recovery also rediscovers soft-deleted shells that have not reached the cleanup queue, covering commits made while the lifecycle reactor was offline.
1010
- Project removal in `apps/web/src/components/settings/ProjectSettingsPanel.tsx` applies the archived-history warning and command options from `apps/web/src/components/settings/ProjectSettingsPanel.logic.ts` independently to each selected project member. Members with known live threads use forced deletion; when a member has no live shell threads, it requests archived-thread deletion without force because archived shells are excluded from normal navigation state. The server removes archived cold bundles but still rejects any unseen live thread. Project removal emits the same per-thread deletion events for archived shells, so already-cold threads pass through the durable lifecycle worker and lose their hot shell, archive manifest, and compressed chunks before cleanup completes. `apps/web/src/components/settings/ProjectSettingsPanel.logic.test.ts` covers standalone and grouped warning text and the per-member thread partitioning that selects each member's command options; `apps/server/src/orchestration/decider.delete.test.ts` covers the archived-only live-thread precondition; and `apps/server/src/orchestration/Layers/ThreadDeletionReactor.test.ts` covers the forced-project decider, lifecycle reactor, and cold-storage boundary together.
1111
- Archive/delete filesystem work runs through a durable background lifecycle queue. Thread creation and first-prompt bootstrap wait for lifecycle events through the creation sequence to be observed, then wait only for pending cleanup of the same thread ID before starting new runtime resources. Unrelated bulk archives do not hold that creation wait; callers that require all lifecycle work to finish can still drain the whole queue. Archive and restore lifecycle work is serialized per project tree, with reference-counted idle-lock eviction after the final user or waiter. Archive eligibility and provider/terminal/log-writer shutdown run under that same tree lock, so a queued archive job that becomes stale after unarchive cannot stop the newly active session or delete its terminal history. Archive creation commits the complete compressed bundle, including its restore metadata, before the main-only destructive transaction deletes hot rows and marks the manifest `cleanup_pending`. This ordering avoids relying on atomic commits across attached databases when the hot database uses WAL. Destructive attachment/log cleanup must finish before the manifest becomes `cold`. Destructive archive transitions require terminal/log-writer shutdown before chunk creation and recheck the archived shell inside the hot-row deletion transaction; `archiveThread` normalizes typed quiesce failures to `ThreadColdStorageError` at the archive operation boundary. Provider shutdown also remains required while the projected session is `starting` or `running`; a missing provider binding is terminal for a settled or absent projected session, allowing legacy subagent shells that never owned a provider binding to move cold safely. Successful provider quiescence explicitly clears process-local background-liveness state, including the no-binding settled-shell path where no later `session.exited` event is guaranteed. The cold-storage boundary chunks child projections only after its caller-provided quiescence effect completes; pending-input coverage verifies that restored session and activity rows reflect stopped and resolved values while the lightweight hot shell retains a zero pending-input count. Failed lifecycle work requests one coalesced delayed rescan of the durable pending state instead of being dropped or introducing a permanent polling loop, enqueue interruptions release their deduplication reservation, and restart recovery preserves the same retry boundary. Filesystem failures other than a genuinely missing directory also keep retry state. Incomplete cleanup, including a `cleanup_pending` manifest whose archived shell has already been removed, and archived shells missing a manifest resume after restart without rebuilding an already durable bundle.
@@ -113,7 +113,7 @@ Failed shell-driven detail-cache removals remain pending while the authoritative
113113

114114
`apps/server/src/server.test.ts` verifies that HTTP-seeded shell subscriptions preserve archive removals published while WebSocket catch-up reads persisted events, then emit the synchronization completion marker. Initial-snapshot event replay in `apps/server/src/ws.ts` and deferred active-thread cache writes in `packages/client-runtime/src/state/threads.ts` remain complementary to authoritative shell refresh. `packages/client-runtime/src/state/shell-sync.test.ts` covers a ready-to-ready authorization session handoff through the authoritative archive snapshot and its detail-cache tombstone, while `apps/web/src/composerDraftArchiveObserver.test.ts` covers retaining the draft baseline during replacement synchronization and releasing its uploads only after the replacement shell becomes authoritative. The remaining cache lifecycle behavior is covered through `packages/client-runtime/src/state/threadCommands.archive.test.ts`, `packages/client-runtime/src/state/threads-sync.test.ts`, `apps/web/src/connection/storage.test.ts`, and the mobile storage/database tests.
115115

116-
Mobile Archive rows omit invalid lifecycle timestamps instead of presenting corrupt or legacy values as newly archived, and `apps/mobile/src/features/archive/archivedThreadList.ts` sorts missing or invalid archive dates after valid dates in both sort directions. General mobile time rendering is unchanged.
116+
Mobile Archive rows omit invalid lifecycle timestamps, including impossible calendar dates, instead of presenting corrupt or legacy values as newly archived. `apps/mobile/src/features/archive/archivedThreadList.ts` uses the shared zoned timestamp validator and sorts missing or invalid archive dates after valid dates in both sort directions. General mobile time rendering is unchanged.
117117

118118
## Development ports
119119

‎apps/mobile/src/features/archive/archivedThreadList.test.ts‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,7 @@ describe("buildArchivedThreadGroups", () => {
154154
title: "Invalid",
155155
});
156156
const anotherInvalid = makeThread({
157-
archivedAt: "also-not-a-timestamp",
157+
archivedAt: "2026-02-30T00:00:00.000Z",
158158
id: ThreadId.make("thread-another-invalid"),
159159
projectId: project.id,
160160
title: "Another invalid",
@@ -203,7 +203,19 @@ describe("buildArchivedThreadGroups", () => {
203203
});
204204

205205
describe("formatArchivedThreadRelativeTime", () => {
206-
it("omits invalid archive timestamps instead of presenting them as recent", () => {
207-
expect(formatArchivedThreadRelativeTime("not-a-timestamp")).toBeNull();
206+
it.each(["not-a-timestamp", "2026-02-30T00:00:00.000Z", "2025-02-29T12:00:00+02:00"])(
207+
"omits invalid archive timestamp %s",
208+
(timestamp) => {
209+
expect(formatArchivedThreadRelativeTime(timestamp)).toBeNull();
210+
},
211+
);
212+
213+
it.each([
214+
"2024-02-29T12:00:00+02:00",
215+
"2026-06-01T23:30:00-05:00",
216+
"0099-01-01T00:00:00Z",
217+
"+010000-01-01T00:00:00Z",
218+
])("keeps the label for valid archive timestamp %s", (timestamp) => {
219+
expect(formatArchivedThreadRelativeTime(timestamp)).not.toBeNull();
208220
});
209221
});

‎apps/mobile/src/features/archive/archivedThreadList.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import {
66
type EnvironmentThreadShell,
77
} from "@t3tools/client-runtime/state/shell";
88
import type { EnvironmentId } from "@t3tools/contracts";
9+
import { parseTimestamp as parseZonedTimestamp } from "@t3tools/shared/dateTime";
910
import * as Arr from "effect/Array";
1011
import * as Order from "effect/Order";
1112

@@ -21,7 +22,7 @@ export interface ArchivedThreadGroup {
2122
}
2223

2324
function parseTimestamp(input: string): number | null {
24-
const timestamp = Date.parse(input);
25+
const timestamp = parseZonedTimestamp(input);
2526
return Number.isNaN(timestamp) ? null : timestamp;
2627
}
2728

‎apps/server/src/orchestration/ThreadColdStorage.test.ts‎

Lines changed: 101 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { assert, it } from "@effect/vitest";
2-
import { vi } from "vitest";
2+
import { vi } from "vite-plus/test";
33
import * as NodeServices from "@effect/platform-node/NodeServices";
44
import { ThreadId } from "@t3tools/contracts";
55
import * as Effect from "effect/Effect";
@@ -1122,53 +1122,81 @@ layer("ThreadColdStorage", (it) => {
11221122
}),
11231123
);
11241124

1125-
it.effect("ignores traversal attachment entries while restoring", () =>
1126-
Effect.gen(function* () {
1127-
const sql = yield* SqlClient.SqlClient;
1128-
const storage = yield* ThreadColdStorage.ThreadColdStorage;
1129-
const config = yield* ServerConfig.ServerConfig;
1130-
const fs = yield* FileSystem.FileSystem;
1131-
const path = yield* Path.Path;
1132-
const threadId = ThreadId.make("thread-traversal");
1133-
const attachmentName = "thread-traversal-00000000-0000-4000-8000-000000000001.png";
1134-
1135-
yield* insertArchivedThread(threadId, "Traversal thread");
1136-
yield* sql.unsafe(
1137-
`INSERT INTO projection_thread_messages (
1125+
it.effect(
1126+
"retains cold data when an unsafe attachment entry blocks restore, then retries after repair",
1127+
() =>
1128+
Effect.gen(function* () {
1129+
const sql = yield* SqlClient.SqlClient;
1130+
const storage = yield* ThreadColdStorage.ThreadColdStorage;
1131+
const config = yield* ServerConfig.ServerConfig;
1132+
const fs = yield* FileSystem.FileSystem;
1133+
const path = yield* Path.Path;
1134+
const threadId = ThreadId.make("thread-traversal");
1135+
const attachmentName = "thread-traversal-00000000-0000-4000-8000-000000000001.png";
1136+
1137+
yield* insertArchivedThread(threadId, "Traversal thread");
1138+
yield* sql.unsafe(
1139+
`INSERT INTO projection_thread_messages (
11381140
message_id, thread_id, turn_id, role, text, attachments_json,
11391141
is_streaming, created_at, updated_at
11401142
) VALUES (?, ?, NULL, 'user', 'validate attachment name', ?, 0, ?, ?)`,
1141-
[
1142-
"message-traversal",
1143-
threadId,
1144-
encodeUnknownJsonString([
1145-
{
1146-
type: "image",
1147-
id: attachmentName.slice(0, -4),
1148-
name: "image.png",
1149-
mimeType: "image/png",
1150-
},
1151-
]),
1152-
"2026-07-01T00:00:00.000Z",
1153-
"2026-07-01T00:00:00.000Z",
1154-
],
1155-
);
1156-
yield* fs.writeFileString(path.join(config.attachmentsDir, attachmentName), "image bytes");
1157-
yield* storage.archiveThread(threadId);
1158-
const escapedPath = path.join(config.attachmentsDir, "..", "thread-traversal-escape");
1159-
yield* sql`
1143+
[
1144+
"message-traversal",
1145+
threadId,
1146+
encodeUnknownJsonString([
1147+
{
1148+
type: "image",
1149+
id: attachmentName.slice(0, -4),
1150+
name: "image.png",
1151+
mimeType: "image/png",
1152+
},
1153+
]),
1154+
"2026-07-01T00:00:00.000Z",
1155+
"2026-07-01T00:00:00.000Z",
1156+
],
1157+
);
1158+
yield* fs.writeFileString(path.join(config.attachmentsDir, attachmentName), "image bytes");
1159+
yield* storage.archiveThread(threadId);
1160+
const escapedPath = path.join(config.attachmentsDir, "..", "thread-traversal-escape");
1161+
yield* sql`
11601162
UPDATE cold_archive.archive_thread_chunks
11611163
SET kind = 'attachment:../thread-traversal-escape'
11621164
WHERE thread_id = ${threadId} AND kind LIKE 'attachment:%'
11631165
`;
11641166

1165-
assert.isTrue(yield* storage.restoreTree(threadId));
1166-
const manifest = yield* sql<{ readonly status: string }>`
1167+
const failure = yield* Effect.flip(storage.restoreTree(threadId));
1168+
assert.strictEqual(failure.operation, "restore");
1169+
const manifest = yield* sql<{ readonly status: string }>`
11671170
SELECT status FROM thread_archive_manifests WHERE thread_id = ${threadId}
11681171
`;
1169-
assert.deepStrictEqual(manifest, [{ status: "restored" }]);
1170-
assert.isFalse(yield* fs.exists(escapedPath));
1171-
}),
1172+
assert.deepStrictEqual(manifest, [{ status: "cold" }]);
1173+
assert.deepStrictEqual(
1174+
yield* sql`SELECT message_id FROM projection_thread_messages WHERE thread_id = ${threadId}`,
1175+
[],
1176+
);
1177+
assert.isAbove(
1178+
(yield* sql<{
1179+
count: number;
1180+
}>`SELECT COUNT(*) AS count FROM cold_archive.archive_thread_chunks WHERE thread_id = ${threadId}`)[0]
1181+
?.count ?? 0,
1182+
0,
1183+
);
1184+
assert.isFalse(yield* fs.exists(escapedPath));
1185+
yield* sql`
1186+
UPDATE cold_archive.archive_thread_chunks
1187+
SET kind = ${`attachment:${attachmentName}`}
1188+
WHERE thread_id = ${threadId} AND kind LIKE 'attachment:%'
1189+
`;
1190+
assert.isTrue(yield* storage.restoreTree(threadId));
1191+
assert.strictEqual(
1192+
yield* fs.readFileString(path.join(config.attachmentsDir, attachmentName)),
1193+
"image bytes",
1194+
);
1195+
assert.deepStrictEqual(
1196+
yield* sql`SELECT text FROM projection_thread_messages WHERE thread_id = ${threadId}`,
1197+
[{ text: "validate attachment name" }],
1198+
);
1199+
}),
11721200
);
11731201

11741202
it.effect("keeps cold SQL data authoritative when attachment restore fails", () =>
@@ -1468,6 +1496,41 @@ layer("ThreadColdStorage", (it) => {
14681496
UPDATE projection_threads SET archived_at = NULL WHERE thread_id = ${threadId}
14691497
`;
14701498

1499+
yield* sql`
1500+
INSERT INTO cold_archive.archive_threads (thread_id, root_thread_id, archive_version, archived_at, original_bytes, compressed_bytes, created_at)
1501+
VALUES (${threadId}, ${threadId}, 1, '2026-07-02T00:00:00.000Z', 1, 1, CURRENT_TIMESTAMP)
1502+
`;
1503+
yield* sql`
1504+
INSERT INTO cold_archive.archive_thread_chunks (thread_id, chunk_index, kind, row_count, data)
1505+
VALUES (${threadId}, 0, 'table:projection_thread_messages', 0, ${new Uint8Array([1])})
1506+
`;
1507+
const unsafe = sql.unsafe.bind(sql);
1508+
const spy = vi
1509+
.spyOn(sql, "unsafe")
1510+
.mockImplementation((query, params) =>
1511+
unsafe(
1512+
query.includes("DELETE FROM thread_archive_manifests")
1513+
? "SELECT * FROM missing_aborted_archive_cleanup_fixture"
1514+
: query,
1515+
params,
1516+
),
1517+
);
1518+
yield* Effect.flip(storage.archiveThread(threadId)).pipe(
1519+
Effect.ensuring(Effect.sync(() => spy.mockRestore())),
1520+
);
1521+
assert.deepStrictEqual(
1522+
yield* sql`SELECT status FROM thread_archive_manifests WHERE thread_id = ${threadId}`,
1523+
[{ status: "pending" }],
1524+
);
1525+
assert.deepStrictEqual(
1526+
yield* sql`SELECT thread_id FROM cold_archive.archive_threads WHERE thread_id = ${threadId}`,
1527+
[],
1528+
);
1529+
assert.deepStrictEqual(
1530+
yield* sql`SELECT chunk_index FROM cold_archive.archive_thread_chunks WHERE thread_id = ${threadId}`,
1531+
[],
1532+
);
1533+
assert.include(yield* storage.listPendingArchiveThreadIds, threadId);
14711534
yield* storage.archiveThread(threadId);
14721535
const messages = yield* sql<{ readonly text: string }>`
14731536
SELECT text FROM projection_thread_messages WHERE thread_id = ${threadId}

0 commit comments

Comments
 (0)