Skip to content

refactor(sqlite): centralize verified snapshot publication#105412

Merged
vincentkoc merged 1 commit into
mainfrom
fix/sqlite-snapshot-stress
Jul 12, 2026
Merged

refactor(sqlite): centralize verified snapshot publication#105412
vincentkoc merged 1 commit into
mainfrom
fix/sqlite-snapshot-stress

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

What Problem This Solves

SQLite archive backup had a private VACUUM INTO implementation that could not be reused safely by snapshot/restore work. A second implementation would risk weaker integrity, sanitization, publication, and durability guarantees.

Why This Change Was Made

  • Add one shared SQLite snapshot primitive that validates the source and final artifact with quick, full integrity, index, and foreign-key checks.
  • Compact after any snapshot transform so removed sensitive rows cannot remain in free pages.
  • Publish only verified private staging files, using no-overwrite hard-link or exclusive streamed-copy paths.
  • Fsync files with Windows-compatible handles, verify file identity across publication, and clean up only artifacts proven to be owned by the operation.
  • Export the existing global state owner/schema validator for future offline snapshot and restore validation.
  • Move archive backup onto the shared primitive without changing its CLI contract.

This is the current-main foundation for the snapshot direction explored in #94805. The commit preserves Gio Della-Libera's co-author credit while replacing the stale prototype's direct-copy restore assumptions with a smaller verified primitive.

User Impact

openclaw backup create keeps the same interface, but SQLite snapshots now use a reusable publication path hardened against unsafe indexes, foreign-key corruption, target races, partial fallback copies, post-sanitization remnants, and Windows fsync behavior.

Evidence

  • node scripts/run-vitest.mjs run src/infra/sqlite-snapshot.test.ts src/infra/backup-create.test.ts src/commands/backup-verify.test.ts src/commands/doctor-state-sqlite-compact.test.ts
    • 96 tests passed on Blacksmith Testbox through Crabbox, id tbx_01kxb7nzpq84zv43m20x3ep69t
  • corepack pnpm check:changed
    • passed on the same Testbox
  • oxfmt --check on all touched files
    • passed
  • .agents/skills/autoreview/scripts/autoreview --mode local
    • clean, no accepted/actionable findings

@vincentkoc vincentkoc self-assigned this Jul 12, 2026
@openclaw-barnacle openclaw-barnacle Bot added commands Command implementations size: L maintainer Maintainer-authored PR labels Jul 12, 2026
@vincentkoc

Copy link
Copy Markdown
Member Author

Land-ready at 2ec26fe13dd6e4a72dffd364e9acbc77486cd19d.

What changed:

  • centralized verified SQLite VACUUM INTO snapshot creation;
  • recompacts after sanitization so deleted bytes do not survive;
  • publishes through no-overwrite hard-link or exclusive streamed-copy paths;
  • fsyncs with Windows-compatible handles and rejects target replacement races;
  • preserves only identity-proven concurrent/partial cleanup semantics;
  • reuses the global owner/schema validator from the state DB owner module.

Proof:

  • node scripts/run-vitest.mjs run src/infra/sqlite-snapshot.test.ts src/infra/backup-create.test.ts src/commands/backup-verify.test.ts src/commands/doctor-state-sqlite-compact.test.ts
    • 96 tests passed on Blacksmith Testbox through Crabbox tbx_01kxb7nzpq84zv43m20x3ep69t
  • corepack pnpm check:changed
    • passed on the same Testbox
  • touched-file oxfmt --check
    • passed
  • mandatory autoreview
    • clean, no accepted/actionable findings
  • OPENCLAW_TESTBOX=1 scripts/pr prepare-run 105412
    • passed in hosted_exact_or_recent_rebase mode for the exact PR head

Known gap: the cross-platform snapshot/restore/vacuum stress matrix is the next batch; this PR lands the shared primitive it will exercise.

@vincentkoc
vincentkoc marked this pull request as ready for review July 12, 2026 13:46
@vincentkoc
vincentkoc merged commit ef196b6 into main Jul 12, 2026
113 of 119 checks passed
@vincentkoc
vincentkoc deleted the fix/sqlite-snapshot-stress branch July 12, 2026 13:47
@vincentkoc

Copy link
Copy Markdown
Member Author

Merged via squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations maintainer Maintainer-authored PR size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant