Add SQLite snapshot artifacts under backup#94805
Conversation
|
Codex review: needs changes before merge. Reviewed July 6, 2026, 11:59 PM ET / 03:59 UTC. Summary PR surface: Source +767, Tests +885, Docs +95, Other +27. Total +1774 across 16 files. Reproducibility: not applicable. this is a new CLI/artifact feature rather than a current-main bug. The review used source inspection plus the PR body's after-fix Windows source-harness proof instead of a failing reproduction. Review metrics: 2 noteworthy metrics.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Copy recommended automerge instructionNext step before merge
Maintainer decision needed
Security Review findings
Review detailsBest possible solution: Land only after removing the lockfile churn and stale root Do we have a high-confidence way to reproduce the issue? Not applicable: this is a new CLI/artifact feature rather than a current-main bug. The review used source inspection plus the PR body's after-fix Windows source-harness proof instead of a failing reproduction. Is this the best way to solve the issue? Unclear until maintainer product direction is set: the nested backup command shape is plausible and reuses the existing SQLite backup primitive, but the PR is not the best merge candidate while lockfile churn, stale startup policy, and the core-vs-plugin decision remain unresolved. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against dc5d4138f9ae. Label changesLabel justifications:
Evidence reviewedPR surface: Source +767, Tests +885, Docs +95, Other +27. Total +1774 across 16 files. View PR surface stats
Security concerns:
Acceptance criteria:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (2 earlier review cycles) |
|
@clawsweeper re-review I updated the PR body with packaged CLI proof for |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Codex maintainer review: one blocker remains before this snapshot artifact contract should ship. Findings:
Notes from independent review:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Superseded by the hardened canonical implementation in #105718, landed as 2f25e9c. Thank you, @giodl73-repo, for establishing the SQLite-safe artifact direction and the original backup-surface design. This PR could not be updated because The concurrent stress concept remains open in #94967 and will be adapted onto the landed API with author credit rather than closing that validation work here. |
Summary
This PR adds a SQLite-safe snapshot artifact path under the existing backup command surface.
It includes:
openclaw backup sqlite snapshot create|verify|restore|list--target globaland--agent <id>--db <path>support for advanced/manual SQLite files--dbpaths before snapshottingdelivery_queue_entries, matching the backup recovery invariant-wal,-shm, and-journalsidecarsThe important surface decision is that this is no longer a top-level
openclaw snapshotcommand. It is now a smaller SQLite-specific backup primitive:openclaw backup sqlite snapshot.openclaw backup createremains the broad.tar.gzrecovery archive for state, config, credentials, sessions, and optional workspaces.Why
The goal from #94675 is not to make hosts copy live SQLite files more carefully. The goal is for OpenClaw to produce a SQLite-safe sync artifact.
This PR establishes that boundary:
.sqlite,-wal,-shm, and-journalfiles remain hot local runtime stateopenclaw backup sqlite snapshot creatematerializes a portable artifact directory containingmanifest.jsonanddatabase.sqliteThe Lobster-shaped host case is covered by this contract: hosts ask OpenClaw for named database targets and sync completed artifacts. This PR intentionally does not add a
memory-searchtarget, because memory-search currently shares the per-agent database and a memory-only export needs a separate design.Scout Pilot Checkpoint
Before asking maintainers to land this core path, we are exercising the same SQLite-safe artifact boundary in Microsoft Scout/Lobster through the Lobster-owned snapshot plugin path.
That pilot is the operational evidence checkpoint for this PR. It should tell us whether Scout can reliably sync only completed
manifest.jsonanddatabase.sqlitesnapshot artifacts while ignoring live SQLite files, temp files, and sidecars; how often snapshots are needed; what artifact size and creation time look like in real hosted use; and whether the long-term surface should be coreopenclaw backup sqlite snapshot, a host-owned plugin command, or both.If the pilot shows full snapshots are too large, too slow, or too frequent for the hosted path, that is the data we should use before greenlighting WAL bundles or other Phase 2 work.
Stack
This PR combines the previous core implementation slices:
backup sqlite snapshotcommand surfaceFollow-up validation PR: #94967
RFC: openclaw/rfcs#20
Related: #94675, #94646
Scope
This does not add memory-search-only export, WAL bundle deltas, object storage upload, scheduling, leases, failover automation, restore-on-boot behavior, production orchestration, or a new memory-search path config. Those remain future phases or separate maintainer decisions. This PR ships the local SQLite artifact contract under the backup command surface.
Real Behavior Proof
Behavior addressed:
OpenClaw can convert OpenClaw-owned SQLite state into syncable artifact directories for global and per-agent databases, verify them, copy them independently from live SQLite files, and restore them into verified fresh SQLite databases. Global snapshot artifacts also scrub transient delivery queue rows, and restore removes stale SQLite sidecars before opening the restored database.
Real environment tested:
Windows checkout at
C:\src\openclaw-snapshot-sqlite-command, PR head409d54170f, Node v22.22.3, source Vitest harnesses with WAL-mode SQLite databases, named OpenClaw database targets, realpath coverage, global transient queue scrubbing, rollback journal cleanup, config schema validation, and CLI registration coverage foropenclaw backup sqlite snapshot.Exact steps or command run after this patch:
node scripts/run-vitest.mjs run src/cli/program/register.backup.test.ts src/cli/program/command-registry.test.ts src/commands/snapshot.test.ts src/snapshot/snapshot-provider.test.tsgit diff --checknode scripts/format-docs.mjs --checkpnpm exec oxfmt --check src/cli/program/register.backup.ts src/cli/program/register.backup.test.ts src/cli/program/command-registry-core.ts src/cli/program/core-command-descriptors.ts docs/cli/backup.md docs/cli/index.md docs/docs.jsonpnpm docs:check-linksEvidence after fix:
Focused validation passed 3 Vitest shards:
src/cli/program/register.backup.test.ts: backup archive commands plus nestedbackup sqlite snapshot create|verify|restore|listwiringsrc/cli/program/command-registry.test.ts: top-level command registry behavior after removing standalonesnapshotsrc/commands/snapshot.test.tsandsrc/snapshot/snapshot-provider.test.ts: SQLite snapshot create/list/verify/restore behavior and provider invariantsgit diff --check origin/main...HEADpassed at409d54170fafter rebasing on currentorigin/main.node scripts/format-docs.mjs --checkreports no formatting issue in touched docs after formattingdocs/cli/index.md; the only remaining reported file is pre-existing unrelateddocs/reference/templates/CLAUDE.md.pnpm exec oxfmt --check ...was blocked before formatting by dependency installation timeout while fetching large optional/native packages. No formatter findings were produced.pnpm docs:check-linkstimed out locally after dependency/tool startup; no broken-link finding was produced before timeout. A repo search confirms no remainingcli/snapshotoropenclaw snapshotreferences for this feature; remainingsnapshotmatches are browser snapshot docs or unrelated runtime/test wording.Observed result after fix:
The public CLI surface is now
openclaw backup sqlite snapshot create|verify|restore|list. The standalone top-levelsnapshotcommand registration, docs page, and docs-nav entry were removed. Existing SQLite artifact behavior remains covered by the provider and command tests, while backup registration tests prove the new nested command path forwards options to the same implementation.What was not tested:
No live gateway runtime, Azure Blob upload, scheduled backup loop, restore-on-boot path, lease/failover path, cross-platform stress, memory-search-only export, or WAL bundle/delta behavior was tested in this PR. Microsoft Scout/Lobster plugin-path evidence is being collected separately before the core command is treated as the accepted path. Stress validation is tracked separately in #94967.