Skip to content

feat(backup): restore a staged backup into a fresh root, verified against its manifest (#731) - #983

Merged
github-actions[bot] merged 2 commits into
mainfrom
fix/731-restore-verify
Nov 11, 2026
Merged

github-actions[bot] merged 2 commits into
mainfrom
fix/731-restore-verify

Conversation

@cdeust

@cdeust cdeust commented Nov 11, 2026

Copy link
Copy Markdown
Owner

Refs #731 (slice 4; the issue stays open, see "Still open").

What

Until now nothing read a staging tree back, and the manifest was returned to the caller but written nowhere.

  • Manifest in the backup. snapshot_backup writes cortex-backup-manifest.json at the top of the staging tree (0600, never over an existing file).
  • Restore. New restore_backup(archive, target) in mcp_server/handlers/backup_restore.py.
  • Refusals before any write:
    • the manifest is absent, a symlink, not JSON, or of another format, version or resource table;
    • the manifest is a managed PostgreSQL one;
    • the tree does not carry exactly the manifest's files: each absent, extra or changed file is named by the new mcp_server/core/backup_verify.py;
    • a SQLite store fails integrity_check, read with the file opened immutable, so nothing is written beside the backup.
  • Target. The target must not exist. Files are copied owner-only into <target>.partial, that copy is compared with the manifest again, and only then renamed to the target. A failed restore leaves .partial and no target.

Proof

A memory inserted in a live SqliteMemoryStore is read back by get_memory from the restored root (tests_py/handlers/test_backup_restore.py). Refusal cases and the target handling are in test_backup_restore_target.py, tests_py/core/test_backup_verify.py and tests_py/infrastructure/test_backup_stage.py.

Checks

  • Backup tests, 10 files across core, handlers and infrastructure: 147 passed.
  • ruff check and ruff format --check on the changed files: clean.
  • scripts/check_craftsmanship.py: OK. scripts/check_doc_claims.py: OK.
  • Mutation, whole files, with the CI's own test selection: backup_verify.py 47/47, backup_restore.py 110/110, backup_snapshot.py 90/90, backup_stage.py 78/78, sqlite_snapshot.py 62/62 killed. No equivalent registered.

Not verified

  • Windows: the symlink and permission tests are skipped there, and the rename of .partial to the target is untested there.
  • The full suite and pyright were not run locally; this pull request's CI runs the suite.
  • The mutation figures come from a local run of mutation_check.sh with mutmut 3.8.1, not from the CI's diff-scoped run.

Still open on #731

  1. Managed PostgreSQL cluster: refused on both the backup and the restore side.
  2. Tested recovery and upgrade-rollback documentation per platform.
  3. No MCP tool or CLI is registered for backup or restore; these are handler functions only.
  4. "Preserving source links and version chains" is not demonstrated by a dedicated query; the test reads one memory and one wiki page.

Generated by Claude Code

…ed (#731, slice 4)

A staging tree could be taken (slice 3) but nothing read it back: its manifest
was returned to the caller and written nowhere, and no code copied the tree
into a root or compared it with its digests.

snapshot_backup now writes the manifest at the top of the staging tree
(cortex-backup-manifest.json, 0600, never over a file). restore_backup reads
it and refuses the backup before any write when the manifest is absent, of
another format, version or resource table, or of a managed PostgreSQL install;
when the tree does not carry exactly the manifest's files (core/backup_verify
names each absent, extra or changed one, and the scan still refuses a link);
or when the SQLite store fails integrity_check, read on the file opened
immutable so nothing is written beside the backup.

The target must not exist. The files are copied owner-only into
<target>.partial, that copy is scanned and compared with the manifest again,
and only then renamed to the target: a failed restore leaves no target.

Tests run on the real handlers: a memory written to a live SqliteMemoryStore
is read from the store of the restored root. Mutation gate run locally per
changed module with the CI test selection: 0 survivors.

Not verified here: Windows, pyright, the full suite (torch cannot be installed
in this container). The managed cluster is still refused on both sides.

Refs #731

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_018kyZYrKg46KKYpvLPXcW4M
Signed-off-by: Claude <[email protected]>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-11T09:26:15.993067Z 58f0621 New commits
🔒 Security Review ✅ Completed 2026-11-11T09:27:22.454487Z 58f0621 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Claude finished @cdeust's task in 2m 45s —— View job


Review of PR #983 (review 1 of 3)

  • Gather context (diff, CLAUDE.md, /review-pr instructions)
  • Read the diff myself
  • Run the 5 /review-pr reviewers (code quality, performance, test coverage, documentation accuracy, security)
  • Check their findings against the code
  • Post the review (review comment)

Result: I found no Blockers. The review comment lists 3 Notes, none of which needs action before merge:

  • The SQLite integrity check reads the backup, not the copy that becomes the install.
  • The rename at the end can silently replace an empty directory created at the target during the restore.
  • Two docstrings say more than the code does: the tree is said to carry "exactly the manifest's files", and backup_verify.py is credited with failures that other modules report.

I didn't run the tests or the linters; this review comes from reading the code.

Verdict: approve

@claude

claude Bot commented Nov 11, 2026

Copy link
Copy Markdown

Review 1 of 3: PR #983 (restore a staged backup, #731 slice 4)

I found no Blockers. Five reviewers looked at this PR (code quality, performance, test coverage, documentation accuracy, security), and I traced their findings in the code myself.

Checked and found safe

  • Path traversal through the manifest: parse_manifest rebuilds each entry from the resource table. Every file path must be canonical and must sit under its resource's path, so copy_files and private_dirs only receive checked paths.
  • Symlinks: a symlink in the backup tree is refused by the scan, and a manifest that is a symlink is refused by read_text_file.
  • Writes: output files are opened O_EXCL|O_NOFOLLOW, so nothing is written through a link or over an existing file.
  • The restored copy: it is verified against the manifest digests again before the rename, so content swapped during the copy is caught.
  • Layers and limits: the layer rule, the file and method size limits and the # source: rule all hold.

Notes, no action before merge

  • mcp_server/handlers/backup_restore.py:97 runs check_snapshot on the backup, not on the copy. If memory.db is swapped between the integrity check and the copy, the copy is never integrity-checked. Running it on partial / entry["path"] after line 105 closes that gap.
  • backup_restore.py:106: on POSIX, partial.rename(target) silently replaces an empty directory created at target while the restore runs. The "never writes over an install" check happens only once, at line 98.
  • The docstrings at backup_restore.py:10-12 and core/backup_verify.py:7-8 say the tree carries "exactly the manifest's files" and that core/backup_verify.py names each failure. In the code:
    • extra files are detected only inside the resource paths in the table;
    • symlinks are refused by backup_scan, and a missing store by build_manifest.

View job run
Verdict: approve

…r refused (#731, slice 4)

fix(backup): one audited write site in backup_stage (check unit, run 42990175181)

write_new_file opened its own os.open/os.fdopen pair, a second storage
primitive the ADR-1110 redaction-gate audit counts once. Both writers now go
through _new_private_file, so the audited count stays exact.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018kyZYrVXPdxGjW7GSfGfmC

cdeust commented Nov 11, 2026

Copy link
Copy Markdown
Owner Author

ci-autopilot — tentative 1

Diagnostic : le check unit (run 42990175181, image ubuntu-26.04 20261101.186.1) échoue sur tests_py/architecture/test_storage_redaction_gate.py::test_every_storage_primitive_is_an_audited_site ; merge n'échoue que parce que unit est rouge. La cause est dans la PR : le nouveau write_new_file de infrastructure/backup_stage.py ouvrait sa propre paire os.open/os.fdopen, soit deux sites d'écriture là où l'audit ADR-1110 en compte un. Au lieu d'augmenter le compteur de l'audit, les deux écrivains (copy_files et write_new_file) passent désormais par un seul _new_private_file, de sorte que le site audité reste unique et que la raison inscrite reste exacte.

Commit poussé : 58f0621 (fixup! de d975f92, absorbé au prochain rebase autosquash).

Vérifié localement : le test de gate échoue sur d975f92 (1 failed) et passe sur 58f0621 ; test_storage_redaction_gate, test_backup_stage, test_sqlite_snapshot, test_backup_verify, test_backup_restore, test_backup_restore_target : 57 passed ; ruff check, ruff format --check et scripts/check_craftsmanship.py OK.
Non vérifié : la suite complète (environnement sans torch ni PostgreSQL ; les tests PostgreSQL ont été sautés) ; la CI sur 58f0621 est en cours, aucun check vert n'est affirmé ici.


Generated by Claude Code

@claude

claude Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Claude finished @cdeust's task in 2m 3s —— View job


Review 2 of 3: PR #983

  • Check earlier Blockers: review 1 had none.
  • Review the only commit pushed after d975f92: 58f0621 (backup_stage.py, which now has one audited write site).
  • Run the 5 /review-pr reviewers (code quality, performance, test coverage, documentation accuracy, security).
  • Post the review (review comment).

Result: I found no Blockers. The refactor keeps the flags (O_EXCL|O_NOFOLLOW) and the 0600 mode, and it brings the ADR-1110 gate counts back to the audited 1/1/1. The review comment lists 3 Notes, none of which needs action before merge:

  • The audit reason in the gate test doesn't mention the manifest write.
  • A descriptor can leak if os.fdopen fails.
  • Nothing tests a symlink at the write target directly.

I didn't run the tests or the linters, because pytest needs approval in this job. This review comes from reading the code.

Verdict: approve

@claude

claude Bot commented Nov 11, 2026

Copy link
Copy Markdown

Review 2 of 3: PR #983 (restore a staged backup, #731 slice 4)

Earlier Blockers: none. Review 1 approved d975f92 with Notes only.

Scope: only the fixup commit pushed after it, 58f0621 (mcp_server/infrastructure/backup_stage.py). Unchanged code was not re-audited. Five reviewers (code quality, performance, test coverage, documentation accuracy, security) looked at the commit, and I checked their findings against the code.

What the commit does: copy_files and write_new_file both open their file through _new_private_file now. Nothing else changes:

  • The flags are the same: _CREATE keeps O_EXCL|O_NOFOLLOW and adds no O_TRUNC or O_APPEND.
  • The mode is still 0600, and exceptions still pass through.
  • In with source, _new_private_file(target), source is still closed if the open raises.

This brings the ADR-1110 gate's counts for this module back to the audited 1/1/1 (os.open-write, open-write, shutil-copy, tests_py/architecture/test_storage_redaction_gate.py:98-110). The method and file size limits still hold, and no new constant needs a # source:.

I found no Blockers.

Notes, no action before merge

  • The audit reason at tests_py/architecture/test_storage_redaction_gate.py:100-101 says the bytes written at this site are "a copy of class A files already written through the gate". Since this commit, the site also writes the manifest (handlers/backup_snapshot.py:96), which holds paths, digests and sizes but no memory content. Naming both writers there would keep the audit reason accurate.
  • backup_stage.py:49: if os.fdopen raises, the descriptor from os.open is never closed, and the empty 0600 file stays behind. That needs an async exception in practice.
  • Nothing tests that a symlink placed at the write target is refused, so a regression that drops O_EXCL from _CREATE would only be caught indirectly (tests_py/infrastructure/test_backup_stage.py).

I didn't run the tests or the linters: pytest needs approval in this job. This review comes from reading the code.

View job run
Verdict: approve

@github-actions
github-actions Bot merged commit 656a14e into main Nov 11, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants