Skip to content

fix(scripts): add database-first legacy store guard#91629

Merged
galiniliev merged 17 commits into
mainfrom
bug-004-database-first-legacy-store-guard
Jun 15, 2026
Merged

fix(scripts): add database-first legacy store guard#91629
galiniliev merged 17 commits into
mainfrom
bug-004-database-first-legacy-store-guard

Conversation

@galiniliev

Copy link
Copy Markdown
Contributor

Summary

  • Adds check:database-first-legacy-stores, an AST-based guard for legacy runtime state store writes under source, plugin, and package code.
  • Wires the guard into architecture checks, changed checks for core/plugin lanes, and the normal check preflight.
  • Adds focused regression coverage for direct, aliased, destructured, helper-based, and path-variable legacy writes while allowing doctor, migration, and known fixture owners.
  • Intentionally out of scope: migrating any remaining legacy stores; this PR adds the missing guard that prevents new steady-state regressions.
  • Success looks like the database-first plan having a deterministic command instead of relying on noisy manual searches.

Reviewers should focus on the allowlist boundaries and whether the guard catches realistic write shapes without blocking migration/doctor owners.

Linked context

Closes #91628

Related #81402

Requested from the local BUG-004 database-first validation bundle.

Real behavior proof (required for external PRs)

  • Behavior addressed: The database-first validation plan references pnpm check:database-first-legacy-stores, but that command was missing, so new legacy runtime state writes could be reintroduced without a focused check.
  • Real environment tested: Linux source checkout with installed dependencies, Node/pnpm repo tooling, branch bug-004-database-first-legacy-store-guard.
  • Exact steps or command run after this patch:
pnpm check:database-first-legacy-stores
node scripts/run-vitest.mjs test/scripts/changed-lanes.test.ts test/scripts/check.test.ts test/scripts/check-database-first-legacy-stores.test.ts -- --reporter=verbose
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode local
  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):
$ pnpm check:database-first-legacy-stores
$ node scripts/check-database-first-legacy-stores.mjs
Database-first legacy-store guard passed.

$ node scripts/run-vitest.mjs test/scripts/changed-lanes.test.ts test/scripts/check.test.ts test/scripts/check-database-first-legacy-stores.test.ts -- --reporter=verbose
[test] passed 2 Vitest shards in 12.30s
Test Files  2 passed (2)
Tests  73 passed (73)

$ git diff --check
# no output

$ .agents/skills/autoreview/scripts/autoreview --mode local
autoreview clean: no accepted/actionable findings reported
Overall correctness confidence: patch is correct (0.83)
  • Observed result after fix: The new guard command exists and exits successfully on the current source tree, the targeted regression tests pass, whitespace checks pass, and autoreview reports no accepted/actionable findings.
  • What was not tested: Full pnpm check, broad CI, and live provider/channel runs were not run; this is a repository tooling/architecture guard and does not change provider or channel runtime behavior.
  • Proof limitations or environment constraints: This proof uses local repository commands and copied terminal output. No private runtime session or credentialed live setup is required for this guard.
  • Before evidence (optional but encouraged):
Source: SQLite plan execution
RAW: MISSING check:database-first-legacy-stores

Source: SQLite plan local summary
RAW: check:database-first-legacy-stores: missing; manual searches captured
RAW: manual legacy hits: 13095
RAW: manual state write hits: 1224

Tests and validation

  • pnpm check:database-first-legacy-stores passed.
  • node scripts/run-vitest.mjs test/scripts/changed-lanes.test.ts test/scripts/check.test.ts test/scripts/check-database-first-legacy-stores.test.ts -- --reporter=verbose passed 73 tests across 2 Vitest shards.
  • git diff --check passed.
  • .agents/skills/autoreview/scripts/autoreview --mode local reported no accepted/actionable findings.

Regression coverage was added in test/scripts/check-database-first-legacy-stores.test.ts, and test/scripts/changed-lanes.test.ts was updated to prove the guard is included for core production changes.

Risk checklist

Did user-visible behavior change? (Yes/No)

No.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

False positives or false negatives in the architecture guard.

How is that risk mitigated?

The scanner is covered against representative write forms, aliases, path variables, helper APIs, path segment assembly, filename constants, payload false positives, identifier shadowing, and migration/doctor owner exceptions. The guard is also wired into changed checks for source lanes so regressions are caught before broader CI.

Current review state

Next action: maintainer review and CI.

Waiting on CI and maintainer judgment for the allowlist boundaries.

Autoreview findings from earlier local runs were addressed; the latest local autoreview run is clean.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts size: L maintainer Maintainer-authored PR labels Jun 9, 2026
@clawsweeper

clawsweeper Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed June 14, 2026, 10:53 PM ET / 02:53 UTC.

Summary
The PR adds an AST-based check:database-first-legacy-stores script, wires it into architecture/preflight/changed checks, narrows the database-first docs, and adds regression tests.

PR surface: Tests +8464, Docs -9, Config +1, Other +9786. Total +18242 across 7 files.

Reproducibility: yes. Source inspection shows current main documents check:database-first-legacy-stores while package.json does not define it, and the PR scanner gap is source-reproducible through resolveDefaultSessionStorePath() returning sessions.json. I did not run local tests because this review is read-only.

Review metrics: 1 noteworthy metric.

  • Custom guard scale: 9,783-line script, 8,461-line test. This unusually large custom analyzer is being added to mandatory repository automation, so maintainers should review long-term false-positive and false-negative ownership before merge.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Add resolver-call detection and regression coverage for current legacy path APIs.
  • [P2] Get maintainer sign-off on the required guard scope and sequencing with the database-first migration.

Risk before merge

  • [P1] The required guard still misses writes whose path comes from current resolver APIs, so a new sessions.json write can pass without using a literal legacy filename.
  • [P1] The PR makes a large custom flow-sensitive analyzer part of preflight, architecture, and changed-check automation, so false positives or false negatives can block unrelated source work.
  • [P1] The protected maintainer label and open canonical database-first migration mean the permanent guard scope and rollout sequencing need explicit owner approval.

Maintainer options:

  1. Fix Resolver Coverage Before Merge (recommended)
    Detect writes whose path comes from current legacy-store resolver APIs and add direct plus assigned-call regression cases before the guard becomes required.
  2. Accept A Narrow Guard Contract
    Maintainers may merge the current scanner only if they explicitly accept resolver-returned legacy paths as a known follow-up gap.
  3. Sequence With The SQLite Migration
    Pause this PR until the canonical database-first migration settles the permanent state boundary and guard ownership.

Next step before merge

  • [P2] Manual review is needed because the PR has a protected maintainer label and the remaining blocker is the owner-approved guard contract and migration sequencing, not only a mechanical code edit.

Security
Cleared: No concrete security or supply-chain regression was found; the diff adds local static-analysis tooling, tests, docs, and package-script wiring without new dependencies, lockfile changes, secrets handling, or broader permissions.

Review findings

  • [P2] Cover resolver-returned legacy paths — scripts/check-database-first-legacy-stores.mjs:8548-8549
Review details

Best possible solution:

Add focused resolver-call detection and tests, or explicitly narrow the required guard with maintainer approval, before treating this scanner as mandatory database-first automation.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main documents check:database-first-legacy-stores while package.json does not define it, and the PR scanner gap is source-reproducible through resolveDefaultSessionStorePath() returning sessions.json. I did not run local tests because this review is read-only.

Is this the best way to solve the issue?

No, not yet. The package/check wiring is the right repair shape, but the required scanner should cover current resolver-returned legacy paths or carry an explicit maintainer-approved scope limitation before merge.

Full review comments:

  • [P2] Cover resolver-returned legacy paths — scripts/check-database-first-legacy-stores.mjs:8548-8549
    pathArgumentContainsLegacyStore only delegates to literal/tracked-expression detection, so imported resolver calls such as fs.writeFile(resolveDefaultSessionStorePath(), ...) are not violations even though current src/config/sessions/paths.ts returns sessions.json. Add resolver-call recognition and regression cases before making this guard required.
    Confidence: 0.92

Overall correctness: patch is incorrect
Overall confidence: 0.92

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against ac1042b09b5c.

Label changes

Label changes:

  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🦐 gold shrimp, so this older rating label is no longer current.

Label justifications:

  • P2: The PR addresses a normal-priority repository tooling bug in database-first validation rather than an active user-facing outage.
  • merge-risk: 🚨 automation: The diff changes required preflight, architecture, and changed-check automation with a scanner that still has a concrete false-negative gap.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body includes after-fix terminal output from a real Linux source checkout for the new command, focused tests, git diff --check, and local autoreview; that is sufficient real behavior proof for this tooling change.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a real Linux source checkout for the new command, focused tests, git diff --check, and local autoreview; that is sufficient real behavior proof for this tooling change.
Evidence reviewed

PR surface:

Tests +8464, Docs -9, Config +1, Other +9786. Total +18242 across 7 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 2 8464 0 +8464
Docs 1 7 16 -9
Config 1 2 1 +1
Generated 0 0 0 0
Other 3 9786 0 +9786
Total 7 18259 17 +18242

What I checked:

  • Repository policy read and applied: Root AGENTS.md and scoped guides for scripts/, docs/, and test/ were read; the database-first storage policy and script-wrapper guidance affected this review. (AGENTS.md:1, ac1042b09b5c)
  • Current main lacks the documented package command: Current main defines check, check:architecture, and nearby guards, but has no check:database-first-legacy-stores package script in the script block. (package.json:1455, ac1042b09b5c)
  • Current docs require the missing guard: The database-first plan on current main describes check:database-first-legacy-stores as the guard for new legacy runtime store writes and bridge contracts. Public docs: docs/refactor/database-first.md. (docs/refactor/database-first.md:1407, ac1042b09b5c)
  • PR makes the guard required automation: The PR head adds the package script, appends it to check:architecture, and runs it from preflight and changed-check lanes for all/core/extension changes. (package.json:1456, 8d477a1bbc2f)
  • Resolver-returned legacy paths remain a false negative: The scanner's path check delegates to expression/literal tracking, while current main exposes resolveDefaultSessionStorePath() returning sessions.json; an imported resolver call used directly in fs.writeFile(...) has no literal or tracked local legacy identifier for the scanner to match. (scripts/check-database-first-legacy-stores.mjs:8548, 8d477a1bbc2f)
  • Focused tests do not cover resolver APIs: The PR test file has extensive literal, alias, wrapper, copy, and helper coverage, but no resolveDefaultSessionStorePath or resolveSessionStorePath regression case was found. (test/scripts/check-database-first-legacy-stores.test.ts:1, 8d477a1bbc2f)

Likely related people:

  • vincentkoc: Current-main blame attributes the documented guard text, package check script block, check runner preflight area, and session path resolver to Vincent Koc's recent commit. (role: current area contributor; confidence: high; commits: ccf5976d0621; files: package.json, scripts/check.mjs, scripts/check-changed.mjs)
  • steipete: The open canonical database-first migration is authored by steipete and defines the broader SQLite state boundary that this guard is meant to enforce. (role: database-first refactor owner; confidence: high; commits: 2d0147905305, f91de52f0d23, 694ca50e9775; files: docs/refactor/database-first.md, src/state/openclaw-state-schema.sql, src/state/openclaw-agent-schema.sql)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels Jun 9, 2026
@galiniliev galiniliev self-assigned this Jun 9, 2026
@galiniliev
galiniliev force-pushed the bug-004-database-first-legacy-store-guard branch from 7414971 to d93ece9 Compare June 12, 2026 06:22
@galiniliev

Copy link
Copy Markdown
Contributor Author

Maintainer review after rebase to current origin/main (d93ece95d5d10867ec877e20e85bf87369209c4f): one blocker remains.

The new guard is wired into check, check:architecture, and changed-check lanes, and the alias/wrapper coverage is much stronger after the latest commit. Local proof on the rebased branch passed:

pnpm check:database-first-legacy-stores
node scripts/run-vitest.mjs test/scripts/changed-lanes.test.ts test/scripts/check.test.ts test/scripts/check-database-first-legacy-stores.test.ts -- --reporter=verbose
git diff --check origin/main...HEAD

Finding:

  • [P2] Align the guard with the documented non-write contract before making it required. docs/refactor/database-first.md says check:database-first-legacy-stores also fails reintroduced sessionFile, storePath, SessionManager.open(...), old static SessionManager facades, the old session JSONL downloader hook/class, sidecar-shaped plugin-state/task SQLite helper names, and old memory-index table names. The pushed scanner still only treats transcriptLocator and sqlite-transcript:// as bridge markers in scripts/check-database-first-legacy-stores.mjs, so probes like export const sessionFile = currentSqlitePath, export const storePath = currentSqlitePath, SessionManager.open("sessions.json"), and SessionManager.listAll() return no violations. Because this guard is now in preflight/architecture/changed-check paths, it should either implement the documented contract with explicit current-owner handling, or the docs/PR scope should be narrowed to a filesystem-write-only guard before merge.

Best-fix verdict: acceptable mitigation only if the contract is deliberately narrowed; otherwise too narrow for the documented database-first guard.

@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 12, 2026
@openclaw-barnacle openclaw-barnacle Bot added the docs Improvements or additions to documentation label Jun 12, 2026
@galiniliev

Copy link
Copy Markdown
Contributor Author

Prep update for b6b37afe040aeb38ba8653a1b3187c212e254816:

  • Resolved the earlier documented-contract blocker by narrowing docs/refactor/database-first.md to the guard this PR actually implements: legacy store names paired with filesystem writes plus the retired transcriptLocator / sqlite-transcript://... bridge markers. The broader sessionFile / storePath / old SessionManager facade guard is now called out as separate migration-guard follow-up work because those names still have current owners.
  • Fixed the CI check-lint failures in scripts/check-database-first-legacy-stores.mjs: unnecessary type conversion, shadowed names, lonely if, and implicit boolean coercion.

Local proof after the prep patch:

pnpm lint:scripts
pnpm check:database-first-legacy-stores
node scripts/run-vitest.mjs test/scripts/changed-lanes.test.ts test/scripts/check.test.ts test/scripts/check-database-first-legacy-stores.test.ts -- --reporter=verbose
pnpm format:docs:check docs/refactor/database-first.md
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode local

Result: all local proof passed, and autoreview reported autoreview clean: no accepted/actionable findings reported. GitHub CI has restarted on the new SHA and is still in progress.

@galiniliev
galiniliev force-pushed the bug-004-database-first-legacy-store-guard branch from b6b37af to e4d52a2 Compare June 12, 2026 21:32
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels Jun 12, 2026
@galiniliev
galiniliev force-pushed the bug-004-database-first-legacy-store-guard branch from e4d52a2 to 8d477a1 Compare June 15, 2026 02:43
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 15, 2026
@openclaw-barnacle openclaw-barnacle Bot added the extensions: memory-core Extension: memory-core label Jun 15, 2026
@galiniliev

Copy link
Copy Markdown
Contributor Author

Land-ready proof for head 34dde2c.

Work done before landing:

  • Rebased onto current main ac1042b.
  • Fixed the accepted review gaps in scripts/check-database-first-legacy-stores.mjs, including nested wrapper/default/spread coverage and the overly broad extension migration exemption.
  • Added focused regressions in test/scripts/check-database-first-legacy-stores.test.ts, including the migration-like extension runtime write case.
  • Included one unrelated current-main lint unblocker in extensions/memory-core/src/memory/manager-embedding-ops.ts after CI check-lint failed on an unnecessary non-null assertion.

Local verification:

pnpm check:database-first-legacy-stores
pnpm lint:scripts
node scripts/run-vitest.mjs test/scripts/check-database-first-legacy-stores.test.ts -- --reporter=verbose
node scripts/run-oxlint.mjs extensions/memory-core/src/memory/manager-embedding-ops.ts
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main

Result: all local commands passed. The Vitest guard suite passed 468 tests, and the final branch autoreview reported no accepted/actionable findings.

GitHub verification:

  • PR is mergeable against main at head 34dde2c.
  • Relevant CI checks are green, including check-lint, check-guards, check-docs, check-test-types, check-prod-types, build-artifacts, check:database-first guard coverage through CI lanes, Windows test, security-fast, dependency guard, Opengrep, Socket, and ci-timings-summary.
  • Routine skipped/noise lanes remain skipped as expected: selected Critical Quality shards, Mintlify, macos-swift, node22 compat, and native Telegram proof lanes that were not selected for this tooling PR.

Known proof gap: no live provider/channel run was needed or performed because this PR changes repository tooling/architecture guard behavior, not runtime provider or channel behavior.

@galiniliev
galiniliev merged commit 50c82b3 into main Jun 15, 2026
166 of 167 checks passed
@galiniliev
galiniliev deleted the bug-004-database-first-legacy-store-guard branch June 15, 2026 03:08
@galiniliev

Copy link
Copy Markdown
Contributor Author

Landed.

Source head: 34dde2c
Landed commit: 50c82b3

Linked issue #91628 was closed by the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation extensions: memory-core Extension: memory-core maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. scripts Repository scripts size: XL status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Missing database-first legacy-store guard script

1 participant