Skip to content

Persist subagent registry in SQLite#88260

Merged
steipete merged 2 commits into
mainfrom
codex/sqlite-subagent-registry
May 30, 2026
Merged

Persist subagent registry in SQLite#88260
steipete merged 2 commits into
mainfrom
codex/sqlite-subagent-registry

Conversation

@steipete

@steipete steipete commented May 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replaces the subagent registry JSON persistence path with the shared OpenClaw SQLite state database.
  • Uses the existing generated Kysely schema for subagent_runs reads, upserts, and snapshot deletes.
  • Imports existing subagents/runs.json once when SQLite is empty, then removes the legacy file so stale runs cannot be re-imported.

Replaces #88166 with the SQLite-backed approach discussed there.

Verification

  • node scripts/run-vitest.mjs src/agents/subagent-registry.store.sqlite.test.ts src/agents/subagent-registry.persistence.test.ts src/gateway/session-utils.subagent.test.ts
  • pnpm tsgo:core
  • pnpm tsgo:core:test
  • node scripts/run-oxlint.mjs src/agents/subagent-registry-state.ts src/agents/subagent-registry.store.sqlite.ts src/agents/subagent-registry.store.sqlite.test.ts
  • .agents/skills/autoreview/scripts/autoreview --mode local -> autoreview clean: no accepted/actionable findings reported

Real behavior proof

Behavior addressed: Subagent registry runs persist through the shared SQLite state database instead of a standalone JSON registry file, while existing JSON state migrates on first read.
Real environment tested: Local macOS checkout with Node/Vitest/tsgo/oxlint; SQLite state database exercised through temp OPENCLAW_STATE_DIR test fixtures.
Exact steps or command run after this patch: node scripts/run-vitest.mjs src/agents/subagent-registry.store.sqlite.test.ts src/agents/subagent-registry.persistence.test.ts src/gateway/session-utils.subagent.test.ts.
Evidence after fix: Focused tests covered SQLite persistence, whole-registry snapshot deletion, and one-shot legacy JSON import/removal.
Observed result after fix: 3 test files passed, 48 tests passed; tsgo:core, tsgo:core:test, targeted oxlint, and autoreview also passed.
What was not tested: Full pnpm check:changed; an earlier Testbox delegation was stopped after it became stale relative to a local cleanup patch.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M maintainer Maintainer-authored PR labels May 30, 2026
@clawsweeper

clawsweeper Bot commented May 30, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed May 30, 2026, 3:06 AM ET / 07:06 UTC.

Summary
The PR switches subagent registry persistence from subagents/runs.json to the shared SQLite subagent_runs table with one-shot JSON import/removal and new focused tests.

PR surface: Source +318, Tests +134. Total +452 across 3 files.

Reproducibility: not applicable. this is a backend persistence replacement PR rather than a standalone bug report. Current-main source verifies the existing JSON-backed behavior, and the PR proof is focused fixture coverage of the new SQLite path.

Review metrics: 1 noteworthy metric.

  • Persistent state backend changed: 1 registry backend replaced; 1 legacy import/removal path added. This changes upgrade behavior for persisted subagent run state, not just an internal helper implementation.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add real behavior proof for a preexisting subagents/runs.json migrating into SQLite across an OpenClaw restart, with private paths, endpoints, phone numbers, and credentials redacted.
  • [P1] Coordinate whether this PR or Cache subagent read snapshots on gateway hot paths #88166 is the canonical registry-state path before merge.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body reports focused Vitest/temp-state fixture coverage and local type/lint checks, but no real OpenClaw subagent restart or upgrade run after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The migration deletes subagents/runs.json after import, so a mapping or restore mistake could affect persisted subagent session/delivery state during upgrade.
  • [P1] The proof in the PR body is focused tests and type/lint checks only; it does not show a real OpenClaw subagent run with preexisting JSON state surviving restart into SQLite.
  • [P1] The PR body says it replaces Cache subagent read snapshots on gateway hot paths #88166 while that pull request remains open, so maintainers should choose one canonical subagent registry direction before landing both.

Maintainer options:

  1. Prove the upgrade path before merge (recommended)
    Run a real OpenClaw subagent workflow with an existing subagents/runs.json, restart on the patched build, and show that the run restores from SQLite and the legacy file removal is safe.
  2. Accept maintainer-owned migration risk
    A maintainer can intentionally land the SQLite migration with the current focused tests, but should own the possibility of upgrade/session-state fallout if real restart proof is skipped.
  3. Pause until the sibling PR is resolved
    If Cache subagent read snapshots on gateway hot paths #88166 remains the preferred hot-path fix, pause or close this PR rather than keeping two competing registry-state approaches open.

Next step before merge

  • [P1] The PR has a protected maintainer label, session-state migration risk, mock-only proof, and an open sibling PR it claims to replace, so the next action is maintainer review rather than automated repair.

Security
Cleared: No concrete security or supply-chain concern found; the PR adds SQLite persistence code and tests without new dependencies, workflow changes, secret handling, or external code execution.

Review details

Best possible solution:

Land a single canonical SQLite-backed subagent registry path only after proving a real JSON-to-SQLite upgrade/restart flow and coordinating the replaced hot-path pull request.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a backend persistence replacement PR rather than a standalone bug report. Current-main source verifies the existing JSON-backed behavior, and the PR proof is focused fixture coverage of the new SQLite path.

Is this the best way to solve the issue?

Unclear: the SQLite direction matches the existing subagent_runs schema and is plausible, but the best merge path needs real upgrade proof and a maintainer decision that this supersedes the sibling hot-path PR.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 26bf8f0dc87b.

Label changes

Label changes:

  • add P2: This is a normal-priority agent persistence improvement with limited scope but meaningful upgrade/session-state blast radius.
  • add merge-risk: 🚨 compatibility: The PR changes the persisted registry backend and removes the legacy JSON file after import, so existing upgraded checkouts depend on a correct migration path.
  • add merge-risk: 🚨 session-state: The changed data is active subagent run registry state used for restart, completion delivery, cleanup, and gateway read snapshots.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports focused Vitest/temp-state fixture coverage and local type/lint checks, but no real OpenClaw subagent restart or upgrade run after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a normal-priority agent persistence improvement with limited scope but meaningful upgrade/session-state blast radius.
  • merge-risk: 🚨 compatibility: The PR changes the persisted registry backend and removes the legacy JSON file after import, so existing upgraded checkouts depend on a correct migration path.
  • merge-risk: 🚨 session-state: The changed data is active subagent run registry state used for restart, completion delivery, cleanup, and gateway read snapshots.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports focused Vitest/temp-state fixture coverage and local type/lint checks, but no real OpenClaw subagent restart or upgrade run after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +318, Tests +134. Total +452 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 2 323 5 +318
Tests 1 134 0 +134
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 457 5 +452

What I checked:

  • Current main uses JSON-backed subagent registry persistence: On current main, persistSubagentRunsToDisk, restoreSubagentRunsFromDisk, and read snapshots call the JSON store helpers from subagent-registry.store.js, so the PR is not already implemented on main. (src/agents/subagent-registry-state.ts:1, 26bf8f0dc87b)
  • PR replaces the persistence calls with SQLite helpers: The PR branch changes the state wrapper to call saveSubagentRegistryToSqlite and loadSubagentRegistryFromSqlite for persistence, restore, and read snapshots. (src/agents/subagent-registry-state.ts:1, af5d77d4969e)
  • SQLite store imports and removes legacy JSON only when SQLite is empty: The new store reads SQLite rows first, imports subagents/runs.json only when no SQLite runs exist, then removes the legacy file after saving the imported snapshot. (src/agents/subagent-registry.store.sqlite.ts:278, af5d77d4969e)
  • Current schema already contains the target table: Current main's generated state schema includes subagent_runs with typed run, delivery, completion, and payload columns used by the PR branch. (src/state/openclaw-state-schema.sql:967, 26bf8f0dc87b)
  • Focused tests cover SQLite persistence and legacy import: The added test file covers saving/restoring from SQLite, whole-registry snapshot deletion, and legacy JSON import/removal, but it uses temp state fixtures rather than a real OpenClaw subagent run. (src/agents/subagent-registry.store.sqlite.test.ts:74, af5d77d4969e)
  • Repository policy treats this as compatibility and session-state sensitive: Root policy calls auth/session state, persisted preferences, migrations, fallback behavior, and upgrade-sensitive paths merge-risk surfaces; this directly affected the risk classification. (AGENTS.md:20, 26bf8f0dc87b)

Likely related people:

  • steipete: Recent path history includes SQLite runtime-state work and revert commits, plus delivery-state refactoring adjacent to the persistence shape this PR changes. (role: recent area contributor; confidence: high; commits: 694ca50e9775, f91de52f0d23, d73f3ac85d5f; files: src/agents/subagent-registry.store.ts, src/agents/subagent-delivery-state.ts, src/state/openclaw-state-db.ts)
  • galiniliev: Authored the recent cached subagent registry read work on the same registry store/state path, which is directly affected by switching the backing store to SQLite. (role: recent area contributor; confidence: high; commits: 935f84b8e936; files: src/agents/subagent-registry-state.ts, src/agents/subagent-registry.store.ts, src/agents/subagent-registry.persistence.test.ts)
  • yetval: Co-authored the recent strict initial subagent registry persistence fix that established current failure-handling expectations around registry saves. (role: introduced adjacent behavior; confidence: medium; commits: 214f718be7b3; files: src/agents/subagent-registry.ts, src/agents/subagent-registry-state.ts, src/agents/subagent-registry.persistence.test.ts)
  • takhoffman: The recent strict persistence commit records approval by takhoffman, making them a useful routing candidate for persistence and upgrade semantics. (role: reviewer; confidence: medium; commits: 214f718be7b3; files: src/agents/subagent-registry.ts, src/agents/subagent-registry-state.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels May 30, 2026
@steipete

Copy link
Copy Markdown
Contributor Author

Real behavior proof

Behavior addressed: migrate legacy JSON subagent registry data into the SQLite subagent_runs table, remove the stale legacy JSON file, and keep gateway-backed subagent listing usable after migration.

Real environment tested: AWS Crabbox cbx_286c59e69b0d (pearl-hermit), run run_f14c45c72840, Linux c7a.8xlarge, real installed OpenClaw packages, real OpenAI key provided via redacted env profile, model openai/gpt-5.5.

Exact steps or command run after this patch:

node scripts/crabbox-wrapper.mjs run \
  --provider aws \
  --idle-timeout 90m \
  --ttl 240m \
  --timing-json \
  --env-from-profile <redacted-openai-profile> \
  --allow-env OPENAI_API_KEY \
  --download /tmp/openclaw-sqlite-migration-proof.json=/tmp/openclaw-sqlite-migration-proof.remote.json \
  --script /tmp/openclaw-crabbox-sqlite-migration-install-e2e.sh

Evidence after fix: baseline pre-SQLite origin/main tarball installed as baseline_version:2026.5.28; replacement PR tarball installed as replacement_version:2026.5.28; baseline gateway + agent created two JSON-backed subagents; replacement gateway migrated them and the post-migration agent turn returned the expected token. Proof JSON downloaded from Crabbox showed legacyJsonRemoved: true and final_sqlite_rows:2.

Observed result after fix: SQLite contained two migrated sqlite_migrate_child_* rows with child session keys, requester session key agent:main:sqlite-migration-live, completion timestamps, and completion payloads.

What was not tested: published [email protected] as the legacy baseline. It failed before this PR's migration path with Unable to resolve bundled plugin public surface speech-core/runtime-api.js, so the migration proof used an installed origin/main pre-SQLite package tarball as the legacy baseline.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae69064010

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +280 to +281
if (runs.size > 0) {
return runs;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove stale legacy registry even when SQLite has rows

If subagent_runs is already populated while an old subagents/runs.json still exists (for example, a first post-upgrade write occurs before the migration read, or the process crashes after the SQLite save but before removeLegacySubagentRegistryFile()), this early return leaves the legacy file behind. Once the current SQLite rows are later swept/released, the next loadSubagentRegistryFromSqlite() sees an empty table and imports that stale JSON, resurrecting old subagent runs despite the intended one-way migration.

Useful? React with 👍 / 👎.

@steipete
steipete merged commit 5374c7a into main May 30, 2026
107 checks passed
@steipete
steipete deleted the codex/sqlite-subagent-registry branch May 30, 2026 09:44
mcaxtr pushed a commit that referenced this pull request May 30, 2026
* fix(agents): persist subagent registry in sqlite

* test(agents): mock sqlite subagent registry in loop guard
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 31, 2026
* fix(agents): persist subagent registry in sqlite

* test(agents): mock sqlite subagent registry in loop guard
SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026
* fix(agents): persist subagent registry in sqlite

* test(agents): mock sqlite subagent registry in loop guard
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
* fix(agents): persist subagent registry in sqlite

* test(agents): mock sqlite subagent registry in loop guard
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling maintainer Maintainer-authored PR merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant