Skip to content

refactor(matrix): move ephemeral state to plugin sqlite#88387

Merged
steipete merged 2 commits into
mainfrom
codex/matrix-plugin-state-sqlite
May 30, 2026
Merged

refactor(matrix): move ephemeral state to plugin sqlite#88387
steipete merged 2 commits into
mainfrom
codex/matrix-plugin-state-sqlite

Conversation

@steipete

@steipete steipete commented May 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • move Matrix inbound dedupe, startup verification, and thread bindings from plugin-local JSON files into Matrix plugin-state SQLite namespaces
  • import legacy JSON once with migration markers, remove stale fallback reads/writes, and preserve account/storage-root isolation
  • keep Matrix crypto, recovery key, and IDB sync storage on existing disk files
  • update the channel session-binding contract helper so bundled Matrix contract tests provide the same plugin-state runtime seam

Verification

  • pnpm test extensions/matrix/src/matrix/client/storage.test.ts extensions/matrix/src/matrix/monitor/inbound-dedupe.test.ts extensions/matrix/src/matrix/monitor/startup-verification.test.ts extensions/matrix/src/matrix/thread-bindings.test.ts -- --reporter=verbose
  • pnpm test:contracts:channels -- src/channels/plugins/contracts/session-binding.registry-backed.contract.test.ts --reporter=verbose
  • /Users/steipete/Projects/agent-scripts/skills/autoreview/scripts/autoreview --mode local --prompt 'Final review after removing storage root legacy startup JSON deviceId fallback. Confirm startup verification no longer relies on legacy JSON for deviceId fallback, thread binding SQLite writes still claim current storage root, all legacy JSON imports are one-shot with markers, no fresh legacy JSON writes, inbound persistence remains best-effort, and account/state-dir isolation is preserved.'
  • /Users/steipete/Projects/agent-scripts/skills/autoreview/scripts/autoreview --mode local --prompt 'Review the incremental fix after CI channel contract failure. Confirm Matrix contract test runtime now exposes plugin-state openKeyedStore through the SDK test helper, no production fallback/shim was added, and the Matrix SQLite state behavior remains intact.'
  • pnpm check:changed via Blacksmith Testbox tbx_01kswxxmpby4963btjwtm0aqf8, GitHub Actions run 26689939748

Real behavior proof

Behavior addressed: Matrix plugin-local ephemeral state now persists in plugin-state SQLite for inbound dedupe, startup verification, and thread bindings, with one-shot import from legacy JSON and no movement of crypto/recovery/IDB stores.

Real environment tested: local macOS checkout plus delegated Blacksmith Testbox Linux environment.

Exact steps or command run after this patch: focused Matrix Vitest command above, channel session-binding contract command above, structured autoreview commands above, and pnpm check:changed in Testbox tbx_01kswxxmpby4963btjwtm0aqf8.

Evidence after fix: focused Matrix tests passed 1 Vitest shard with 43 tests; session-binding contract passed 4 Vitest shards with 20 tests; both autoreview runs returned autoreview clean: no accepted/actionable findings reported; Testbox changed gate exited 0.

Observed result after fix: Matrix SQLite state migration and persistence tests pass; bundled channel contract runtime covers Matrix plugin-state access; changed typecheck, lint, import-cycle, dependency, and guard checks pass.

What was not tested: live Matrix homeserver traffic.

@openclaw-barnacle openclaw-barnacle Bot added channel: discord Channel integration: discord channel: matrix Channel integration: matrix size: XL maintainer Maintainer-authored PR labels May 30, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 363ac064cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

metadata.deviceId = verification.deviceId.trim();
}

return metadata;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve legacy startup device metadata before routing storage

For upgraded Matrix installs where the only recorded device id is still in the existing startup-verification.json, removing this fallback makes readStoredRootMetadata() return no deviceId. resolvePreferredMatrixStorageRoot() then rejects sibling roots unless they explicitly declare the same device, so after an access-token rotation it selects a fresh canonical root instead of the existing crypto/thread state; the new SQLite startup-verification migration cannot fix this because it also resolves its state path through resolveMatrixStoragePaths() first. Keep importing or honoring this legacy device id before applying the explicit-device match.

Useful? React with 👍 / 👎.

@steipete
steipete force-pushed the codex/matrix-plugin-state-sqlite branch from 363ac06 to d2df089 Compare May 30, 2026 17:05
@openclaw-barnacle openclaw-barnacle Bot removed the channel: discord Channel integration: discord label May 30, 2026
@clawsweeper

clawsweeper Bot commented May 30, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed May 30, 2026, 1:12 PM ET / 17:12 UTC.

Summary
The PR moves Matrix inbound dedupe, startup verification, and thread bindings from plugin-local JSON files to plugin-state SQLite with one-shot legacy imports while leaving crypto, recovery key, and IDB storage on disk.

PR surface: Source +384, Tests +92. Total +476 across 10 files.

Reproducibility: yes. for the patch defect from source inspection: current main reads legacy startup-verification deviceId during storage-root selection, while the PR removes it and only migrates startup state after resolving a root. I did not execute a failing PR-head test because this was a read-only review.

Review metrics: 1 noteworthy metric.

  • Matrix state stores migrated: 3 moved from JSON to plugin-state SQLite. Inbound dedupe, startup verification, and thread bindings are persisted state surfaces, so upgrade and one-shot import behavior need maintainer-visible scrutiny before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Preserve or pre-migrate legacy startup-verification deviceId before storage-root selection filters sibling roots.
  • [P1] Add a regression test covering an old populated Matrix root whose only device metadata is legacy startup-verification.json.
  • [P1] Add redacted live Matrix proof, such as terminal output or logs from startup/token rotation; redact IPs, tokens, phone numbers, private endpoints, and other private details.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body reports focused Vitest, autoreview, and changed-check proof, but no live Matrix startup/token-rotation run or redacted runtime logs showing the changed persistence behavior after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] Merging as-is can strand upgraded Matrix installs on a fresh current-token storage root when the previous root's only device metadata is legacy startup-verification.json, losing access to existing crypto and thread-binding state until repaired manually.
  • [P1] The proof currently shows focused tests/checks and autoreview only; it does not show an after-fix live Matrix homeserver startup or token-rotation path.

Maintainer options:

  1. Preserve legacy device metadata before routing (recommended)
    Keep or import the legacy startup-verification deviceId during root selection, then add a focused upgrade test where the old populated root has no storage-meta deviceId.
  2. Accept a breaking Matrix storage migration
    Maintainers could intentionally accept fresh-root selection for that legacy state, but this should be explicit because existing crypto and thread state can become unreachable after upgrade.

Next step before merge

  • [P1] A maintainer/contributor needs to address the storage-upgrade blocker and add real Matrix behavior proof; this should not be auto-repaired while the contributor proof gate remains unsatisfied.

Security
Cleared: No concrete security or supply-chain regression was found; the diff uses existing Matrix/plugin-state storage APIs and does not change workflows, dependencies, lockfiles, or secret handling.

Review findings

  • [P1] Preserve legacy startup device metadata before routing storage — extensions/matrix/src/matrix/client/storage.ts:507-528
Review details

Best possible solution:

Preserve or migrate legacy startup-verification deviceId before storage-root routing enforces explicit device matching, add a regression test for that upgrade state, and then provide redacted live Matrix startup/token-rotation proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes for the patch defect from source inspection: current main reads legacy startup-verification deviceId during storage-root selection, while the PR removes it and only migrates startup state after resolving a root. I did not execute a failing PR-head test because this was a read-only review.

Is this the best way to solve the issue?

No. Moving Matrix ephemeral state into plugin-state SQLite is a reasonable direction, but the safe implementation must preserve legacy device metadata before root selection rather than recording it only after a root has already been chosen.

Full review comments:

  • [P1] Preserve legacy startup device metadata before routing storage — extensions/matrix/src/matrix/client/storage.ts:507-528
    recordCurrentStorageMetaDeviceId() only writes the deviceId after startup verification has already resolved a storage path. Because this PR also stops readStoredRootMetadata() from reading legacy startup-verification.json, an upgraded install whose old token root only has that legacy device marker will fail explicit device matching, choose the fresh canonical root, and lose access to the existing Matrix crypto/thread state. Keep or import that legacy deviceId before resolvePreferredMatrixStorageRoot() filters sibling roots, and add the upgrade regression test.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against e708a872a1fd.

Label changes

Label changes:

  • add P2: This is a compatibility-sensitive Matrix persistence refactor with a concrete pre-merge blocker but it is not yet affecting released users.
  • add merge-risk: 🚨 compatibility: The PR removes a shipped legacy storage-root fallback that existing upgraded Matrix installations may rely on after token rotation.
  • add merge-risk: 🚨 session-state: Choosing a fresh Matrix storage root can make existing crypto and thread-binding state unreachable for the same account/device.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports focused Vitest, autoreview, and changed-check proof, but no live Matrix startup/token-rotation run or redacted runtime logs showing the changed persistence behavior after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a compatibility-sensitive Matrix persistence refactor with a concrete pre-merge blocker but it is not yet affecting released users.
  • merge-risk: 🚨 compatibility: The PR removes a shipped legacy storage-root fallback that existing upgraded Matrix installations may rely on after token rotation.
  • merge-risk: 🚨 session-state: Choosing a fresh Matrix storage root can make existing crypto and thread-binding state unreachable for the same account/device.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports focused Vitest, autoreview, and changed-check proof, but no live Matrix startup/token-rotation run or redacted runtime logs showing the changed persistence behavior after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +384, Tests +92. Total +476 across 10 files.

View PR surface stats
Area Files Added Removed Net
Source 6 584 200 +384
Tests 4 190 98 +92
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 10 774 298 +476

What I checked:

  • Repository policy read: Full root AGENTS.md and scoped extensions/AGENTS.md were read; their compatibility guidance treats persisted state, migrations, setup/startup checks, and removed fallbacks as upgrade-sensitive review surfaces. (AGENTS.md:16, e708a872a1fd)
  • Current main honors legacy startup device metadata: Current main reads startup-verification.json inside readStoredRootMetadata() and uses its deviceId when storage-meta.json lacks one, allowing old Matrix storage roots to pass explicit device matching during token rotation. (extensions/matrix/src/matrix/client/storage.ts:136, e708a872a1fd)
  • PR head removes that fallback: The PR head version of readStoredRootMetadata() returns after storage-meta.json parsing and no longer reads startup-verification.json, so old roots that only recorded deviceId there become incompatible before sibling-root reuse is considered. (extensions/matrix/src/matrix/client/storage.ts:135, d2df089a287c)
  • New SQLite migration is too late for old-root discovery: PR head resolves the startup-verification legacy file path through resolveMatrixStoragePaths() before reading or migrating the legacy JSON, so if storage-root selection already chose the fresh canonical root, the old root's legacy startup file is never imported. (extensions/matrix/src/matrix/monitor/startup-verification.ts:298, d2df089a287c)
  • Existing tests document token-rotation storage-root behavior: Current and PR-head storage tests cover storage-root reuse after access-token rotation, including startup-verification marker scenarios, but there is no focused PR-head test for an old populated root whose only device metadata remains in legacy startup-verification.json. (extensions/matrix/src/matrix/client/storage.test.ts:608, e708a872a1fd)
  • History provenance: Git blame ties the current Matrix storage-root metadata fallback, token-rotation tests, and backfill path to commit 5eb7192 by steipete; the PR head d2df089 changes the same files. (extensions/matrix/src/matrix/client/storage.ts:106, 5eb71927b7e7)

Likely related people:

  • steipete: Git blame and commit metadata tie the current Matrix storage-root fallback, Matrix token-rotation tests, and this PR head to steipete, making them the clearest routing candidate for the compatibility decision. (role: introduced behavior and recent area contributor; confidence: high; commits: 5eb71927b7e7, d2df089a287c; files: extensions/matrix/src/matrix/client/storage.ts, extensions/matrix/src/matrix/client/storage.test.ts, extensions/matrix/src/matrix/monitor/startup-verification.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels May 30, 2026
@steipete
steipete merged commit 84a965a into main May 30, 2026
107 of 109 checks passed
@steipete
steipete deleted the codex/matrix-plugin-state-sqlite branch May 30, 2026 17:26
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 31, 2026
* refactor(matrix): persist ephemeral state in plugin sqlite

* test(channels): wire matrix contract plugin state
SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026
* refactor(matrix): persist ephemeral state in plugin sqlite

* test(channels): wire matrix contract plugin state
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
* refactor(matrix): persist ephemeral state in plugin sqlite

* test(channels): wire matrix contract plugin state
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: matrix Channel integration: matrix maintainer Maintainer-authored PR merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XL status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant