Skip to content

refactor(apns): move registrations to shared SQLite#108543

Merged
steipete merged 12 commits into
mainfrom
codex/sqlite-apns-registrations
Jul 16, 2026
Merged

refactor(apns): move registrations to shared SQLite#108543
steipete merged 12 commits into
mainfrom
codex/sqlite-apns-registrations

Conversation

@steipete

Copy link
Copy Markdown
Contributor

Closes #108535

What Problem This Solves

Existing APNs registrations live in a JSON sidecar even though they are durable runtime state. Whole-file rewrites have only process-local serialization, and simply deleting or continuing to read that store would either lose shipped iOS push registrations or leave two competing authorities that can resurrect invalidated devices.

Why This Change Was Made

APNs registration reads, writes, batch lookup, and compare-and-delete now use the shared Kysely-backed SQLite state database. A strict, exclusive Doctor migration streams the retired JSON store without a whole-file memory limit, preserves canonical SQLite rows, honors deletion tombstones, records and verifies a transactional receipt, and removes the legacy source only after the import commits.

The APNs delivery/authentication module remains separate from the new storage owner. Runtime code has no JSON fallback, dual-write, or lazy import path; legacy handling exists only under openclaw doctor --fix.

User Impact

Gateways keep direct and relay APNs registrations across upgrade and restart while moving to one transactional state authority. Invalidated registrations stay deleted even if an old or interrupted JSON source reappears. Existing installations run openclaw doctor --fix to import the shipped file before the SQLite-only runtime uses those registrations.

Evidence

  • Focused APNs/store/Doctor/gateway/schema/tooling proof: 656 tests passed, 1 skipped.
  • Upgrade coverage includes direct and relay shapes, concurrent canonical writes, deletion tombstones, transaction rollback, interrupted claims, receipt-only cleanup retries, source mutation, invalid UTF-8, symlink/hardlink rejection, and a valid 2,500-record store larger than 4 MiB.
  • Generated Kysely schema verification passed.
  • Database-first legacy-store guard passed.
  • Targeted Oxlint and diff checks passed.
  • Mandatory autoreview: no accepted/actionable findings; confidence 0.91.
  • Exact-head hosted CI will be linked before merge.

Implementation and review used AI assistance.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation scripts Repository scripts commands Command implementations size: XL maintainer Maintainer-authored PR labels Jul 16, 2026
@steipete
steipete force-pushed the codex/sqlite-apns-registrations branch from b37d306 to fef6063 Compare July 16, 2026 02:18
@clawsweeper

clawsweeper Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 15, 2026, 10:35 PM ET / July 16, 2026, 02:35 UTC.

Summary
Moves direct and relay APNs registrations from a JSON sidecar into shared Kysely-backed SQLite, adding tombstones and an explicit Doctor migration for existing state.

PR surface: Source +1144, Tests +545, Docs +6, Generated +13, Other 0. Total +1708 across 19 files.

Reproducibility: yes. from source and documented workflow: replace the package through the supported manual npm, pnpm, or Bun path, leave push/apns-registrations.json in place, and restart without doctor --fix; the PR runtime then reads only SQLite.

Review metrics: 2 noteworthy metrics.

  • Persistent-state surfaces: 1 legacy JSON store retired; 1 tombstone table and 1 column added. The patch changes durable registration and deletion authority, so upgrade behavior matters beyond ordinary refactor validation.
  • Upgrade gates in patch: 1 explicit Doctor migration; 0 manual-package startup gates. The supported direct package replacement path can reach the new runtime without invoking the only migration entry point.

Stored data model
Persistent data-model change detected: database schema: src/state/openclaw-state-db.test.ts, database schema: src/state/openclaw-state-schema.generated.ts, database schema: src/state/openclaw-state-schema.sql, migration/backfill/repair: src/commands/doctor.e2e-harness.ts, migration/backfill/repair: src/infra/state-migrations.apns.test.ts, migration/backfill/repair: src/infra/state-migrations.apns.ts, and 14 more. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #108535
Summary: This PR is the implementation candidate for the linked APNs shared-SQLite migration issue.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Integrate the migration or a fail-closed repair gate into every supported update/startup path.
  • Show a redacted installed upgrade transcript for both direct and relay registrations, including successful post-upgrade lookup or push behavior.

Risk before merge

  • [P1] A documented direct npm, pnpm, Bun, or recovery-installer upgrade can restart the Gateway with the legacy APNs file still present while all runtime reads use an empty SQLite table, suppressing push tests, background wakes, and approval delivery until the device reconnects or the operator discovers openclaw doctor --fix.
  • [P1] This changes persistent secret-bearing state and deletion authority; merge should include installed-upgrade proof showing both legacy direct and relay registrations survive the actual supported update/startup path.

Maintainer options:

  1. Guard every upgrade path (recommended)
    Run the APNs migration before restart or block Gateway readiness while an unreceipted legacy store remains, with focused direct-package upgrade coverage.
  2. Require an explicit breaking upgrade step
    Maintain the SQLite-only runtime but document and visibly enforce Doctor repair as a mandatory prerequisite for every upgrade mechanism.
  3. Pause the migration
    Keep the shipped JSON runtime until a unified startup-safe migration boundary is available rather than landing a partially integrated authority switch.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Ensure direct npm, pnpm, Bun, installer, and managed updates cannot restart into SQLite-only APNs reads while an unreceipted legacy JSON store remains: migrate before restart or fail Gateway readiness with actionable `openclaw doctor --fix` guidance, and add focused direct and relay upgrade tests.

Next step before merge

  • [P1] Add the missing supported-upgrade/startup guard, then rerun focused migration and update-path validation on the exact PR head.

Security
Cleared: No dependency, workflow, permission, or execution-source expansion was found; secret-bearing legacy data is confined to a Doctor migration with transactional receipt and link-rejection coverage.

Review findings

  • [P1] Guard upgrades before switching APNs reads to SQLite — src/infra/push-apns.ts:32-38
Review details

Best possible solution:

Make every supported upgrade converge APNs state before Gateway readiness: run the migration in managed upgrade finalization and fail startup with actionable openclaw doctor --fix guidance whenever an unreceipted legacy APNs store remains.

Do we have a high-confidence way to reproduce the issue?

Yes from source and documented workflow: replace the package through the supported manual npm, pnpm, or Bun path, leave push/apns-registrations.json in place, and restart without doctor --fix; the PR runtime then reads only SQLite.

Is this the best way to solve the issue?

No. The database-first store and Doctor importer are appropriate, but retiring the runtime reader is safe only after every supported upgrade path migrates first or startup fails closed with an actionable repair path.

Full review comments:

  • [P1] Guard upgrades before switching APNs reads to SQLite — src/infra/push-apns.ts:32-38
    Current update docs support direct npm, pnpm, or Bun replacement followed by Gateway restart, but these exports make SQLite the sole runtime authority while the legacy import runs only through explicit Doctor repair. That path starts with an empty apns_registrations table and silently disables push tests and wake/approval delivery until the phone reconnects or the operator discovers doctor --fix. Run migration in every supported update finalizer or refuse Gateway readiness with actionable repair guidance while an unreceipted legacy file exists.
    Confidence: 0.94

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 154d1277e12e.

Label changes

Label changes:

  • add P1: A supported upgrade path can immediately break iOS push and wake delivery for existing paired devices.
  • add merge-risk: 🚨 compatibility: Existing installations can restart with shipped JSON registrations stranded outside the new SQLite-only runtime.
  • add merge-risk: 🚨 message-delivery: Stranded registrations suppress APNs push tests, wake nudges, and background approval or reconnect delivery.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.

Label justifications:

  • P1: A supported upgrade path can immediately break iOS push and wake delivery for existing paired devices.
  • merge-risk: 🚨 compatibility: Existing installations can restart with shipped JSON registrations stranded outside the new SQLite-only runtime.
  • merge-risk: 🚨 message-delivery: Stranded registrations suppress APNs push tests, wake nudges, and background approval or reconnect delivery.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
Evidence reviewed

PR surface:

Source +1144, Tests +545, Docs +6, Generated +13, Other 0. Total +1708 across 19 files.

View PR surface stats
Area Files Added Removed Net
Source 9 1599 455 +1144
Tests 5 766 221 +545
Docs 1 13 7 +6
Config 0 0 0 0
Generated 2 13 0 +13
Other 2 2 2 0
Total 19 2393 685 +1708

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs src/infra/state-migrations.apns.test.ts.
  • [P1] node scripts/run-vitest.mjs src/infra/push-apns.store.test.ts.
  • [P1] node scripts/run-vitest.mjs src/cli/update-cli/update-command.test.ts.
  • [P1] node scripts/check-changed.mjs -- src/infra/push-apns.ts src/infra/push-apns-store.ts src/infra/state-migrations.apns.ts src/infra/state-migrations.apns.test.ts src/cli/update-cli/update-command.test.ts docs/install/updating.md.

What I checked:

Likely related people:

  • mbelinky: Authored and merged the original APNs registration work and adjacent push-test and disconnected-node wake flows, making them the strongest history-based routing candidate for registration continuity. (role: introduced behavior and feature owner; confidence: high; commits: c2d12b7; files: src/infra/push-apns.ts, apps/ios/Sources/OpenClawApp.swift)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete force-pushed the codex/sqlite-apns-registrations branch from 4b91c26 to 22edde7 Compare July 16, 2026 08:07
@steipete
steipete merged commit 723d12f into main Jul 16, 2026
181 of 183 checks passed
@steipete
steipete deleted the codex/sqlite-apns-registrations branch July 16, 2026 08:52
@steipete

Copy link
Copy Markdown
Contributor Author

Merged via squash.

steipete added a commit that referenced this pull request Jul 16, 2026
The canonical schema gate rejects non-STRICT tables, failing every
gateway-suite shard since #108543 added the tombstones table without
STRICT. Align it with every sibling table and regenerate the schema
mirror.
xydt-tanshanshan added a commit to xydt-tanshanshan/openclaw that referenced this pull request Jul 17, 2026
…nsistent SQLite-only path

- Remove sqliteReadReady flag and markSqliteReadReady() — SQLite reads
  and writes are now unified: when stateDb is available, both use SQLite
- Remove useSqliteForReads getter — replaced with direct this.stateDb check
- createStore() now only catches the specific 'SQLite support unavailable'
  error (permanent Node version limitation); other errors propagate
- Doctor migration no longer calls markSqliteReadReady() — unnecessary
  when reads and writes are consistent

Follows the established APNs SQLite migration pattern (PR openclaw#108543):
- SQLite-only runtime when state DB is available
- Doctor-only legacy file import (no runtime fallback to filesystem)
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 17, 2026
* refactor(apns): move registrations to SQLite

* fix(apns): satisfy strict migration typing

* chore(apns): remove obsolete lint suppression

* chore: prune APNs max-lines baseline

* refactor(apns): keep store internals private

* fix(apns): preserve stored relay origins

* fix(apns): advance versions across deletion

* fix(apns): harden legacy state import

* fix(apns): redact legacy import errors

* fix(apns): reserve version successor

* fix(apns): bound legacy timestamp imports

* refactor(apns): extract payload builders
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 17, 2026
The canonical schema gate rejects non-STRICT tables, failing every
gateway-suite shard since openclaw#108543 added the tombstones table without
STRICT. Align it with every sibling table and regenerate the schema
mirror.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations docs Improvements or additions to documentation maintainer Maintainer-authored PR scripts Repository scripts size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move APNs registration state to shared SQLite

1 participant