Skip to content

fix(shared): checkpoint WAL without blocking foreground writes - #1000

Merged
JustYannicc merged 8 commits into
mainfrom
fix/779-background-wal-admission
Oct 17, 2026
Merged

JustYannicc merged 8 commits into
mainfrom
fix/779-background-wal-admission

Conversation

@JustYannicc

@JustYannicc JustYannicc commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Problem

A foreground SQLite write can checkpoint the WAL while the server handles a request. The #779 reproduction held about 16,000 pending frames and stalled a small delete for 5.48 seconds.

Change

Run checkpoints on a worker and pause new writes when it falls behind. The writer acknowledges a pause before the worker takes the exclusive truncation lock. Final admission and synchronous SQL execution share one callback, and worker generations prevent stale replies from reopening the gate.

Each statement holds its selected connection permit through execution. Transactions retain that permit through completion. Waiting statements, transactions and reserved connections honor the caller's cancellation mask. Ordinary waits can cancel, while masked cleanup and claimed webhook delivery finish. Cancellation hands an elected wake to the next writer and cannot leave a claimed permit without cleanup. Reads and active transaction completion stay available.

Mutating pragmas and partially committed statement failures update checkpoint accounting. Unsupported SQLite versions or worker failure retain the existing writer-checkpoint fallback.

The 10,000-frame threshold bounds admission between statements and transactions. A single statement or transaction can cross it, and an external reader can hold the write gate closed until its snapshot ends. This affects the shared Node SQLite adapter used by the app server. Cloud Durable Objects use a separate store.

Scope and approval

Addresses item 2 of #779. The other slowdown fixes and database-growth retention work have separate PRs.

Verification

  • Fifteen focused concurrency and cancellation regressions pass, including both reserved-connection wake cancellation windows, cancellation while a transaction waits for the writer permit, cancellation at permit grant, and masked cleanup at both permit and checkpoint waits.
  • The full adapter file has 51 passing tests and one large-blob binding failure, where the length returns null instead of 65537. The same failure was reproduced on the pinned base with the same Node version and dependencies. No assertion was weakened or skipped. This failure is local to the macOS verification environment: main CI at ee1f042a passed all 31 adapter tests on Node 24.22.0, including keeps large text and blob parameters off cached statements. The precise local cause has not been isolated.
  • Four server cases failed in the first CI run because forced interruptibility overrode deliberate caller masks. The two fast cases failed on the first candidate and passed on the pinned PR base. All four now pass on that base with only the fixed adapter files copied in: tracked setup cleanup, canceled Codex and Claude delegated setup, and delivery after a webhook sender disconnects. Application code is unchanged.
  • Targeted formatting, lint and shared-package typecheck pass.
  • Synthetic fixtures cover acknowledged truncation, final admission, stale worker replies, event-loop progress, read routing, transaction completion, partial writes and mutating pragmas, worker failure, and reclamation after an external reader releases its snapshot. Live app performance and the patched desktop build remain unverified.

Independent review: GPT 6.1 Sol xhigh (Opus unavailable: rate-limited at 2026-10-17 15:47:05 UTC). Candidate 34bf59676ffcddd3b717071635383872a3fd6498 passed independent standards and input reviews with no findings, including the caller-mask correction after the first CI run.

Implemented with GPT 6.0 Luna max; integrated and reviewed with GPT 6.1 Sol through the Codex harness in T3 Code.

Canceled elected wakes could strand a later reserved SQL statement after WAL admission reopened. Keep one waiter elected through its retry's final synchronous gate check, hand cancellation to the next eligible waiter, and make only transaction and reserve permit acquisition interruptible.

Focused cancellation, ownership, fairness, BEGIN, and admission tests pass (11 tests). The full adapter has 47 passing tests and one blob failure matching the pinned-base result (expected 65537, got null). Scoped format, lint, and shared typecheck pass.

Implemented with GPT-6 Luna max in the T3 Code Codex harness.

This branch was previously deployed

1 inactive deployment
cloud-preview — 34bf5967 Deployed Oct 17, 2026 by JustYannicc via Delete preview cloud #35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant