Skip to content

feat(v1.0.0): close watch_table + subscribe_to_channel smoke gaps + ADR-0016 - #18

Merged
0xquinto merged 7 commits into
mainfrom
feat/v1.0.0-ship-surface
May 2, 2026
Merged

0xquinto merged 7 commits into
mainfrom
feat/v1.0.0-ship-surface

Conversation

@0xquinto

@0xquinto 0xquinto commented May 2, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the deferred-from-ADR-0015 verification gaps for v1.0.0 ship surface. Six commits, four PASS receipts:

  • Spike (eval/spike-edge-warmup.ts) — 20-trial Edge warm-up distribution: 20/20 delivered, p99 5322ms, recommended smoke wall budget 12s
  • Probe (eval/probe-edge-payload.ts) — diagnostic that pinned the GRANT + RLS chain root cause for new: {} + 401 substrate finding
  • watch_table E2E smoke — 5/5 PASS in 10.8s; case wall 638ms (warm) / 5252ms (post-warm-up)
  • subscribe_to_channel E2E smoke — 6/6 PASS in 19.0s; case wall 3084ms
  • MCP SDK floor bumped to ^1.27.1 (onerror fix)
  • ADR-0016 filed as Proposed

Manifest v2.0.0 / n=300 stays a separate PR per recon Decision 4. v1.0.0 tag waits on both.

Two non-trivial substrate findings (spike-surfaced)

  1. Single-INSERT@+100ms unsafe on Edge — per-request subscribe handshake exceeds 100ms; INSERTs land pre-SUBSCRIBED. Multi-INSERT schedule (+100ms / +5s / +10s) is the v1.0 pattern.

  2. Realtime row payload requires GRANT + RLS chain for anon JWT consumers — first decoded via 7-variant probe. Without the chain, events.length === 1 passes vacuously while new: {} carries errors: ["Error 401: Unauthorized"]. The chain: enable row level security + create policy ... using (true) + grant select on ... to anon, authenticated, service_role. Standard Supabase contract; not a bug in the skill — but invisible until the spike measured it. Documented in docs/spike-findings.md § T7-Edge sub-finding 2.

Both findings shape the smoke design directly. Without the spike, the smoke would have shipped with single-INSERT + count-only assertions and silently flaked or silently passed-with-empty-rows in consumer hands.

Test plan

  • bun run typecheck — passes
  • bun run lint — passes
  • bun run test:fast — 49/49 pass
  • bun run vitest run tests/smoke/edge-deploy.smoke.test.ts — 6/6 PASS (2 new it() blocks + 4 from ADR-0015)
  • bun run eval/spike-edge-warmup.ts — n=20 with multi-INSERT + GRANT+RLS chain; 20/20 delivered, populated new
  • Reviewer: confirm ADR-0016 framing / receipt completeness before promoting to Accepted

Operator next steps

  1. Review ADR-0016 § Decisions + § Empirical refinement; promote Proposed → Accepted if comfortable.
  2. Decide whether to ship this PR as 0.3.0 standalone (manifest blocks) or hold for the manifest PR (v1.0.0 within same calendar week).
  3. CLAUDE.md currency drift sweep is now MORE overdue (16 ADRs filed, plus the v0.1.x → v0.2.0 → 0.3.0 candidate). Worth its own follow-up PR per the prior handoff.

🤖 Generated with Claude Code

0xquinto and others added 6 commits May 2, 2026 16:38
20-trial spike against the deployed Edge Function: 20/20 delivered, p99
5.49s wall, recommended smoke wall budget 12s. Multi-INSERT schedule
(+100ms / +5s / +10s) is mandatory — first-pass single-INSERT@+100ms
saw 17/20 timeouts (per-request subscribe handshake exceeds 100ms, so
INSERT lands pre-SUBSCRIBED).

Two sub-findings worth their own follow-ups:
- new-payload arrives null on Edge (likely replica-identity-related;
  schema permits null but consumers expect row data — references/
  revision pending)
- anon-JWT path unverified; ADR-0016 smoke should test consumer-shaped
  auth (anon + permissive RLS policy) not just service_role

EVAL_HOST_DB_URL added to .env.example for the spike's host-direct DDL.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Diagnostic probe (eval/probe-edge-payload.ts) decomposed the T7-Edge
'new: {}' + 401 finding across 7 variants × auth/grant/RLS combinations:

  C bare service_role     → events=1, new: {}, errors: 401
  D + GRANT               → events=1, new: POPULATED, errors: null
  E + GRANT + RLS policy  → events=1, new: POPULATED
  F + GRANT under anon    → events=0 (anon needs RLS for delivery)
  G + GRANT + RLS anon    → events=1, new: POPULATED — consumer path

Root cause: Realtime broker authorizes the row payload separately from
PostgREST. service_role + GRANT alone reveals row data; anon needs
RLS-enabled with a permissive 'select using (true)' policy on top.

Spike updated to apply the chain after CREATE TABLE. Re-run shows
20/20 delivered with populated new, p99 5322ms (within margin of
first run's 5490ms), recommendation unchanged at 12s wall budget.
RLS evaluation adds ~100-200ms per event, shifting bucket
distribution (n0 65% → 30%) but not the p99.

Findings appended to docs/spike-findings.md § T7-Edge sub-finding 2:
explicit table of probe variants + consumer-shaped chain that
ADR-0016's smoke must apply.

Methodology update for ADR-0016: smoke uses anon JWT, applies full
RLS + policy + GRANT chain on temp table, asserts populated `new`
content (not just events.length === 1) to catch chain regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Closes the deferred-from-ADR-0015 watch_table verification gap. Smoke
extends tests/smoke/edge-deploy.smoke.test.ts with a fifth it() block:

  - Creates a fresh temp table on the host project (gated on
    EVAL_HOST_DB_URL; smoke skips cleanly if absent)
  - Applies the GRANT + RLS chain pinned by spike's probe
    (ENABLE RLS + permissive select policy + GRANT) — without this,
    Realtime broker 401-strips the row payload silently
  - Dispatches JSON-RPC tools/call watch_table with timeout_ms 12_000
    (sized from spike p99 5322ms × 2.25 headroom)
  - Fires multi-INSERT schedule (+100ms / +5s / +10s) — single-shot
    +100ms INSERTs land pre-SUBSCRIBED and get dropped
  - Asserts events.length === 1 + closed_reason: max_events +
    populated `new.n` content (regression check for chain integrity)

PASS receipt: 5/5 smoke pass against live deploy in 10.8s total
runtime; watch_table case wall 638ms (warm path).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…0016 step 2)

Closes the second deferred-from-ADR-0015 verification gap. Smoke
extends edge-deploy.smoke.test.ts with a sixth it() block exercising
end-to-end fan-out:

  - External raw createClient sender on a public channel (separate
    from the function's per-request listener — CLAUDE.md
    "single-client channel dedup" foot-gun)
  - tools/call subscribe_to_channel against the deployed function
    with timeout_ms 10s, max_events 1
  - Multi-broadcast schedule (+500ms / +3s / +6s) — single-shot
    +500ms broadcasts race the function's per-request subscribe
    handshake (~700ms-5s per spike T7-Edge); first one to land
    post-SUBSCRIBED wins
  - Asserts broadcasts.length === 1 + closed_reason: max_events +
    payload.from === "external-sender"

PASS receipt: 6/6 smoke pass against live deploy in 19s total
runtime; subscribe_to_channel case wall 3084ms (the +3s broadcast
landed first).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Captures WebStandardStreamableHTTPServerTransport.onerror fix
(modelcontextprotocol/typescript-sdk#1580, merged 2026-02-24) — without
the floor, the deployed function silently swallows transport-layer
errors before they reach the onerror callback.

Lock regen resolves to 1.29.0 (within range; latest stable). No
behavior change in the currently-deployed bundle — this widens the
contract declaration, doesn't tighten the runtime. Redeploy not
strictly required; recommended on next major bump for source-tree-
vs-deployed alignment per ADR-0015.

Recon: docs/recon/2026-05-02-v1.0.0-ship-surface-recon.md § Decision 5.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Mirrors ADR-0015's six-decisions structure. Status: Proposed pending
operator review of the PASS receipts. Captures:

- Adopt v1.0.0 ship event = bundled-evidence completeness
- Add watch_table E2E smoke (PASS receipt: 5/5 in 10.8s)
- Add subscribe_to_channel E2E smoke (PASS receipt: 6/6 in 19.0s)
- Floor MCP SDK at ^1.27.1
- Defer manifest v2.0.0 to ADR-0017 / ADR-0007 amendment
- Frame as 'ship surface' not 'completeness'

Empirical refinement section captures two non-trivial substrate
findings the recon's adversarial pass made room for but didn't
predict in detail:

  1. Single-INSERT@+100ms unsafe on Edge (per-request subscribe
     handshake exceeds 100ms; multi-INSERT schedule mandatory)
  2. Realtime row payload requires GRANT + RLS chain for anon JWT
     consumers — root-cause-pinned via 7-variant probe

Both findings have receipts in spike-findings § T7-Edge; both shape
the smoke design directly (12s wall budget, multi-INSERT, full
GRANT+RLS chain in temp table setup).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
PR #18 self-review surfaced four issues before merge:

1. SDK floor regression gate was missing — `package.json` carried `^1.27.1`
   but no test would catch a future regex regression. Added
   `tests/fast/sdk-floor.test.ts` reading the resolved SDK version via
   `require.resolve("@modelcontextprotocol/sdk/server")` walked back to
   `package.json` (the SDK doesn't expose package.json via `exports`).
   Closes the load-bearing gap behind ADR-0016 step 3.

2. Substrate-drift visibility — watch_table smoke logged `wall=...ms` only.
   If Edge cold-start widens past +5s and the smoke starts habitually
   delivering n=2 instead of n=0, the smoke still passes silently. Added
   `delivered_n=${event?.new?.n}` to the smoke log so operator can spot
   the shift in the CI run output.

3. Probe matrix collapsed to F+G during the live debugging session — the
   trail-of-evidence value was the full C/D/E/F/G walk that pinned
   GRANT+RLS as the root cause. Restored all five variants with comments
   explaining each row's contribution. Asymmetric-key variant intentionally
   excluded (1002 protocol error is a separate Realtime-broker issue,
   documented in spike-findings § T7-Edge).

4. ADR-0016 § Implementation status said `Cost: ~$0 incremental` —
   technically true but understated. Reframed to `no metered cost (Pro
   project covers compute); ~5 min host-project DDL/INSERT time per
   smoke run`, which is what the operator actually pays.

Verification: bun run typecheck + bun run lint clean; 50 fast tests pass
(was 49; +1 from the new sdk-floor regression gate).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@0xquinto
0xquinto merged commit 0a7c2b5 into main May 2, 2026
2 checks passed
@0xquinto
0xquinto deleted the feat/v1.0.0-ship-surface branch May 2, 2026 23:04
0xquinto added a commit that referenced this pull request May 3, 2026
* docs(adr-0016): promote to Accepted on green post-merge smoke

Smoke receipt against deployed Edge Function on `main` (post-PR-#18 merge):
- watch_table E2E:       wall=5442ms delivered_n=1   (45% headroom)
- subscribe_to_channel:  wall=3110ms                 (under 10s budget)
- 4 existing tool smokes: all green
- 6/6 PASS, 22.0s total wall

n=1 (the +5s INSERT landed first, not n=0 at +100ms) is consistent with
the spike's cold-start handshake distribution — the warm-up window dropped
the +100ms INSERT exactly as predicted. One-off; if it becomes habitual
the budget needs a fresh spike.

ADR-0016's Status flips Proposed → Accepted; receipts table updated to
record both the pre-merge (feat branch) and post-merge (main) runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(adr-0016): commit smoke receipt log + cite main SHA

Self-review #3 fix: the post-merge smoke receipt cited in ADR-0016 was
ephemeral — by next month the "6/6 PASS, 22.0s wall" claim would be
unfalsifiable. Now backed by:

- `logs/smoke-edge-deploy/2026-05-03-postmerge.txt` — full vitest output
  with main SHA, host project ref, command used, and timestamps.
- `.gitignore` carve-out for `logs/smoke-edge-deploy/` (mirrors existing
  spike-latency + spike-edge-warmup pattern).
- ADR-0016 receipt section now cites SHA `0a7c2b5` and links the log file
  + `references/edge-deployment.md` for host-project provenance.

Self-review #4 fix (rolled in): host project ref `uetkiogzuwhzshczaajr`
now framed as `EVAL_HOST_PROJECT_REF` env-var binding, not a bare ref —
future readers can find the canonical host without grep.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
0xquinto added a commit that referenced this pull request May 3, 2026
…ns post-v0.3.0 (#26)

* docs(claude.md): currency sweep post-v0.3.0 ship

Drift accumulated since v0.2.0 closed in CLAUDE.md status block. This
sweeps four lines that became inaccurate after the v0.3.0 ship loop
(PRs #21, #23, #24, #25):

- L19: `49 offline tests` → `50 offline tests` (sdk-floor regression
  gate added in PR #18 / ADR-0016).
- L208: `v0.2.0 shipped (worked-example bundle, ADR-0014)` →
  `v0.3.0 shipped (E2E smoke surface, ADR-0016 — fully verified
  end-to-end on 0.3.0 bytes after Edge redeploy on 2026-05-03)`.
- L210: npm publish list adds `v0.3.0`; ADR-outcome census adds 0016
  to the accept group + the predicted-and-empirically-refined group
  (matches 0013 + 0015's dual-category shape — chain root cause emerged
  from spike+probe and was folded back into the ADR), removes the
  proposed-pending-receipts category since 0016 is now Accepted. Six
  shapes → five.
- L212: `49 fast tests` → `50 fast tests` (matches the test:fast line
  bump above).
- L214-218: operator follow-ups — drops the now-completed "Promote
  ADR-0016 from Proposed → Accepted" item (PR #21); reflows the manifest
  / v1.0.0 narrative to point at the recon's fallback path that
  actually played out (smoke shipped as 0.3.0, manifest deferred);
  reframes EVAL_HOST_DB_URL from "New" to "required" (it's not new
  anymore).

No code changes. Mirrors the b5078a0 + 043cc75 sweep-PR pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(claude.md): drift fixes from claude-md-improver audit

Three additional drift items the claude-md-improver skill surfaced. Same
audit category as the v0.3.0 currency sweep already on this branch, so
folding them in rather than opening a separate PR.

1. Smoke test count unit mismatch — was '8 online smoke tests' (which is
   actually files); fast block uses individual-test count. Now both
   match the same unit: '13 online smoke tests across 8 files'.

2. Repo layout row for `docs/ship-status.md` claimed '5 follow-ups
   awaiting operator action' — the file is dated 2026-04-30, references
   the v0.1.0 build, and has zero numbered follow-ups currently. The
   Status block at the end of CLAUDE.md is the actual source of truth
   now. Reframed as 'v0.1.0-era ship snapshot (historical)' with a
   pointer to the live source.

3. 'Where to put new info' table routed 'New design' and 'Implementation
   plan' to `docs/superpowers/specs/` and `docs/superpowers/plans/` —
   neither directory exists in this repo. Holdover from supabase-mcp-evals
   upstream that wasn't carried over. Replaced with the pattern that
   actually runs here: `docs/recon/` for pre-ADR research, `docs/decisions/`
   for the ADR itself. Also dedupes the existing 'Architecture decision'
   row that the new 'Architecture decision' supersedes.

Verified with file-system check: `docs/superpowers` confirmed missing,
13/8 smoke counts confirmed via `bun run vitest run --project smoke`.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(claude.md): three lived-from-this-session additions

Captures the trips this session's v0.3.0 ship loop made me rediscover.
Same audit category as the existing two commits on this branch (drift
fixes / hard-won-from-this-session); folding rather than splitting the PR.

1. Quick start now carries the release-loop one-liner: `git tag` →
   `gh run watch` → `npm view`. Without this the next ship has to
   rediscover the workflow shape.

2. New convention `Release-PR vs Edge-redeploy-PR are always two separate
   PRs`: the deno.json import range bump cannot ship in the same PR as
   the package.json version bump — chicken-and-egg (the new npm version
   doesn't exist until the tag fires the publish workflow, so
   `deno cache --reload` fails at lock-regen). PR #22 hit this live.
   Pattern documented as a 5-step shape with the v0.3.0 loop (PRs #23 /
   #24 / #25) as the reference.

3. New convention `npm tag judgment ≠ project milestone label`: ADRs
   sometimes label a milestone "v1.0.0 ship surface" (project narrative)
   but that label does NOT determine the npm tag. Cite ADR-0016 recon's
   explicit fallback: "v1.0.0 = both [smoke + manifest n=300] same week,
   otherwise smoke PR is 0.3.0 and v1.0.0 stays unclaimed." Caught
   drafting against wrong branch of recon decision once already.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(claude.md): self-review polish — release ritual delimiter + cross-ref to deno-lock-regen

Two fixes from the PR #26 self-review:

#2: Release loop fence in Quick start was mixing two abstraction levels
(normal dev commands vs post-merge release ritual). Pulled it out into
its own subsection 'Release ritual (run only after a release PR merges
to main)' with explicit eligibility framing in the heading. Also added a
pointer to the Edge-redeploy convention since the ritual is incomplete
without that follow-up step.

#1: 'Release-PR vs Edge-redeploy-PR' convention referenced 'deno cache
--reload' but didn't tell the reader to actually run the lock-regen
sequence (rm -f deno.lock + reload + check). Now explicitly cross-refs
the 'Deno lock regen' convention above with 'composes with §' framing.
Plus retargeted step 2 to point at the new 'Release ritual' subsection
rather than a vague 'merge → tag → publish' phrase.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant