Skip to content

feat(MUN-0021): transactional outbox, local lease relay, and CommittedResultRefV0 adoption gate - #13

Merged
Arcanada merged 8 commits into
mainfrom
feat/MUN-0021-outbox-delivery
Jul 31, 2026
Merged

Arcanada merged 8 commits into
mainfrom
feat/MUN-0021-outbox-delivery

Conversation

@Arcanada

Copy link
Copy Markdown
Contributor

What

Implements ARCA-0194 WP-02 on the MUN-0020 execution-authority seam, and closes the MUN-0021 adoption gate ratified in datarim/research/ARCA-0194/muneral-result-reference-consilium.md.

Two slices:

  1. Transactional outbox + stoppable local lease relay — typed outbox events committed atomically with authority transitions, durable consumer inbox/deduplication, fenced lease bookkeeping, append-only delivery-attempt and quarantine evidence, deterministic reconciliation readback.
  2. Muneral-owned CommittedResultRefV0 — the adoption gate. The consilium ruled the prior result contract NO-GO and allowed MUN-0022 to stay a pure contract proof only on the condition that the additive Muneral relation lands here.

The result-authority contract

Muneral authors the committed-result reference and the deterministic completion receipt in the same transaction as the accepted node mutation, transition journal, aggregate update and outbox fact. An execution adapter proposes an OwnedResultMutationV0 and relays the already-authored receipt — it can never fabricate a receipt or supply an authoritative digest.

The instruction, projection and result integrity domains stay distinct:

cardDigest       = sha256("task-card-v0\0"                   + canonicalCardBytes)
projectionDigest = sha256("task-card-projection-v0\0"        + canonicalProjectionBytes)
resultDigest     = sha256("task-card-result-node-v0\0"       + canonicalCommittedNodeBytes)
resultRefId      = sha256("muneral-result-ref-v0\0"          + canonicalReferenceFields)
receiptId        = sha256("assembly-completion-receipt-v0\0" + canonicalReceiptFields)

The rejected legacy proposal receipt.digest == projection.cardDigest fails closed as an unknown field on a closed schema, and a database CHECK refuses any row whose result, reference or receipt digest equals an instruction digest. A receipt bound to instructions could otherwise claim success before any result exists.

Durable shape

Additive, forward-only, no backfill, no existing-row rewrite.

  • task_result_nodes — the accepted owned node mutation, append-only
  • task_committed_result_refs — append-only reference plus receipt identity; unique on transition identity, (task, attempt, card, node, node version), (task, mutation) and receipt id; every foreign key ON DELETE RESTRICT

Both relations carry SECURITY DEFINER append-only triggers with search_path = pg_catalog, revoked from PUBLIC. rollback.sql refuses to execute.

Round-5 boundary

Preserved and re-proved. No fleet registry, lifecycle, placement, staged rollout, watchdog, telemetry aggregation, or direct command routing. No NestJS module, controller, provider registration, HTTP endpoint, broker, or runtime wiring. Disabled by default.

Per-message closed validators replace the global wrong-plane denylist for the result plane, exactly as ratified: a valid Task Card nodeId is accepted, while Supervisor lifecycle, placement, watchdog and command-authority fields remain rejected — and Supervisor principals are now rejected too, a check that did not previously exist.

Verification

Contract tests were written RED first, against a module that did not exist, and confirmed failing before implementation.

Suite Tests Result
result-authority.contract.spec.ts 52 ✅
result-authority.migration.spec.ts 26 ✅
result-authority.postgres.spec.ts (disposable PG:16) 16 ✅
outbox.relay.spec.ts 60 ✅
outbox.migration.spec.ts 65 ✅
outbox.relay.postgres.spec.ts (disposable PG:16) 29 ✅
Full suite 518 489 pass, 21 skip, 8 fail

The 8 failures are the pre-existing DATABASE_URL-dependent e2e specs (app-module-boot.e2e, field-changes.e2e), attributed to MUN-0038. Verified cause: DATABASE_URL environment variable is not set. They are reported as failing, not represented as green.

Static: tsc --noEmit clean, prisma validate valid, eslint src 0 errors / 36 pre-existing warnings, npm run build clean.

Two defects found only by the real-database run

Neither was visible to the mocked contract tests:

  1. The MUN-0020 reducer permits attempt:succeeded only from a running attempt. The seam accepted issued, leaking an InvalidTransitionError instead of a typed refusal.
  2. A concurrent loser collided on the aggregate version and StaleVersionError escaped as an untyped throw. It is now caught and returned as a ResultBindingError, matching the P2002 path.

Negative-control rehearsal (never committed)

Control Weakened guard Named test observed failing Restore
NC1 receipt-identity recomputation F8: a receipt whose stated receiptId does not match its bytes fails closed SHA-256 exact-byte match ✅
NC2 task_committed_result_refs_domain_separation_check rejects a receipt or reference derived from an instruction digest SHA-256 exact-byte match ✅

Disposable-PostgreSQL discipline

Every empirical database proof runs against a uniquely named throwaway container on a dynamic loopback port. Cleanup is fail-closed: a surviving container, task-named volume, or listening port fails the suite. No shared or production database is reachable from the test path. The bootstrap and its cleanup contract now live in test/support/disposable-postgres.ts, shared by both PostgreSQL suites so one copy of the guarantee exists rather than two that can drift.

Secret hygiene

Credential-shaped redaction fixtures are assembled at runtime rather than committed as literals. The redaction guard still sees complete, well-formed tokens, but no scanner-matching string enters the history.

Scope held

No distributed broker, no exactly-once claim, no new repository or deployable service, no provider/model/tool/Assembly invocation, no Datarim cutover, no secrets, no shared or production database access.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PTicw7at1VFMqzgj4LG7pX

valentov and others added 8 commits July 30, 2026 13:48
Add disabled-by-default PostgreSQL-local outbox relay that atomically
commits server-derived typed events with MUN-0020 authority transitions.

Schema (additive, no existing-row mutation):
- task_outbox_events: one immutable row per committed terminal transition
- outbox_leases: mutable fenced delivery bookkeeping (only mutable table)
- delivery_attempt_evidence: append-only delivery record
- quarantine_evidence: append-only poison record (unique per event)
- consumer_inbox: durable idempotency boundary per consumer

All foreign keys use RESTRICT (no silent cascade-delete). Four tables
have SECURITY DEFINER append-only triggers; outbox_leases is mutable.

Relay (plain class, no NestJS wiring):
- Poll: pending + expired leases, FIFO by recordedAt
- Lease: atomic UPDATE with fence token (holder + delivery_ordinal)
- Dispatch: wrong-plane rejection → fence check → lease expiry →
  inbox dedup → consumer invocation → deliver/quarantine
- Quarantine after maxRetries; no head-of-line blocking
- Crash prefix: relayId prefix in every lease holder
- Reconciliation: lease summary, orphans, stale leases, attempt counts

Wrong-plane guard: 34 forbidden fleet/supervisor top-level JSON keys
rejected before any consumer or database side effect, without importing
a Supervisor domain model. Muneral task facts only.

Forward-only rollback — evidence tables are immutable by design.
…on boundaries (MUN-0021)

Two reviewer findings fixed:

F1 — Lease acquisition reset failure_count=0 on every cycle, and consumer
failure inside dispatch()'s $transaction threw the original error, rolling
back the failure_count increment and delivery-attempt evidence. Poison
quarantine was unreachable.

F2 — Wrong-plane quarantine was recorded inside the same $transaction that
then threw WrongPlanePayloadError, rolling back quarantine evidence and
lease status update. Events were endlessly re-polled.

Changes:
- lease(): remove failure_count=0 reset from both raw SQL and fallback
  Prisma paths, preserving accumulated failure state across cycles.
- dispatch(): move wrong-plane validation BEFORE the main transaction;
  record quarantine in a separate durable mini-transaction via new
  recordWrongPlaneQuarantine() helper, then throw.
- dispatch() consumer-failure path: return 'expired' instead of throwing,
  so the transaction commits and failure_count + delivery-attempt evidence
  are durable. Consumer side effects were never committed (consumer threw
  inside the tx) — that is correct.

Tests:
- 7 new F1/F2 regression unit tests (outbox.relay.spec.ts, 38→41 total).
- 2 new PostgreSQL integration tests (outbox.relay.postgres.spec.ts,
  10→12 total) with real Prisma/disposable-DB proving failure_count
  accumulation across lease cycles and wrong-plane quarantine durability.

Co-Authored-By: Claude <[email protected]>
…-0021)

Close the MUN-0021 adoption gate ratified in the ARCA-0194 result-reference
consilium. MUN-0022 may remain a pure contract proof; the additive Muneral
relation and the authoritative commit seam are mandatory here.

Muneral now authors the committed-result reference and the deterministic
completion receipt in the same transaction as the accepted node mutation,
transition journal, aggregate update and outbox fact. An execution adapter
proposes an OwnedResultMutationV0 and relays the already-authored receipt; it
can never fabricate a receipt or supply an authoritative digest.

The instruction, projection and result integrity domains stay distinct:

  cardDigest       = sha256("task-card-v0\0"                   + card)
  projectionDigest = sha256("task-card-projection-v0\0"        + projection)
  resultDigest     = sha256("task-card-result-node-v0\0"       + node)
  resultRefId      = sha256("muneral-result-ref-v0\0"          + ref fields)
  receiptId        = sha256("assembly-completion-receipt-v0\0" + receipt fields)

The rejected legacy proposal `receipt.digest == projection.cardDigest` fails
closed as an unknown field, and a database CHECK refuses any row whose result,
reference or receipt digest equals an instruction digest.

Durable shape (additive, forward-only, no backfill):
- task_result_nodes — the accepted owned node mutation, append-only
- task_committed_result_refs — append-only reference plus receipt identity,
  unique on transition identity, (task, attempt, card, node, node version),
  (task, mutation) and receipt id, every FK ON DELETE RESTRICT

Per-message closed validators replace the global wrong-plane denylist for the
result plane. A valid Task Card nodeId is accepted; Supervisor lifecycle,
placement, watchdog and command-authority fields stay rejected, and Supervisor
principals are now rejected too.

Round-5 boundary unchanged: no fleet registry, lifecycle, placement, staged
rollout, watchdog, telemetry aggregation or direct command routing. No NestJS
module, controller, provider or runtime wiring.

Tests, written RED first: 52 contract, 26 migration, 16 disposable-PostgreSQL.
Two defects surfaced only by the real-database run and fixed: a result was
accepted from an `issued` attempt the reducer would refuse, and a concurrent
loser escaped as an untyped StaleVersionError instead of a typed refusal.

The disposable-PostgreSQL bootstrap and its fail-closed cleanup contract move
to test/support so both PostgreSQL suites share one implementation.

Full suite: 489 pass, 21 skip, 8 pre-existing DATABASE_URL e2e failures
attributed to MUN-0038.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PTicw7at1VFMqzgj4LG7pX
The disposable-PostgreSQL harness shelled out to `sudo docker`. SEC-0028
deliberately removed docker escalation from the ci-general runner, so both
database suites failed in CI with "sudo: a password is required" — 45 tests
that were only ever green on a developer machine.

The harness now resolves its provisioning mode at start():

  container — a uniquely named container on a dynamic loopback port, when
              docker is reachable (plain `docker` preferred, `sudo -n docker`
              as fallback)
  database  — a uniquely named database created on an already-ephemeral
              PostgreSQL server and dropped afterwards, when docker is not
              reachable but MUN0021_PG_BASE_URL or DATABASE_URL is. This is
              the CI path: the job already provisions a per-job postgres:16
              service.

Both modes stay task-owned, disposable and fail-closed. Database mode verifies
the scratch database is gone via pg_database and refuses to start against any
non-local host or a URL containing prod/rds/supabase/neon, so the create-and-
drop path cannot be aimed at a shared or production server.

When neither mode is available the harness throws. It never degrades to
"skipped but reported green" — an unrun database proof is not a passing one.

psql and pg_isready are no longer required: readiness, migration application
and cleanup all go through the `pg` client, now an explicit devDependency
rather than a transitive one. Migrations are applied over the simple query
protocol, which is what dollar-quoted trigger bodies and multi-statement files
need.

Verified locally in both modes: container 16/16, database 45/45 (both suites,
docker hidden behind a failing stub).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PTicw7at1VFMqzgj4LG7pX
…al PostgreSQL

QA Layer 3b found two operator expectations declared `evidence_type: empirical`
with no real-database proof behind them. Unit mocks cannot settle either one.

- `lease-reclaims-same-event`: a driven clock expires a lease, a second holder
  reclaims it, and the test asserts the same outbox/transition/attempt identity
  comes back while the authority fingerprint — transitions, attempts, outbox
  identity, aggregate versions — is byte-identical either side of the reclaim.
  Only the delivery ordinal and holder move; failure count stays 0.

- `poison-quarantine-no-hol`: a poison event and a healthy event belonging to
  two different task aggregates are interleaved through one relay and one
  consumer. The poison event exhausts its bounded retry budget and lands in
  quarantine with immutable evidence, while the unrelated aggregate is
  delivered and its inbox row written in the same run. No side effect leaks
  from the quarantined event.

Both relays poll the maximum batch window: earlier tests in the suite leave
pending rows behind, so a default batch made inclusion depend on seeding order
rather than on the predicate under test.

Outbox PostgreSQL suite: 31/31. Full suite: 491 pass, 21 skip, 8 pre-existing
DATABASE_URL e2e failures attributed to MUN-0038.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01PTicw7at1VFMqzgj4LG7pX
@Arcanada
Arcanada merged commit 6361621 into main Jul 31, 2026
4 checks passed
@Arcanada
Arcanada deleted the feat/MUN-0021-outbox-delivery branch August 12, 2026 10:09
Arcanada added a commit that referenced this pull request Sep 14, 2026
…ESM move dropped

Dependabot #103 proposes node 24.20.0 -> 26.8-alpine. Measured against
nodejs.org/dist/index.json: the 24.x line is the CURRENT LTS (Krypton, v24.21.0
released 2026-09-07) and every 26.x release carries `lts: false` — 26 is Current,
not LTS. Moving production off a supported line is not a dependency bump, so this
takes the security patch WITHIN the LTS line instead: 24.21.0 everywhere.

There is a concrete reason to take it. v24.21.0 fixes an out-of-bounds write in
`mkdtemp` for long prefixes (nodejs/node#64770), and this repository calls
`fs.mkdtempSync` in three places, one of them the evidence path itself
(mutation-harness.js gitSupplement). Five pins move together, because
regenerate-derived-artefacts.yml REFUSES a mismatch between ci.yml and
.github/workflows/.node-version-for-derived: apps/api/Dockerfile (x2), ci.yml,
that pin file, and the root engines field.

THE DEFENCE THAT WAS LOST. Two defects, both introduced by the ESM move in #113
and both invisible until a full mutation run was attempted:

1. The harness spawns jest directly and did not pass --experimental-vm-modules,
   so under ESM every battery failed with `SyntaxError: Cannot use import
   statement outside a module` and the run stopped at `baseline: FAIL`. Measured:
   20 suites / 394 tests pass with the flag, 20 suites fail with 0 tests without
   it. `pnpm test` sets the flag; the harness never did. It went unnoticed because
   the only caller of a full run is regenerate-derived-artefacts.yml, gated on
   author == 'dependabot[bot]' — it never ran for #113. NODE_OPTIONS is appended,
   not assigned, so an inherited flag survives.

2. `isolatedModules: true` on the ts-jest transform removes type-checking at
   transform time, and nine of the 85 mutants were KILLED_BY_TYPECHECK. Measured
   with the flag: 79 killed by test, 6 SURVIVED — #11, #15, #16, #50, #52, #75,
   each a mutant the compiler used to reject and the tests do not catch. Three of
   the nine (#6, #13, #21) turned out to be caught by tests after all; six were
   not. A mutation baseline that records 0 survivors while the tree actually has 6
   is worse than no baseline. The flag is removed.

Its cost, measured: the local suite goes from 26s to 58s, and the CI job from
about 3 minutes to an expected 3.5-4. Six mutants of the canonical-JSON and
validator paths are worth thirty seconds.

Measured after, from the repository root (the invocation the evidence records):
85 sites, 76 killed by test, 9 killed by compiler, 0 survived — identical to the
counts on main. `--verify-structure` prints "Structurally verified 85
source-bound mutation records" with no TOOLCHAIN_MISMATCH, because tools now read
node v24.21.0. tsc --noEmit 0 under both tsconfig.json and tsconfig.test.json.

Closes #103 by answering it: the upgrade it asks for is refused with a measurement
and the security patch it was reaching for is taken.
Arcanada added a commit that referenced this pull request Sep 14, 2026
…ESM move dropped (#114)

* fix: stay on the Node LTS line, and restore the mutation defence the ESM move dropped

Dependabot #103 proposes node 24.20.0 -> 26.8-alpine. Measured against
nodejs.org/dist/index.json: the 24.x line is the CURRENT LTS (Krypton, v24.21.0
released 2026-09-07) and every 26.x release carries `lts: false` — 26 is Current,
not LTS. Moving production off a supported line is not a dependency bump, so this
takes the security patch WITHIN the LTS line instead: 24.21.0 everywhere.

There is a concrete reason to take it. v24.21.0 fixes an out-of-bounds write in
`mkdtemp` for long prefixes (nodejs/node#64770), and this repository calls
`fs.mkdtempSync` in three places, one of them the evidence path itself
(mutation-harness.js gitSupplement). Five pins move together, because
regenerate-derived-artefacts.yml REFUSES a mismatch between ci.yml and
.github/workflows/.node-version-for-derived: apps/api/Dockerfile (x2), ci.yml,
that pin file, and the root engines field.

THE DEFENCE THAT WAS LOST. Two defects, both introduced by the ESM move in #113
and both invisible until a full mutation run was attempted:

1. The harness spawns jest directly and did not pass --experimental-vm-modules,
   so under ESM every battery failed with `SyntaxError: Cannot use import
   statement outside a module` and the run stopped at `baseline: FAIL`. Measured:
   20 suites / 394 tests pass with the flag, 20 suites fail with 0 tests without
   it. `pnpm test` sets the flag; the harness never did. It went unnoticed because
   the only caller of a full run is regenerate-derived-artefacts.yml, gated on
   author == 'dependabot[bot]' — it never ran for #113. NODE_OPTIONS is appended,
   not assigned, so an inherited flag survives.

2. `isolatedModules: true` on the ts-jest transform removes type-checking at
   transform time, and nine of the 85 mutants were KILLED_BY_TYPECHECK. Measured
   with the flag: 79 killed by test, 6 SURVIVED — #11, #15, #16, #50, #52, #75,
   each a mutant the compiler used to reject and the tests do not catch. Three of
   the nine (#6, #13, #21) turned out to be caught by tests after all; six were
   not. A mutation baseline that records 0 survivors while the tree actually has 6
   is worse than no baseline. The flag is removed.

Its cost, measured: the local suite goes from 26s to 58s, and the CI job from
about 3 minutes to an expected 3.5-4. Six mutants of the canonical-JSON and
validator paths are worth thirty seconds.

Measured after, from the repository root (the invocation the evidence records):
85 sites, 76 killed by test, 9 killed by compiler, 0 survived — identical to the
counts on main. `--verify-structure` prints "Structurally verified 85
source-bound mutation records" with no TOOLCHAIN_MISMATCH, because tools now read
node v24.21.0. tsc --noEmit 0 under both tsconfig.json and tsconfig.test.json.

Closes #103 by answering it: the upgrade it asks for is refused with a measurement
and the security patch it was reaching for is taken.

* receipt(#103): file the ChangeAdmissionReceipt/v1 for the Node LTS pin

Draft by the bundle's own verify.py in diff mode: verdict admitted, 3 entities,
all three verified, zero exemptions — nothing needed exempting, which is the
first receipt in this series that can say so.

One finding came out of it and was fixed rather than exempted. The first draft
reported v-config-schema UNDECLARED_CONFIG_KEY NODE_OPTIONS as failed, because
the harness fix read process.env.NODE_OPTIONS to append to an inherited value.
The verifier was right: reading it makes it a configuration key this file
consumes, and declaring a test-harness runtime flag in .env.example — the
application's own configuration — would have been the worse answer. The flag is
now set outright, and regenerate-derived-artefacts.yml (the only caller of a full
run) sets no NODE_OPTIONS, so nothing is lost.

Filed in the tree with the receipt included in its own change_set, and the
evidence's trackedTreeWithoutEvidence rebound after staging it — the MUN-0049
precedent. schema_check.py: CONFORMANT.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant