Skip to content

[ABI-09A.13] Prove cross-engine allocation, transfer, copy-out, and failure settlement #162

Closed as not planned

Description

@wsdt

Implementer reading order: Start with this issue, then read every linked dependency and the authoritative repository specifications named below.

Repository baseline used to prepare this issue: main at ae25c6a49dc09ec4613c7a6f52a27daa69dcd3f3. Before changing code, re-read the current milestone document and inspect the latest main; do not assume paths, diagnostics, or APIs are unchanged.

[ABI-09A.13] Prove cross-engine allocation, transfer, copy-out, and failure settlement

Summary

Complete the runtime-execution portion of PG-7 by proving that Public Generic Descriptor v1 and Logical Carrier v1 have equal checked behavior across:

  • the reference interpreter boundary;
  • native C11 provider compiled at -O0;
  • native C11 provider compiled at -O2;
  • Core Wasm;
  • generated Rust caller;
  • generated TypeScript/Wasm caller;
  • generated C11 caller;
  • generated C++17 caller.

The corpus must exercise real bounded allocation, exact input copy-in, whole-value ownership transfer, endpoint execution, private result staging, atomic result commit, exact copy-out, sticky failure selection, canonical reverse cleanup, stale/double-use rejection, and zero live resources.

A “no crash” test does not satisfy PG-7. The test must compare semantic output, normalized status, ownership transition trace, release order, and resource counters.

Required shared settlement corpus

Create a versioned machine-readable test corpus equivalent in purpose to:

semaprax.public-generic-settlement-corpus.v1

Recommended case fields:

case_id
description
input_semantic_value
input_carrier_digest
failure_injection_id or none
expected_endpoint_invoked
expected_primary_status
expected_result_semantic_value or none
expected_result_carrier_digest or none
expected_normalized_trace
expected_release_order
expected_peak_allocations
expected_peak_handles
expected_final_live_allocations
expected_final_live_handles
applicable_engines
applicable_consumers

Payload data may live in compact fixture files where necessary, but the manifest must bind exact digests and expected outcomes.

Generate canonical input carriers from trusted descriptor facts. Do not hand-author different carrier bytes per engine.

Engine adapter contract

Create a common harness interface with responsibilities equivalent to:

SettlementEngine {
    prepare_exact_subject()
    load_verified_descriptor()
    run_case(input_carrier, injection)
    return EngineOutcome
}

EngineOutcome must include:

accepted
endpoint_invoked
primary_status
secondary_cleanup_statuses[]
result_carrier_bytes/digest
normalized_trace
release_order
peak/live allocation counts
peak/live handle counts
provider/consumer route identity

The comparison layer must reject missing fields. An engine cannot pass by reporting less evidence.

Reference authority

Use the checked Semaprax semantics and existing cleanup/settlement facts as the reference. The interpreter is a runtime implementation, not an excuse to bypass descriptor/carrier validation.

The interpreter route must execute through a boundary adapter that:

  • consumes the same verified descriptor;
  • parses the same logical carrier;
  • stages the same owned leaves;
  • commits the same transfer;
  • invokes the same checked endpoint;
  • stages/commits the same result;
  • emits the same normalized trace vocabulary;
  • uses deterministic failure injection.

Do not compare a direct interpreter function call to native/Wasm carrier calls; that would skip the boundary being proven.

Required semantic fixtures

At minimum include:

1. Minimal success

  • one concrete generic input record;
  • one owned Bytes leaf;
  • one substituted Copy scalar;
  • one concrete generic result record;
  • exact expected result;
  • exact result carrier bytes/digest;
  • exact release order;
  • zero final resources.

2. Zero-length owned bytes

  • zero-length input leaf;
  • zero-length and/or nonzero result according to fixture;
  • no null/zero ambiguity;
  • no phantom allocation obligation unless the existing allocator contract explicitly creates one.

3. Embedded zero bytes

  • payload contains multiple 0x00 bytes;
  • exact length preserved;
  • no C-string behavior.

4. Every admitted Copy scalar argument

Exercise the boundary profile with each admitted concrete argument from Public Generic Type Grammar v1. Where one endpoint cannot cover all arguments, generate a bounded fixture family under the same profile.

5. Nested owned records

  • at least two record levels;
  • at least two owned leaves;
  • structural leaf order visible;
  • reverse cleanup order visible.

6. Maximum admitted payload

  • exactly max per-leaf bytes;
  • exactly max total carrier payload where practical;
  • successful allocation and result;
  • peak counters pinned.

7. First-over-bound

  • max + 1 per leaf;
  • max + 1 total;
  • max + 1 leaf count;
  • max + 1 allocation/handle count;
  • refusal before endpoint execution;
  • zero semantic allocations afterward.

8. Repeated invocation

  • reuse a live provider for multiple independent calls if profile permits;
  • each input/result has independent lifecycle;
  • no digest/handle state leaks between calls;
  • exact result repeated;
  • zero resources after each iteration;
  • bounded repeat count and a higher stress repeat in a non-default/hosted gate if appropriate.

9. Provider recreation

  • close and reopen provider;
  • same descriptor/provider binding succeeds;
  • stale child handles from prior provider generation fail;
  • no resource carry-over.

Required failure-injection cases

Use the logical semantic event IDs from #153. Each case must run on all applicable engines.

Input preparation

  • fail before first allocation;
  • fail on each input leaf allocation;
  • fail after allocation, before payload copy;
  • fail after each payload copy;
  • fail while constructing nested whole input;
  • fail immediately before input transfer commit.

Expected:

  • endpoint not invoked;
  • caller/pre-transfer side owns completed leaves;
  • completed leaves released in reverse order;
  • primary injected status preserved;
  • zero final resources.

Immediately after transfer

Inject immediately after input transfer commit but before endpoint body.

Expected:

  • endpoint body not invoked unless the logical event is defined after invocation starts;
  • provider owns whole input;
  • caller cannot release/reuse;
  • provider settles all input leaves;
  • exact provider-side release trace;
  • zero final resources.

Endpoint execution

  • requires/precondition failure;
  • arithmetic or existing normalized contract failure applicable to fixture;
  • explicit test-injected runtime failure;
  • success result suppressed on failure.

Expected:

  • input already provider-owned;
  • no result committed;
  • input settled exactly once;
  • sticky endpoint status;
  • zero final resources.

Result staging

For every result leaf:

  • allocation failure;
  • failure after allocation before payload copy;
  • failure after payload copy;
  • failure during nested result construction;
  • failure immediately before result commit.

Expected:

  • consumer sees no result;
  • only completed result leaves release;
  • release order is canonical reverse;
  • provider-owned input still settles;
  • primary status preserved;
  • zero final resources.

Immediately after result commit

Inject a failure in the consumer copy-out/export path after the result handle is committed.

Expected:

  • result ownership is on the result handle/consumer boundary;
  • export may fail without losing or duplicating the result;
  • retry behavior follows the adapter contract;
  • explicit release works;
  • no endpoint re-execution;
  • zero final resources.

Cleanup failures

Inject failure at each logical release position:

  • with no prior failure;
  • after an input/runtime failure;
  • after a result-staging failure.

Expected:

  • first authoritative failure remains primary;
  • release failures appear in deterministic secondary evidence;
  • later safe cleanup continues;
  • no double release;
  • final resource count matches contract;
  • if a deliberately unreleaseable test resource remains, the case must fail the gate unless the specification explicitly defines a terminal leaked state—which v1 should avoid.

Exact cross-engine equality

For every case, compare:

  • accept/reject;
  • endpoint invoked flag;
  • normalized primary status;
  • semantic result;
  • canonical result carrier digest;
  • normalized trace event sequence;
  • canonical leaf release order;
  • live resource counts;
  • peak counts where target-neutral;
  • descriptor/instance identities referenced in evidence.

Target-local pointer values, handles, module offsets, compiler symbols, and timing are excluded from equality.

Where native and Wasm allocation implementation counts necessarily differ, compare a normalized logical allocation count and separately assert target-local counters reach zero. Document any non-comparable peak field rather than omitting it silently.

Native optimization equivalence

Compile the exact same provider fixture at -O0 and -O2.

Require:

  • same descriptor/provider semantic binding;
  • same input and result carrier semantics;
  • same normalized status/trace;
  • same release order;
  • no optimizer-dependent uninitialized bytes;
  • no undefined behavior;
  • sanitizer success on an appropriate instrumented build;
  • no accidental difference hidden by regenerating expected output separately per optimization level.

The provider artifact digest may differ by optimization. Provider-binding fixtures must reflect that while semantic descriptor equality remains clear.

Wasm equivalence

Require Core Wasm to:

  • accept the same canonical input carrier;
  • produce the same semantic result and result carrier;
  • use the same normalized failure IDs;
  • settle handles/allocations to zero;
  • reject hostile memory ranges before semantic execution;
  • avoid exposing partial results;
  • avoid host-object ownership assumptions.

Generated consumer execution

Each generated consumer must run through its real provider path.

At minimum, every consumer executes:

  • minimal success;
  • nested success;
  • max bound;
  • first-over-bound;
  • endpoint failure;
  • result-staging/export failure;
  • stale/cross-paired descriptor/provider;
  • release/double-use case appropriate to the language;
  • zero-resource assertion.

The complete failure-injection matrix may be driven through provider test hooks rather than exposing production APIs.

Cancellation and concurrency

The v1 profile is synchronous and effect-free. Do not add cancellation or concurrent execution merely to satisfy a generic checklist.

Document and test:

  • cancellation is not admitted in v1;
  • no hidden timeout retries execution;
  • thread/concurrent use has the exact adapter policy defined in native/Wasm specs;
  • absence of concurrency support is a nonclaim, not a failure of settlement.

Evidence artifacts

Produce deterministic evidence for each engine/case, such as a canonical summary containing:

case_id
engine_id
provider_binding_digest
descriptor_digest
primary_status
result_digest
trace_digest
release_order_digest
final_live_resource_counts

Evidence must contain no payload bytes unless the existing test fixture policy intentionally includes them.

Independent replay must reject:

  • changed case ID;
  • changed engine ID;
  • changed status/result/trace/release digest;
  • another provider’s evidence;
  • truncated/reordered/unknown fields;
  • reminted evidence against the wrong trusted case.

Implementation steps

1. Inventory current engine/runtime settlement tests

Mandatory repository reconnaissance

Before editing anything:

git fetch origin
git switch main
git pull --ff-only

rg -n "public_generic|Public Generic|PUBLIC-GENERIC|public-generic"   src tests docs .github Cargo.toml

rg -n "public_generic_ownership_milestone|public-generic-ownership-milestone"   src tests docs .github

rg -n "SPX-PG[0-9]{3}|SPX-[A-Z][0-9]{3}" src tests docs

Then write a short implementation note in the PR description identifying:

  1. the current owning module(s) for the type grammar, compatibility, candidate delta, metadata consumers, and settlement obligations;
  2. the current canonical serialization helpers and independent replay pattern to reuse;
  3. the current diagnostic allocation mechanism and the next genuinely free code range;
  4. the current CI selectors and release-blocker inventory that exercise the public-generic milestone;
  5. every existing public projection that must remain closed until PG-9.

Do not create a parallel framework when an owning helper already exists. Do not guess a source path from this issue; locate the current owner by symbol and contract.

Also run:

rg -n "interpreter.*native.*wasm|O0|O2|settlement|cleanup.*event"   src tests docs .github

rg -n "failure.*inject|live.*allocation|live.*handle|trace.*digest"   src tests docs

Reuse current all-engine and owned-data settlement harnesses.

2. Write the corpus/evidence specification

Pin cases, injection IDs, outcome schema, equality rules, bounds, evidence bytes, and nonclaims.

3. Implement boundary-aware interpreter adapter

It must consume descriptor/carrier and emit normalized lifecycle evidence.

4. Connect native O0/O2 and Wasm adapters

Use identical fixture source and logical cases.

5. Add deterministic failure injection

Ensure each engine maps the same logical injection ID to the same transition.

6. Run generated consumers

Use the real external packages/binaries built in issues 007–010.

7. Compare and replay evidence

One focused gate should produce useful diffs by case/engine/trace event.

8. Prove zero resources

Expose test counters from each provider/client layer and assert them after every terminal case, including failures in cleanup.

Acceptance criteria

  • One versioned settlement corpus exists.
  • All engines consume the same descriptor and canonical case carriers.
  • Interpreter route exercises the boundary, not a direct internal shortcut.
  • Native C11 O0/O2 and Core Wasm execute every required case.
  • Generated Rust/TS/C/C++ callers execute the real endpoint.
  • Every logical failure-injection point is covered.
  • Whole input transfer and result commit boundaries are visible in traces.
  • Partial results are never exposed.
  • Sticky primary failure and secondary cleanup evidence are exact.
  • Cleanup order derives from compiler settlement facts.
  • Semantic results/result carrier digests agree.
  • Normalized statuses/traces/release orders agree.
  • Target-local live allocations/handles are zero after every terminal case.
  • Exact max and first-over-bound cases exist.
  • Native O0/O2 are equivalent and sanitizer-clean.
  • Evidence is canonical and independently replayable.
  • Synchronous/no-cancellation scope is explicit.
  • PG-7 documentation is advanced only after the entire corpus passes locally.
  • Hosted-green is not claimed until [ABI-09A.14] Refresh Linux, macOS, and Windows hosted evidence for the expanded milestone corpus #163.

Expected touchpoints

  • settlement corpus specification/manifest;
  • reference boundary adapter;
  • native fixture build at O0/O2;
  • Wasm fixture;
  • generated external consumer harnesses;
  • failure-injection and resource-counter support;
  • normalized trace/evidence renderer/verifier;
  • PG-7 milestone/quality ledgers and changelog.

Diagnostics and refusal behavior

  • Search the complete repository before allocating a diagnostic. Never reuse a code merely because it looks related.
  • Add new diagnostic codes through the current central registry/generator so help text, documentation, and tests remain synchronized.
  • Use a closed reason vocabulary for machine-consumed refusals. Do not parse prose to recover a reason.
  • Define and test deterministic refusal precedence. A recommended order is:
    1. option and total-input bounds;
    2. framing and strict syntax;
    3. schema/version and closed-key validation;
    4. canonical-byte validation;
    5. trusted subject selection;
    6. retained-fact reconstruction;
    7. digest and cross-pair validation;
    8. lifecycle/ownership-state validation;
    9. runtime execution.
  • Error messages may add context, but the stable diagnostic/reason identity must be sufficient for tests and generated consumers.
  • Add exact negative tests for every new reason, including the first value above every configured bound.

Dependencies and follow-on

Depends on:

Blocks #163.

Agent handoff checklist

Before asking for review, the implementing agent must provide:

  • a concise list of changed contracts and why each change is necessary;
  • exact commands run locally and their exit codes;
  • the focused test names added or changed;
  • the before/after public-surface claim, explicitly stating what remains unsupported;
  • any new diagnostic codes and proof that they were free;
  • all changed canonical schema/version identifiers;
  • a note confirming that frozen predecessor bytes and existing public rejection tests still pass;
  • any deliberately deferred work, linked to the issue that owns it rather than hidden in prose.

A handoff that says only “tests pass” is not sufficient.

Definition of done

This issue is done only when:

  • the implementation and owning specification agree exactly;
  • focused positive, hostile, determinism, replay, and first-over-bound tests pass;
  • existing compatibility and public-separation tests remain green;
  • generated artifacts are reproducible from the same trusted inputs;
  • no unsupported/published claim is introduced;
  • documentation catalogues, completion/quality ledgers, and diagnostic help are updated where required;
  • the PR contains enough evidence for another agent to reproduce the result without private context.

Activity

  1. added
    area:abiPublic interfaces, descriptors, carriers, compatibility, and ABI work
    area:runtimeExecution, lifecycle, ownership settlement, and runtime behavior
    area:wasmCore Wasm, Wasm host, memory, and generated TypeScript/Wasm work
    area:nativeNative C ABI, provider, C/C++ consumer, and sanitizer work
    kind:testAssurance, evidence, hostile testing, or CI gate
    priority:p0Critical path; blocks the milestone
    agent-readyDetailed enough for a bounded implementation agent
    on Sep 11, 2026
  2. wsdt commented on Sep 12, 2026

    @wsdt
    MemberAuthor

    Interpreter ≡ Core-Wasm is now enforced by one checker over a shared corpus, and it caught a real pre-existing bug in #155. Landed in 55a87305, d8bad24a, 56dbb870, 8117bff4.

    The bug, and why an agreement-only test would have missed it

    WasmProvider::call (#155) only ever performed the physical free (release_input_physical) after commit. It never called the logical release_input_after_transfer/release_input_before_transfer — the methods that actually record LeafRelease/CarrierRelease trace events. So on every real call, success or failure-after-commit, the emitted normalized trace carried no input-side release event at all.

    The module documentation claimed the adapter "reuses that same method unconditionally … success or failure." The code never called it.

    It surfaced only because one test pins the literal staging and release order — vec![0,1] staged, vec![Some(1), Some(0), None] released — rather than merely asserting the two engines agree. A consistently-wrong order on both sides passes an agreement-only check silently. The test's first run came back with an empty release order instead of the expected exact-reverse sequence.

    Fixed at every site that already performs the physical free, in the Wasm adapter and in the new interpreter adapter that had mirrored the same gap. I confirmed at integration that #172's generated-consumer harnesses (public_generic_native_adapter_v1 16 passed, public_generic_wasm_adapter_v1 15 passed) still pass unchanged, so no downstream consumer depended on the missing events.

    What was actually missing for this issue

    TargetProfile::Interpreter existed but no adapter was bound to it. src/public_generic_abi/interpreter.rs is the third PG-7 physical adapter, and it is what makes a cross-engine comparison possible at all.

    How equivalence is enforced, not merely designed

    carrier::settlement_corpus::compare is a single checker every case funnels through, running the identical case against both adapters — both driving the literal CarrierCallMachine — and asserting, per case: accept/reject against an independently pinned expectation and between engines; normalized status likewise; result carrier bytes between engines and against an independently computed value; the literal TraceEvent sequence the machine recorded (via a new test-only accessor snapshotted at each settle(), not a restatement); the canonical leaf release order extracted from that trace; final live allocation, handle and byte counts independently zero on both sides; the sticky-settlement overwrite-attempt count; and provider close status.

    Four should_panic negative controls prove the checker can fail. Two of them initially fired at the wrong assertion — perturbing a single engine tripped the "engines disagree" check instead of the "zero live allocations" check the test was named for. Both were corrected to perturb identically so the intended check is the one that fires.

    Criterion-by-criterion

    Invariant State
    Failure selection is sticky Met — cleanup_cannot_overwrite_an_earlier_sticky_failure (interpreter), sticky_failure_survives_a_later_cleanup_failure_attempt (Wasm, pre-existing), plus the cross-engine corpus case
    Cleanup order is structural metadata, canonical runtime order Met — structural_leaf_order_is_left_to_right_staged_and_exact_reverse_released, the literal pin that found the bug
    Owned call stages left to right, transfers at commit Met — same test asserts staged_leaf_order == vec![0,1] on both engines
    Equivalent checked behaviour across backends Partial — every_corpus_case_agrees_across_the_interpreter_and_wasm_engines covers 21 cases (7 base shapes + 14 failure-injection ordinals). Native C11 is not included: it has no in-process Rust adapter, only a C-source renderer, so compiled -O0/-O2 equivalence against this corpus is not built here
    Settlement model is proof data, grants no finalizer/spawn/publish Met by inspection, not by test — no std::fs/std::net/std::process/Command::new anywhere in the touched files. Flagged honestly rather than claiming a test that does not exist

    Gaps cited, not duplicated

    #229 — both adapters are in-process Rust models; no compiled .wasm executes anything here, and reverse_probe.mjs never touches CarrierCallMachine. #119 — flat owned-Bytes only; the two_leaves_structural_order case stands in for multiple owned leaves with visible order, not a nested record. Native -O0/-O2 and the generated-consumer execution live in the harnesses leased to #172 this round and were cited by exact file rather than attempted.

    The corpus compares in-process rather than emitting a portable evidence artifact with case_id/engine_id/digests. That is stronger for catching regressions locally and weaker for independent replay; recorded in the document's nonclaims.

    Gates on the merged tree: public_generic_abi 251 passed (was 230); projections 5; both adapter harnesses unchanged; documentation 29; module_size 1; fmt --check and clippy --lib/--bins clean.

  3. 14 remaining items

  4. wsdt commented on Sep 16, 2026

    @wsdt
    MemberAuthor

    Issue #162 — stage 4: calling-consumer settlement and bounded codecs

    Delivery and exact baseline

    This is the fourth implementation increment for #162. It continues the original uploaded semaprax-main.zip plus stages 1–3; it is not a rebase onto an unspecified newer GitHub main.

    • Original uploaded baseline: 6fb62cfcedb948f7b32d5accb38aefdffaf0db68 (recorded for the supplied ZIP).

    • Local reconstruction commit b76f7a1: the original upload.

    • Local reconstruction commit 123b67b: the upload plus stages 1–3; this is the incremental patch base. These local reconstruction commits are not claimed to exist on GitHub.

    • semaprax-issue-162-stage4.patch: apply after all three prior increments.

    • semaprax-issue-162-cumulative-stage4.patch: stages 1–4 together, for a clean copy of the original upload. Do not apply both variants.

    Both routes were applied with git apply --check --whitespace=error and git apply --whitespace=error to freshly extracted copies of the original archive. All 71 cumulative changed files matched the working tree byte-for-byte. Stage 4 changes 48 files, including implementation, generator text assets, tests, workflow, and documentation.

    # After stage 3:
    git apply --check semaprax-issue-162-stage4.patch
    git apply semaprax-issue-162-stage4.patch
    

    Alternative only: original clean upload, with no prior increments:

    git apply --check semaprax-issue-162-cumulative-stage4.patch
    git apply semaprax-issue-162-cumulative-stage4.patch

    Implemented contracts and reasons

    1. C11 explicit settlement, without masking the primary error

    The owning public_generic_consumer::c_calling generator now emits one auditable cleanup path. The new spx_pg_consumer_transform_with_settlement API reports primary_status, native_status, and the explicit result-release release_status independently.

    An earlier encode, prepare, call, export, allocation, or decode error stays primary. When decode succeeds but native result release fails, the caller returns SPX_PG_CONSUMER_RELEASE_FAILED = 9, destroys its decoded leaves in reverse order, and returns no output. Existing enum values 0–8 retain their meanings. The old transform API delegates to this path. It retains its original single-primary-status interface; consumers needing secondary release evidence use the new report API.

    This report is specifically about explicit result release. Provider-internal cleanup failures remain in the provider observations and are not silently claimed to be a complete secondary-status array in the small C report.

    spx_pg_consumer_close_checked returns a status and preserves the client guard when native close refuses while retaining its provider. The legacy void close delegates and logs a best-effort note. Client guard allocation happens before provider open, so a client allocation failure cannot orphan a newly opened provider.

    2. Bounded, fully validated consumer codecs

    C and Rust now preflight 65,536 bytes per leaf, 16,777,216 bytes total payload, and full carrier framing before encoding allocations. Exported required sizes are bounded before allocating an output buffer. Entire output framing, including the final leaf and trailing bytes, is validated before allocating any decoded result leaf. Arithmetic and integer narrowing are checked.

    The C source API requires valid owning allocations. It rejects exact duplicate input-owner pointers, consumes them once, and rejects noncanonical null/zero representations. Forged pointers or interior pointers are not made safe by these checks: they remain outside this owning C source-level contract. Input, partially decoded output, completed output, and C++ staging copies discharge in reverse structural order.

    C and Rust executable shape generation now refuses 0 and 257 leaves and admits 1–256; C++ inherits C's validation. Existing duplicate-identity and input/result-count mismatch precedence is unchanged. The shared internal generator error gains ShapeError::LeafCountOutOfBounds.

    3. C++17 ownership and error propagation

    The existing C++17 wrapper still uses the generated C client; it introduces no second codec or native provider ABI. Error carries explicit secondary release status. Provider::close_checked() preserves its owner on a retriable close refusal. Move assignment does not overwrite either owner when closing the target fails. Copy operations remain deleted, move operations and destructors remain noexcept, and self-move remains safe.

    C++ checks vector lengths before copying into C-owned input storage. Staging allocation failure releases only acquired copies, in reverse order. The test-only allocator substitution must return ordinary malloc/free-compatible storage; it is not a general published allocator ABI. One-shot test injection is cleared even when a C++ local refusal prevents entering the C transform.

    Destruction remains a best-effort, nonthrowing fallback. A destructor cannot return a close error and cannot promise recovery from a provider that deliberately refuses close indefinitely. Explicit checked close is the observable, retryable path.

    4. Rust calling-consumer implementation

    The Rust generator implements fallible buffer reservations, checked sizes, reverse source/result discharge, Error::AllocationFailure, Error::ReleaseFailed(i32), Provider::last_release_status(), and checked/idempotent Provider::close().

    A result guard now releases before a successful output escapes. Export/decode errors remain primary if release also fails; the most recent explicit release error is separately inspectable. Local input refusal consumes its test injection, so another call does not inherit it. Destructor diagnostics use best-effort std::io::Write, not eprintln! or an allocated diagnostic String.

    The Rust generator changes and generated Rust runtime were not compiled or executed locally. Cargo, rustc and rustfmt are unavailable. This is implemented code with regression harnesses, not a Rust execution proof. The C half of its native fault shim did compile warning-free at Clang O2.

    5. Production templates and actual-generator bridge

    Fixed C/C++ text moved into their existing render owners' render/*.txt assets. The Rust generators still assemble and own their output. The standalone test fixture assembler substitutes only trusted field lists into those exact assets; it does not implement a second provider or claim to reconstruct compiler descriptors.

    Locally executed evidence is explicitly labelled production-template-fixture. A new Cargo integration test invokes the actual C/C++ generators for 1, 2, 3 and 256 fields, then passes --generated to require byte-for-byte agreement before compiling and executing them. That route is labelled rust-generator-checked. The bridge is implemented but, like the other Cargo tests, was not run here.

    Shared cases, observations, and independently replayable evidence

    The committed canonical manifest contains 112 distinct cases: 108 C11 and 105 C++17 executions per build, with explicit route applicability. It covers minimal/two-leaf cases; zero and embedded-zero bytes; exact 64 KiB and 16 MiB/256-leaf bounds; oversized/null/duplicate-owner source inputs; all 14 legacy logical injection ordinals; actual result allocation/copy/commit/export/release injections; ten malformed export forms; client allocation failures; partial rollback after two acquired leaves; eight calls through one provider; checked-close retry; refused move assignment; and injection isolation after a local rejection.

    Native and caller allocations use independent observers. Each case checks semantic result bytes, endpoint invocation counts, primary and secondary status, actual release order, allocation observations, and final zero resources. The caller observer counts C-owned buffers and C++-to-C staging, not every C++ standard-library allocation or the Rust heap. ASan/UBSan additionally instruments the entire native C/C++ executable.

    Cross-optimization checks compare complete observations, including caller peaks. Cross-language checks compare the native results/statuses/traces/release order/resource observations. Caller-specific peaks and release identifiers remain explicit target-local fields rather than being dropped or renamed to suggest false equality.

    New identifiers:

    • semaprax.public-generic-consumer-settlement-corpus.v1

    • semaprax.public-generic-consumer-settlement-evidence.v1

    • Fixture scope: flat-owned-bytes-reference-fixture.v1

    Evidence contains no result payload bytes. It binds cases, trusted source digests, descriptor/binding fixture bytes, executable digests, selected route, statuses, result/trace/release digests, and resource observations. Replay recompiles the trusted checkout and requires canonical byte equality with fresh execution; it never executes a submitted binary. It rejects missing/reordered/unknown fields, wrong identities, changed/reminted trace or result facts, and boolean values masquerading as integer zero. Exact artifact replay requires the same compiler/configuration; this is not cross-host binary identity.

    86 evidence/receipt negative controls passed. 11 compiled semantic mutants each first ran a positive control, compiled successfully after mutation, and failed the intended runtime assertion: masked release; overwritten primary; forward input cleanup; forward complete-output cleanup; forward partial-decode cleanup; unbounded exported size; orphaned close guard; allocation before full decode validation; forward C++ staging rollback; owner lost on refused move assignment; and injection leakage after local refusal. An unrelated crash or compile error does not count as a successful mutant rejection.

    Local validation and exact commands

    Commands were run from the reconstructed repository or a fresh patched copy. The absolute output paths below are this validation environment's artifacts; target/... paths may be substituted for reproduction. All successful C/C++ compilations used warnings as errors. Logs and payload-free evidence are in the validation ZIP.

    Command / check Exit Result
    python scripts/public_generic_consumer_settlement.py --sanitizers --output /mnt/data/stage4-validation/final-clang 0 O0 213, O2 213, ASan/UBSan 213; 639 records and 86 negative controls
    python scripts/public_generic_consumer_settlement.py --cc gcc --cxx g++ --output /mnt/data/stage4-validation/final-gcc 0 O0 213, O2 213; 426 records and 86 negative controls
    python scripts/public_generic_consumer_settlement.py --sanitizers --replay /mnt/data/stage4-validation/final-clang/consumer-settlement.json --output /mnt/data/stage4-validation/fresh-replay 0 Fresh cumulative-patch checkout recompiled and reproduced all 639 records exactly
    python scripts/public_generic_consumer_mutations.py 0 All 11 compiled mutants rejected by intended assertions, repeated on the fresh checkout
    python scripts/public_generic_settlement_evidence.py --sanitizers --stress --output /mnt/data/stage4-validation/native-regression 0 Prior 22 shared cases, 61 native failure regressions, 12 lifecycle cases and 72 result-phase cases retained at O0/O2/sanitizers; 124 prior negative controls
    Prior native gate with all three --replay* options, shown below 0 Fresh checkout independently reproduced all three prior native evidence artifacts
    python -m py_compile scripts/public_generic_consumer_fixture.py scripts/public_generic_consumer_cases.py scripts/public_generic_consumer_settlement.py scripts/public_generic_consumer_mutations.py 0 Python syntax checked
    clang -std=c11 -Wall -Wextra -Werror -O2 -c /mnt/data/stage4-validation/rust-shim/provider.c -o /mnt/data/stage4-validation/rust-shim/provider.o 0 Test-only Rust bridge's native C shim compiled; no Rust execution claim
    Both strict-whitespace patch-application routes 0 71 cumulative files equal between both fresh routes and work tree
    git diff --check 0 No patch whitespace errors
    Auxiliary Python line-count and YAML checks 0 All changed Rust modules <=1500 lines; CI YAML parsed, not actionlint/hosted validation
    cargo fmt --all -- --check 127 Not run: Cargo/rustfmt unavailable
    cargo test --locked -p semaprax --test public_generic_native_adapter_v1 consumer_settlement 127 Not run: Cargo unavailable
    cargo clippy --locked --all-targets --all-features -- -D warnings 127 Not run: Cargo unavailable
    sh scripts/quality.sh full 127 Stops at missing Cargo; full quality gate is not green

    Fresh replay of prior evidence:

    python scripts/public_generic_settlement_evidence.py --sanitizers --stress \
      --replay /mnt/data/stage4-validation/native-regression/evidence.json \
      --replay-lifecycle /mnt/data/stage4-validation/native-regression/native-lifecycle-evidence.json \
      --replay-result-phases /mnt/data/stage4-validation/native-regression/native-result-phase-evidence.json \
      --output /mnt/data/stage4-validation/fresh-native-replay

    Toolchains actually used: Clang 17.0.0, GCC 14.2.0 and Python 3.13.5, Linux. No macOS, Windows/MSVC, minimum-Rust, browser, or hosted CI success is inferred.

    Focused Rust test inventory added (not executed locally)

    Six generator unit tests:

    • executable_shape_bounds_are_exact_and_do_not_emit_empty_or_oversized_records in each of C, C++ and Rust generator modules.

    • C: explicit_settlement_api_preserves_status_numbers_and_hides_native_handles.

    • C++: release_errors_and_retained_move_ownership_are_in_the_generated_wrapper.

    • Rust: result_is_settled_before_success_and_destructors_use_nonpanicking_writes.

    Two Unix integration selectors in tests/public_generic_native_adapter_v1/consumer_settlement.rs:

    • generated_c_and_cpp_match_fixture_bytes_and_execute_the_settlement_matrix.

    • generated_rust_explicit_settlement_executes_against_the_physical_provider.

    The external generated Rust crate also gains bounded_codec_tests::input_first_over_bound_refuses_before_encoding and bounded_codec_tests::hostile_u64_length_and_truncations_are_errors_not_panics. Its external explicit_release_export_decode_and_close_preserve_ownership_and_primary_error test contains ten scenario groups against the native test shim. Its no-dependency lockfile is generated offline before --locked --offline execution. Missing tools fail the selected bridge rather than silently skipping it.

    Preserved authority and public surface

    The native provider body and public ABI header are byte-identical to stage 3:

    • src/public_generic_abi/native/provider_body.c: SHA-256 dfe105ccaaa5680fd00add6650dd1f40dfa68e9a8b138c85b08a7cfefe6d7594.

    • src/public_generic_abi/native/spx_pg_v1.h: SHA-256 d24993540ad59112c0e0bbd81a49adb81d5acf60948a3fc0295ae09a4d09f8f7.

    No SPX diagnostic code was allocated, no existing canonical descriptor/carrier format was widened, and existing native evidence schemas were preserved and replayed. C's generated source API adds checked operations/status 9, and Rust's error enum gains variants; these are real caller-contract changes, not a claim that every generated source file is frozen. Metadata consumers, TypeScript/Wasm execution, support status, publication status, and PG-9 promotion are unchanged. Frozen Rust compatibility/public-rejection tests were not executed in this environment and must not be called green on the basis of unchanged files alone.

    The milestone, quality/completion ledgers, owning consumer/corpus specifications, architecture notes, changelog, and existing Linux workflow now identify this subset. The workflow adds sanitizers, fresh replay, and mutation execution; the existing Cargo integration target discovers the new Unix bridges. Workflow editing is not hosted-green evidence.

    Remaining #162 work and merge checks

    This increment is not full #162 acceptance. It does not deliver a compiled Wasm provider ABI (#229), compiler-derived generic endpoints/descriptors, nested owned records and all Copy scalar arguments (#119/profile work), the complete four-language caller matrix (#172), or exhaustive interpreter/native/Wasm logical-event equivalence. The native/reference fixture's flat carrier and known descriptor bytes are intentionally not described as the completed Public Generic compiler pathway.

    The first merge check is to run formatting, actual-generator C/C++ byte comparison, generated Rust/native tests, clippy, and the repository's full quality gate in a Rust-equipped environment. Formatting/compilation adjustments may still be needed because those gates could not run here. Do not mark PG-7 complete, close #162, or claim hosted-green (#163) from this patch. Synchronous/no-cancellation and no-concurrent-execution scope remains unchanged; an export or close retry is not an endpoint retry.

  5. added 2 commits that reference this issue on Sep 16, 2026
  6. wsdt commented on Sep 17, 2026

    @wsdt
    MemberAuthor

    Issue #162 — batch 5 implementation handoff

    Patch baseline and application

    This batch continues the stage 4 tree supplied in this conversation. The original uploaded ZIP identifies main at 6fb62cf. It is not a rebase onto a later moving main.

    • semaprax-issue-162-stage5.patch: incremental; apply after all four earlier batches.

    • semaprax-issue-162-cumulative-stage5.patch: stages 1–5 together; apply only to a clean original uploaded ZIP/baseline.

    Use one route, never both:

    git apply --check semaprax-issue-162-stage5.patch
    git apply semaprax-issue-162-stage5.patch

    The batch changes 34 files, 2,640 insertions and 1,167 deletions. A substantial part of the deletion count moves existing TypeScript string fragments into fixed template assets rather than removing functionality. The corpus manifest is a canonical machine-readable file.

    Evidence class and completion state

    Issue #162 remains partial. PG-7 and hosted-green are not claimed.

    This batch fixes and executes the existing host-owned TypeScript/Wasm reference calling consumer. Its reversal endpoint really executes Wasm instructions under Node, but the small module is the repository's explicitly hand-assembled endpoint fixture, not a Semaprax-compiler-produced generic provider. Allocation, handles, descriptor checks, transfer bookkeeping and cleanup remain in the generated host wrapper. No open/prepare/call/export/release provider ABI is claimed to have been compiled into Wasm.

    The issue #229 discussion explicitly identifies compiler admission/profile work as the prerequisite and rejects confusing a cross-compiled runtime shim with compiler/provider proof. This implementation does not bypass SPX-W115, change generic export admission, or introduce such a shim. The former fixtures and their nonclaims remain.

    The executed package files are assembled from the production template assets, with deterministic shape-specific fragments. A new Cargo bridge writes the actual Rust generator output into a separate, initially empty destination, compares all ten files for each of 16 subjects, and executes those actual files when selected. That Cargo bridge could not run here: Cargo, rustc and rustfmt are unavailable. Production template execution is not presented as proof that the Rust generator compiled.

    Confirmed baseline defects and repairs

    Baseline reproductions are retained in the validation archive. They show the following defects in the stage 4 TypeScript consumer:

    1. A precompiled WebAssembly.Module was accepted without authenticating its original bytes. The wrapper now refuses this input with module-bytes-required; callers must supply the bytes whose digest can actually be checked.

    2. Opening a module retained an alias to caller-controlled bytes across asynchronous digest verification. A mutation between verification and compilation produced a native CompileError. The wrapper now takes an exact private snapshot before the first await, authenticates that snapshot, and compiles the same snapshot. BufferSource subranges are respected.

    3. Mutating publicly exported descriptor/binding byte arrays changed the supposed trust authority. Verification now uses private, captured expected bytes. Exported arrays remain conveniences, not mutable authority.

    4. Multiple outstanding preparations allowed out-of-order release to lose allocator ownership: the baseline retained one live allocation and close failed. The wrapper now admits one outstanding frame/call per provider and refuses a competing operation without releasing another operation's resources.

    5. Encoding invoked input property getters. Exact own data-field validation now rejects accessors and extra/missing/symbol fields rather than executing accessors as part of semantic encoding. Reflective failures, including a proxy throwing a forged error instance, are normalized to the closed input-shape reason. JavaScript proxy traps themselves are not claimed impossible; reentrant provider access is guarded.

    6. A failure injection that was never reached because local encoding failed leaked into the next call. One-shot fault plans now expire with the attempted call, including locally refused attempts.

    7. The advertised 16 MiB payload could not fit in the old 256-page arena once carrier framing was included. The reference configuration now admits a complete 16,779,272-byte frame, including 8 + 8 × 256 framing bytes, using at most 257 Wasm pages. The old reference_wasm_module::build() continues to request 256 pages; the new explicit helper supplies 257 only to these new subjects.

    The baseline reproduction JSON has seven observations, including both live-resource and close-status observations for the same out-of-order-release defect. The maximum-payload reproduction is recorded separately; these are not eight independent bugs.

    Changed contracts and owning code

    Authentication and bounded input

    The owning generator is src/public_generic_consumer/typescript_calling.rs; fixed output assets and shape rendering remain under typescript_calling/render/ and render.rs. This extends that generator rather than replacing it with a new production generator.

    The wrapper verifies immutable descriptor/binding facts, module digest, import/export inventory, memory properties and endpoint shape. Precompiled modules, mutable shared/resizable/detached buffers, oversized modules and invalid module surfaces receive closed reasons. The accepted module-byte bound is exactly 16 MiB. Authenticated hostile fixtures cover real compilation, instantiation, memory-growth and endpoint failures.

    Input byte validation uses captured typed-array intrinsics rather than user-overridable length/buffer/tag/iterator methods. Exact keys and canonical field order are retained. Complete-frame validation finishes before result leaf copying, including the last leaf. Limits are one to 256 leaves, 65,536 payload bytes per leaf, 16 MiB aggregate payload and the framing-inclusive carrier bound above.

    No claim is made to recover from fatal host out-of-memory termination. Recoverable allocation and memory-growth exceptions are normalized and settled.

    Ownership and cleanup

    The existing host reference provider now has privately branded, provider-bound, kind-bound, generation-checked handles. Generation is monotonically bounded; exhaustion refuses new work rather than resurrecting an old handle. Close with a live owner refuses and retains the owner for explicit settlement/retry.

    Input transfer and result commit are visible events. Result construction remains private until commit. Nonconsuming export may be retried and does not invoke the endpoint again. Input and result frames are physically zeroed before allocator ownership is forgotten. An explicit release reports a cleanup failure only after invalidating and settling the resource; callers cannot double-release it to “repair” the reported failure.

    Release failure becomes primary when no earlier operation failed. Otherwise the earlier failure remains primary and cleanup status is attached as immutable secondary evidence. The wrapper does not silently turn release failure into success or replace an earlier execution/export/decode error.

    The 19 named fault events are explicitly host-adapter events, not relabelled CarrierCallMachine events. At most two fault points are admitted. Legacy ordinal meanings are retained. The trace has a 4,096-event budget, reserving cleanup capacity before ordinary events exhaust the bound. Reports and secondary status arrays are immutable snapshots.

    Resource accounting and noncomparability

    Counters measure explicit host-owned frames in Wasm memory, active leaf-container handles, frame bytes, zeroed bytes, endpoint calls and retained memory pages. They do not measure JavaScript garbage-collected objects or prove host heap allocation counts. Provider objects are not mislabeled as leaked child handles. Retained Wasm pages are reported explicitly and are not expected to shrink to zero.

    Zero-live assertions are complemented by scanning the released memory for zero bytes. The 8,192-call stress route asserts zero live frame resources after each invocation.

    Native providers invoke a whole fixture call; the TypeScript reference invokes the Wasm endpoint for individual leaves. Therefore per-leaf call counts, traces, host-frame versus native-leaf release order and resource peaks are not asserted equal across these routes. Shared comparisons check only exact input/result carriers, acceptance, whether the endpoint was invoked and the applicable capacity category. This does not complete the full cross-engine settlement equality required by #162.

    Evidence and diagnostics

    Two new canonical schema identifiers:

    • semaprax.public-generic-typescript-settlement-corpus.v1

    • semaprax.public-generic-typescript-settlement-evidence.v1

    The explicit route identifier is typescript-host-owned-wasm-reference; the provenance field distinguishes production-template assembly from actual generator output. Existing native, carrier, descriptor and public metadata schemas were not reinterpreted. The prior module-artifact digest domain remains unchanged.

    No new SPX-* diagnostic code was allocated. The additions are closed, adapter-local reason unions in the existing typed error model. Tests compare reason/status identities, not prose or host exception messages. No diagnostic registry claim is needed for a code that was not allocated.

    Evidence binds corpus, production source/template digests, generated source and JavaScript digests, module digest, toolchain versions, execution flags and observed outcomes. It contains no input/result payload bytes, checkout-dependent paths or timings. Local command logs separately retain paths and duration for reproducibility.

    Replay never trusts a submitted result as its oracle and never executes submitted code. It reconstructs packages and observations from the trusted checkout, then checks canonical byte equality. Case/reason/order/source/module/result/trace/count and structural mutations, including reminted evidence, are rejected.

    New tests and execution matrix

    tests/fixtures/public-generic-typescript-settlement-v1/cases.json contains 108 settlement cases with independent expected event sequences and results. Recipes reuse existing native corpus helpers; equal shared values do not receive different carrier bytes per engine.

    settlement/runner.mjs executes all 108 cases. settlement/hostility.mjs adds 32 regression groups. settlement/variants.mjs adds 13 authenticated module subjects. Each set runs under both V8's Liftoff-only and TurboFan-only Wasm configurations: 306 checked records total. This is two configurations of Node/V8, not evidence from two independent browser engines.

    The existing generated-package round-trip suite runs at one, two and 256 leaves: 42 legacy tests. The full harness type-checks all 16 packages using TypeScript 5.8.3. Native comparison freshly compiles seven shared cases across C11/C++17 and O0/O2: 28 native executions and 56 comparisons with the two V8 configurations.

    The test suite includes exact payload and module bounds, first-over bounds, compound failures, memory-growth refusal, buffer detachment after memory.grow(0), stale/wrong-kind/forged/recreated handles, explicit close retry, post-commit export retry, physically zeroed memory, shape getters/proxies, private trust facts and repeated use.

    Twelve independently type-checked behavioral mutants must reach and fail their intended assertion. A compile error, unrelated exception or timeout is not counted as a successful mutation control. Eighty-one structural/reminting evidence controls must be refused.

    Added Rust test names (present, not executed here):

    • actual_generated_typescript_matches_all_assets_and_executes_settlement

    • legacy_wasm_fixture_is_unchanged_and_framing_capacity_is_explicit

    • reference_shape_has_exact_zero_max_and_first_over_leaf_bounds

    • generated_reference_runtime_preserves_its_no_ambient_authority_contract

    • authentication_and_cleanup_guards_are_emitted_from_fixed_assets

    • template_tests::every_fixed_asset_has_lf_crlf_equivalent_rendering

    The actual-generator bridge is registered in the existing Wasm integration harness on Unix, not a new integration-test binary. Linux CI also runs the TypeScript gate, replay and mutation controls with the already pinned Node/TypeScript tooling. CI configuration is supplied, not claimed to have executed remotely.

    Exact local commands and outcomes

    All successful commands below were run from the stage 5 tree. Full command arrays, exit codes and logs are in the validation ZIP. python resolves to /opt/pyvenv/bin/python in those records.

    Command | Exit | Observed result -- | -- | -- python scripts/public_generic_typescript_settlement.py --native --output /mnt/data/batch5-validation/final-main | 0 | 306 TS/Wasm records; 42 legacy tests; shared native execution; 81 evidence controls python scripts/public_generic_typescript_mutations.py --output /mnt/data/batch5-validation/final-mutants.json | 0 | Positive control and all 12 independently type-checked mutants checked python scripts/public_generic_typescript_settlement.py --repeats 8192 --output /mnt/data/batch5-validation/final-stress | 0 | Full 306-record gate; 8,192 same-provider calls in each V8 configuration python scripts/public_generic_settlement_evidence.py --sanitizers --stress --output /mnt/data/batch5-validation/native-provider | 0 | Prior native shared, compound/rollback, lifecycle and result-phase gates at O0/O2/ASan+UBSan; 124 evidence controls python scripts/public_generic_consumer_settlement.py --sanitizers --output /mnt/data/batch5-validation/native-consumers | 0 | All 639 prior native C11/C++17 records; 86 evidence/receipt controls python scripts/public_generic_consumer_mutations.py | 0 | All 11 prior native consumer mutation controls git diff --cached --check | 0 | Staged patch whitespace clean python -m py_compile scripts/public_generic_typescript_*.py scripts/public_generic_consumer_settlement.py | 0 | Python sources compile CI workflow parsing via yaml.safe_load | 0 | YAML parses; not a hosted workflow run

    The validation archive records the exact prior mutation command; unlike the shorthand above, its interpreter may be the literal python. No prior timeout is counted as a passing gate. The tool interface imposed a short shell-call ceiling, so final long gates were launched as local processes and their completed exit statuses were read during this same implementation response.

    Rust checks attempted and unavailable, each shell exit 127:

    cargo fmt --all -- --check
    cargo test --locked -p semaprax --test public_generic_wasm_adapter_v1 typescript_settlement
    cargo test --locked -p semaprax --lib public_generic_consumer::typescript_calling
    cargo clippy --locked -p semaprax --lib --bins -- -D warnings

    These are missing-tool outcomes, not evidence of a Rust test failure or a pass. The Rust generator/bridge additions and the earlier batches' Rust changes still require compilation, rustfmt, integration tests, clippy and the complete repository quality gate before merge.

    Clean application and replay

    Both application routes were exercised from the original uploaded tree with git apply --check --whitespace=error-all followed by application:

    1. Original → cumulative stage 4 → incremental stage 5.

    2. Original → cumulative stage 5.

    All 96 files changed by stages 1–5 are byte-identical between both routes and the working tree. Batch 5 itself changes 34 files. The fresh replay checkout path contains spaces to exercise file URL/path handling.

    All five fresh evidence artifacts reproduced byte-for-byte; all three replay commands exited 0. Fresh replay commands and their final outcomes are recorded in fresh-replay.status.json, fresh-native.status.json, and fresh-consumers.status.json in the validation archive. These freshly compile/type-check and execute trusted patched sources rather than reuse earlier binaries.

    Frozen artifacts and public-surface statement

    The stage 4 native provider body, native ABI declarations, native consumer settlement corpus and original settlement corpus remain byte-for-byte unchanged. Their exact hashes are in frozen-artifacts.json. The default old Wasm fixture builder remains the 256-page form; the configurable builder is explicit and tested in the new Rust bridge once Rust is available.

    Public generic support/publication remains unsupported/unpublished before and after this batch. No public export profile, manifest/project admission, diagnostic code range, compiler scalar/byte profile or native ABI is widened. Metadata-only consumers are not relabelled calling providers.

    Unfinished #162 acceptance items remain explicit: compiler-derived generic descriptors and endpoints; nested record/all Copy scalar fixtures; an actual compiler-produced Wasm provider ABI with in-module ownership and descriptor replay; Rust execution and the complete four-language matrix; target-neutral normalized failure/cleanup equivalence for all engines; full Rust/repository quality gates; fresh hosted evidence under #163. No Windows/MSVC, browser, cancellation, concurrent-use or fresh hosted-green claim is made.

  7. added a commit that references this issue on Sep 17, 2026
  8. wsdt commented on Sep 17, 2026

    @wsdt
    MemberAuthor

    Issue #162: stage 6 handoff

    Deliverables and exact baseline

    The incremental semaprax-issue-162-stage6.patch applies after delivered stages
    1–5. The cumulative semaprax-issue-162-cumulative-stage6.patch applies to the
    original uploaded semaprax-main.zip, baseline
    6fb62cfcedb948f7b32d5accb38aefdffaf0db68.
    Do not apply both variants. This is not a rebase onto a moving GitHub main.
    No unverified artifact from the interrupted stage-6 session was reused.

    Both application routes were executed with git apply --check --whitespace=error-all, then actually applied. All 4,119 tracked files in
    both resulting trees matched the tested working tree byte-for-byte. The new
    increment changes 17 files: 1,408 insertions and 28 deletions.

    SHA-256:

    • Incremental: a6ede4a432277a1eaab7abe52c95cc1c5bf8c0e1043c75801772266b7ddb4956

    • Cumulative: 92d77368dac24f5a849636bb982f41452c9d3fcc7dc4d38887bf87b3d14a65d1

    Implemented contracts and reasons

    Enforce the existing single-owner native profile

    src/public_generic_abi/native/provider_body.c now puts a thread identity and
    active-entry bit in one atomic admission word. Every shared-state public
    operation enters before touching registries, allocations, caller buffers or
    output pointers. All accepted exits publish their work before relinquishing
    entry. A foreign caller cannot clear an active owner's admission state.

    An owner remains pinned while any provider in this loaded artifact is live.
    Closing the last provider permits another thread to open a new provider;
    closing only one of multiple providers does not. Live-provider transfer,
    simultaneous endpoint execution, signals, cancellation and recovery after an
    owner thread exits with live resources are not admitted. The provider adds no
    thread creation, blocking wait, spin loop or endpoint retry.

    Threads receive monotonically minted identities rather than recycled OS IDs or
    TLS addresses. The explicit capacity is 2^62 - 1; the first new identity above
    it refuses, and existing identified owners retain the ability to settle. The
    previous 65,536-token provider/child identity pool is unchanged. Thread-capacity
    tests position a private test counter near the limit; they do not pretend to
    create quadrillions of physical threads.

    Isolate fault plans and settlement diagnostics

    Failure slots, trace, sticky status and overwrite counters are thread-local.
    One thread can no longer arm or clear another thread's injected failure. Test
    setters invoked by an active callback are no-ops, and forced settlement is
    refused. Trace reads return only the calling thread's retained evidence.

    The before/after reproducer was compiled and executed against retained stage-5
    source and final source:

    Observation | Stage 5 | Stage 6 -- | -- | -- Foreign provider-open status while the owner is live | 0 (accepted) | 7 (refused) Owner call after the foreign thread arms a failure | 11 (failed) | 0 (succeeded) Final live allocations | 0 | 0

    Fresh replay used a clean original tree with cumulative stage 5 plus the new
    incremental patch, rather than the development checkout. The seven artifacts
    are the two thread-build evidence files, three native-provider evidence files,
    C/C++ consumer evidence and TypeScript reference evidence. This is local
    execution/replay evidence, not hosted-green.

    Tool versions: Clang 17.0.0, GCC 14.2.0, Node 22.16.0, TypeScript 5.8.3, Python
    3.13.5 and Git 2.47.3. cargo, rustc and rustfmt availability checks returned
    127. Exact commands, directories, exit codes and raw logs are included in the
    archive's commands.json and individual status files; unsuccessful development
    iterations are retained separately and are not counted as final validation.

    Reproduction commands

    Run from the patched repository root:

    python3 scripts/public_generic_settlement_threads.py --sanitizers --output target/pg-thread-clang
    python3 scripts/public_generic_settlement_threads.py --sanitizers \
      --replay target/pg-thread-clang/native-thread-admission-evidence.json
    python3 scripts/public_generic_settlement_threads.py --cc gcc --thread-sanitizer --output target/pg-thread-gcc
    python3 scripts/public_generic_settlement_threads.py --cc gcc --thread-sanitizer \
      --replay target/pg-thread-gcc/native-thread-admission-evidence.json
    python3 scripts/public_generic_settlement_thread_mutations.py
    python3 scripts/public_generic_settlement_thread_mutations.py --cc gcc --thread-sanitizer
    python3 scripts/public_generic_settlement_evidence.py --sanitizers --stress --output target/pg-native
    python3 scripts/public_generic_consumer_settlement.py --sanitizers --output target/pg-consumers
    python3 scripts/public_generic_consumer_mutations.py
    python3 scripts/public_generic_typescript_settlement.py --native --output target/pg-typescript

    Still required on a provisioned Rust toolchain before merging:

    cargo fmt --all --check
    cargo test --locked --test public_generic_native_adapter_v1
    cargo test --locked --test public_generic_wasm_adapter_v1
    cargo test --locked --test documentation
    cargo test --locked --test module_size
    # Then the repository's complete current quality-gate selectors.

    CI now selects the new O0/O2 and ASan/UBSan matrix, fresh replay and eleven
    behavioral mutants inside the existing Linux native step. The actual-renderer
    bridge is part of the existing native integration harness. TSan remains an
    explicit provisioned selector; its absence fails that selector rather than
    silently returning green.

    Remaining issue #162 work

    PG-7 remains partial. This batch addresses native admission, re-entry, isolated
    fault selection and settlement protection, not the missing compiler-derived
    generic descriptor/endpoint pipeline, nested records and all Copy scalars, the
    compiled Wasm provider ABI, or the complete four-language cross-engine failure
    matrix. The TypeScript regression run still uses the explicitly labelled
    hand-assembled Wasm endpoint with host-owned bookkeeping. No unsupported/public
    profile was opened, no hosted-green was claimed, and no remote issue/branch was
    changed by this patch delivery.

  9. added a commit that references this issue on Sep 17, 2026
  10. wsdt commented on Sep 17, 2026

    @wsdt
    MemberAuthor

    Issue #162 — stage 7: a compiled Core Wasm reference-provider lifecycle

    Completion decision

    Do not close #162 or #229. PG-7 remains partial and public generic ownership remains unsupported/unpublished.

    This increment makes a material distinction in the evidence: the same existing C11 reference provider that ran natively now runs inside a real compiled Core Wasm module. Allocation, handle registries, copy-in, transfer, endpoint reversal, result staging, export and release execute in that module. The new Node host does not implement the provider or ownership bookkeeping.

    It is still the C11 reference fixture, not a Semaprax compiler-generated generic endpoint or the admitted public Wasm provider ABI. The generated TypeScript caller has not been switched to this private transport. This work is useful cross-target physical-settlement evidence, not a relabelling of a reference endpoint as checked generic-language support.

    Exact base and deliverables

    The base is the original supplied semaprax-main.zip (previously identified as GitHub 6fb62cfcedb948f7b32d5accb38aefdffaf0db68) with the delivered cumulative stage-6 patch applied. The current GitHub main was read and reports d663fbcfa43471f155509068a1a734079386b075, message feat: ref #162 progress nr 6. No unexamined later-main changes are folded into this patch.

    Local reconstruction tags/commits, not GitHub history:

    • original-upload: 6462cff3e6471281cbcac096d9d1789128cbae2d.

    • stage6: 9cc54a363b8d6dbb9aa135bf1618ddf4cac74cea.

    semaprax-issue-162-stage7.patch applies after stage 6. semaprax-issue-162-cumulative-stage7.patch applies to a clean copy of the original ZIP and contains stages 1–7. Choose one route, not both.

    Both routes were applied with strict whitespace checks to separate, freshly extracted copies of the original ZIP. All 4,130 tracked files in both resulting trees matched the stage-7 working tree byte-for-byte. This increment changes 18 files, with 1,764 added lines and 2 removed lines. It makes no changes under src/, no changes to the native public header/provider body, and no changes to the original case manifests. The cumulative patch naturally also contains earlier stages' production-source changes.

    # After all earlier increments:
    git apply --check semaprax-issue-162-stage7.patch
    git apply semaprax-issue-162-stage7.patch
    

    Alternative only: the original clean uploaded repository:

    git apply --check semaprax-issue-162-cumulative-stage7.patch
    git apply semaprax-issue-162-cumulative-stage7.patch

    Implementation and why it belongs here

    1. Reuse the provider and its existing assertions

    The owning production source remains src/public_generic_abi/native/provider_body.c; the public header and Rust reference renderer remain unchanged. scripts/public_generic_compiled_wasm.py reuses the composition already owned by public_generic_settlement_threads.py and the fixture factories/parsers from public_generic_settlement_evidence.py and public_generic_settlement_phases.py.

    The unchanged C allocation observer, shared probe, 61 additional failure regressions, lifecycle selectors, and 72 result-phase tests are compiled both natively and to Core Wasm. This is not a second implementation of native provider semantics and does not recreate expected outputs separately for each optimization.

    2. A private freestanding runtime, not a new published allocator

    tests/public_generic_wasm_adapter_v1/compiled_provider/runtime.c supplies the small set of C functions the existing source actually needs. It does not import WASI, a host allocator, callbacks, filesystem or network functions. Module imports are explicitly required to be empty.

    Its bounded heap is 40 MiB, has 4,096 live allocation records and 16-byte alignment. Address-ordered records support bounded first-fit allocation, arbitrary release order, gap reuse and scrubbing before invalidation. Searches are bounded linear metadata walks, not heap-byte scans. Separate allocator counters must finish at zero independently of the existing provider observer.

    The runtime's own tests exercise zero/SIZE_MAX/first-over-byte refusal, exactly 40 MiB, 4,096 live records plus record 4,097 refusal, alternating release, first-fit reuse, scrubbed returned bytes, reverse cleanup and final zero counters. These checks execute inside the module before the shared corpus. The mutation script also compiles the exact runtime bodies under native ASan/UBSan with symbols renamed so they cannot replace libc's or the sanitizer's allocator.

    This native sanitizer check is not Wasm ASan instrumentation, and does not claim that ASan tracks every suballocation inside the private static arena. The in-module tests, metadata invariants and explicit ownership counters provide the arena-specific checks.

    The module reserves 128 MiB initially, with a 192 MiB maximum and a 1 MiB stack. This includes static test logs, registries, scratch and heap storage. Zero live allocations is not a claim that Wasm memory pages or the static identity table disappear after release.

    3. A real multi-call module transport

    transport.c adds a deliberately private raw-host transport around the existing provider. Open, prepare, call, export, input/result release and close invoke the compiled C implementation. Return lanes contain a low-u32 status and high-u32 handle or required length. Handles are bounded, non-recycled indices into the provider's own identity inventory, not host-provided addresses.

    Only the dedicated scratch window is admitted for submitted pointer/length pairs. Subtraction-based checks avoid pointer-addition overflow; total bounds precede range checks. Descriptor and binding inputs are bounded to 65,536 bytes each. The maximum carrier is 16,777,216 + 2,056 bytes; scratch admits one extra byte so first-over-bound input can be submitted and rejected.

    Existing native status values 0–13 are unchanged. New test-transport-only values are 256 (range), 257 (closed option), and 258 (reset/selector would conceal existing state). No SPX diagnostic is allocated, no central diagnostic code is reused, and no production export or profile is widened.

    The scratch restriction is an API contract, not confidentiality against an embedder that deliberately writes arbitrary private locations through exported linear memory. Cross-instance raw integer aliases are also not an authenticated capability system. The test module is synchronous and unshared; it introduces neither cancellation nor concurrent endpoint execution.

    4. Direct host calls with independent checks

    host.mjs checks the exact export inventory and absence of imports. It executes all 22 existing shared cases through actual module calls, using the original canonical carrier factory. Mutating every ingress byte after preparation must not change the separately owned input. Results are compared byte-for-byte before becoming evidence digests.

    A new canonical manifest separately pins 34 hostile transport subjects: overflow/private/end-plus-one ranges, total bounds, descriptor/binding mismatch, malformed/trailing input, unknown/wrong-provider/wrong-kind/stale handles, input reuse, provider recreation, refused close with children, memory growth and view reacquisition, independent copy-out, non-writing short/failed export, retry without execution, explicit release failure, result staging failure, sticky primary plus secondary cleanup, and reset/option refusal.

    These cases are driven through the module. The existing generated TypeScript wrapper is neither used as a provider nor claimed to have migrated to the new transport.

    5. Exact comparison, portable evidence and replay

    New schema/scope identifiers:

    semaprax.public-generic-compiled-reference-corpus.v1
    semaprax.public-generic-compiled-reference-evidence.v1
    compiled-c11-reference-in-core-wasm-not-compiler-generic-v1

    The evidence reuses existing canonical JSON and domain-separated digest framing. It binds source composition, trusted manifests, original reference descriptor/binding, compiled artifact and engine/route identity. Rows have closed, route-specific fields, strict non-boolean integer domains, fixed case/engine order, exact outcomes and zero-resource assertions. Missing or reordered rows are not accepted as a smaller successful corpus.

    Across native and Wasm, only peak_bytes is target-local: pointer/size_t bookkeeping differs between 64-bit native and 32-bit Wasm. It remains in the evidence rather than being dropped. Logical allocation and handle peaks, statuses, traces, release order and final counts must agree. All four Wasm compiler/tier combinations agree on every measured field, including their byte peaks.

    Evidence contains no payloads, addresses, opaque handles or timings. C assertion logs may intentionally contain this repository's nonsecret fixture bytes. Replay never loads a submitted executable: it recompiles trusted local source, regenerates the same inputs, executes every selected case, and requires byte-identical evidence. Exact artifact replay requires the same compiler/configuration, not cross-host binary identity.

    6. Actual Rust renderer bridge and CI

    tests/public_generic_wasm_adapter_v1/compiled_provider.rs invokes the existing native Rust renderer and passes its output to the external gate. The gate requires exact byte equality before compilation. This test is explicitly ignored by default because unrelated platforms do not necessarily provision a Wasm cross-linker; the new Linux CI step provisions lld and selects this exact test with --ignored --exact.

    The CI step also runs the full stress/sanitizer reference gate, independent replay and semantic mutation tests. Missing tools or missing cases fail rather than quietly skip. No CI job was run remotely or claimed green in this session. The Rust bridge is implemented but was not compiled/executed locally because Cargo/rustc/rustfmt are absent.

    Observed validation

    Local tools: Clang 17.0.0 and wasm-ld/LLD 17.0.0 from the same Swift LLVM build, Node v22.16.0, Python 3.13.5, Linux x86-64. GCC 14.2.0 was inventoried but not used as an additional stage-7 engine. No local Windows, macOS, browser or independent non-V8 Wasm engine coverage is claimed.

    Command or check Exit Observed result
    python scripts/public_generic_compiled_wasm.py --sanitizers --stress --output /mnt/data/stage7-validation/final 0 966 evidence rows, 55 tampering controls; full native/reference-Wasm comparisons passed
    Same command from a fresh cumulative-patch checkout, adding --replay /mnt/data/stage7-validation/final/compiled-reference-evidence.json --output /mnt/data/stage7-validation/fresh-replay 0 Fresh recompilation and execution reproduced every evidence byte
    python scripts/public_generic_compiled_wasm_mutations.py --output /mnt/data/stage7-validation/fresh-mutations from the fresh checkout 0 Private runtime ASan/UBSan tests and all 9 compiled semantic mutants passed their controls
    Python syntax compilation for all three new scripts 0 Passed
    node --check tests/public_generic_wasm_adapter_v1/compiled_provider/host.mjs 0 Passed
    CI YAML parse and changed-Rust-module line-budget checks 0 Passed; not actionlint, Rust compilation or hosted execution
    git diff --check 0 Passed
    Both strict-whitespace patch routes on fresh original ZIP extractions 0 All 4,130 tracked files match
    cargo fmt --all -- --check 127 Cargo/rustfmt unavailable; not validated
    Exact Cargo bridge command below 127 Cargo unavailable; not compiled/executed
    cargo clippy --locked --all-targets --all-features -- -D warnings 127 Cargo unavailable; not validated
    sh scripts/quality.sh full 127 Stops in quality-route at missing Cargo before building; full quality gate is not green

    The 966 rows are 318 native (106 each at O0, O2 and native ASan/UBSan) and 648 Wasm (162 each at C O0/O2 crossed with Liftoff-only/TurboFan-only). Per engine, the existing recorded C routes contain 22 shared cases, 72 result-phase cases and 12 lifecycle cases. Each Wasm engine additionally has 22 raw-transport and 34 hostile-transport rows. The 61 additional C failure assertions and allocator self-test execute too, but are not inflated into persisted row counts.

    Each Wasm optimization is independently compiled twice and required to produce identical module bytes. V8 tier-up is disabled. Exact 16 MiB/256-leaf result-phase success and 8,192-call same-provider stress remain in the executed original corpus.

    The 9 deliberately broken providers first compiled and passed an unchanged positive control, then compiled successfully after mutation and failed their intended assertion: identity instead of reversal, masked explicit release, lost sticky primary, partial failed export, admitted private ranges, lost bound precedence, leaked provider, falsely successful reset while live, and forged partial-result handles. Compiler errors, timeouts and unrelated Wasm traps do not qualify as successful negative controls.

    Reproduction commands

    From a checkout with this patch and provisioned Clang with wasm32 support, wasm-ld, Node and Python:

    python3 scripts/public_generic_compiled_wasm.py --sanitizers --stress --output target/pg-compiled-reference
    python3 scripts/public_generic_compiled_wasm.py --sanitizers --stress \
      --replay target/pg-compiled-reference/compiled-reference-evidence.json
    python3 scripts/public_generic_compiled_wasm_mutations.py
    

    Requires the repository's Rust toolchain and locked dependencies:

    cargo test --locked -p semaprax --test public_generic_wasm_adapter_v1
    compiled_provider::actual_native_renderer_executes_in_core_wasm -- --ignored --exact
    cargo fmt --all -- --check
    cargo clippy --locked --all-targets --all-features -- -D warnings
    sh scripts/quality.sh full

    Omitting --stress omits only the higher repeated-invocation lifecycle case. Omitting --sanitizers omits the extra native sanitizer engine; it does not substitute a weaker engine and label it sanitized. Both Wasm optimizations and both V8 tiers remain mandatory.

    Remaining ticket work

    The completion gap is not hidden by the new compiled fixture. Still required are compiler-derived generic descriptors/endpoints and trusted source admission, nested/multi-level owned records, every admitted Copy scalar, the actual public Wasm provider ABI, all generated callers using that boundary, complete model/caller/engine trace and peak equivalence, and the necessary Rust/full-quality gates. Fresh hosted evidence belongs to #163 and public support remains governed by PG-9.

    The useful next implementation target is checked generic endpoint/descriptor wiring into the real compiler/provider path, not another claim that the reference fixture itself completes the language ABI.

  11. wsdt commented on Sep 22, 2026

    @wsdt
    MemberAuthor

    Landed on main in 3ffc27b (Refs #162/#229). This advances the checked-subject side of PG-7 without claiming closure. The native C11/C++17/Rust generated-caller settlement harnesses now derive one real generic owned-record endpoint from Semaprax source, generate Public Generic Descriptor v1, independently replay it through the retained-program verifier, and carry the exact verified descriptor, native binding, concrete instance, cleanup-plan, and leaf identities into the external compiled settlement driver. The physical endpoint remains the explicitly labelled reversal fixture; compiler-derived endpoint execution, full all-engine equality, and hosted evidence remain open. Focused local evidence: cargo check --locked -p semaprax --lib; project public_generic_wasm_provider 5/5; native consumer_settlement 3/3. No GitHub Actions run was dispatched.

  12. wsdt commented on Sep 22, 2026

    @wsdt
    MemberAuthor

    Pushed a second large implementation tranche in 75df647 (following 3ffc27b).

    This tranche adds the compiler-owned, deterministic, zero-import public-generic Wasm provider artifact and its module-owned lifecycle: exact descriptor/binding replay, scratch management, provider/input/result handles, checked HIR invocation, canonical carrier validation/encoding, two-pass result export, and deterministic release/close operations. Mixed monomorphic plus concrete-generic reachable lowering is included, and ProjectRevision now exposes reproducible artifact generation.

    Focused evidence:

    • cargo check --locked -p semaprax --lib
    • cargo test --locked -p semaprax --test project -- public_generic_wasm_provider --test-threads=1 (6 passed)
    • cargo test --locked -p semaprax --test public_generic_wasm_adapter_v1 compiler_provider_artifact -- --test-threads=1 (2 passed)
    • cargo test --locked -p semaprax --lib public_generic_consumer::typescript_calling -- --test-threads=1 (18 passed)
    • cargo fmt --all -- --check
    • git diff --check

    No GitHub Actions run was triggered manually. Keeping #162 open: this lands the compiler/provider foundation, not the entire issue settlement matrix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyDetailed enough for a bounded implementation agentarea:abiPublic interfaces, descriptors, carriers, compatibility, and ABI workarea:nativeNative C ABI, provider, C/C++ consumer, and sanitizer workarea:runtimeExecution, lifecycle, ownership settlement, and runtime behaviorarea:wasmCore Wasm, Wasm host, memory, and generated TypeScript/Wasm workkind:testAssurance, evidence, hostile testing, or CI gatepriority:p0Critical path; blocks the milestone

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions