[P1] Create a provider adapter SDK and deterministic model-provider conformance suite

Summary

Provide one narrow adapter interface that lets external model providers participate in model.invoke without changing Semaprax core semantics. Adapters must pass a shared conformance and hostility suite before being described as supported.

Why this matters competitively

Neam’s provider breadth is convenient but risks provider concepts leaking into the language. Semaprax should gain breadth through interchangeable adapters whose behavior is normalized, bounded, and evidence-compatible.

Current state at the audit baseline

The planned core boundary is provider-independent and explicitly injected.
Scripted deterministic providers already support bounded tests.
No public stable provider adapter SDK or conformance suite exists.
Credentials and network authority must remain host-owned.

Audit baseline: ae25c6a49dc09ec4613c7a6f52a27daa69dcd3f3 (2026-09-11). Recheck main, this issue, and the owning specifications before changing code; later commits may have superseded individual details.

Required outcome

Publish an adapter contract and generated or hand-implementable SDK for Rust initially, with deterministic fixture, recording/replay, and at least two live provider adapters maintained outside core semantics. Conformance must cover streaming, limits, cancellation, retries, error normalization, usage, redaction, and no ambient authority.

In scope

Provider-neutral request/event/result interfaces.
Credential and endpoint injection by host.
Streaming event normalization.
Timeout/cancellation and uncertainty reporting.
Usage/cost observation.
Structured-output/schema support declarations.
Deterministic fixture and recording/replay adapters.
Adapter conformance report and version negotiation.

Explicitly out of scope

Putting provider API keys in .spx source or Agent deployment documents.
Supporting every provider in the first release.
Normalizing away meaningful uncertainty.
Allowing adapters to execute tools directly.
Letting provider-specific fields become mandatory core-language fields.

Starting points in the repository

model.invoke effect contract
AgentDeployment model binding
generated SDK patterns under current Project Agent transports
network/TLS provider abstractions
model receipt and budget modules

Use rg to find the current owners of the named schema, diagnostic, operation, or test before editing. Do not assume a path is still authoritative merely because it is listed here.

Implementation sequence

Specify adapter ABI/API v1 with methods for capability declaration, request start, event polling/streaming, cancellation request, settlement, and usage observation.
Require adapters to declare supported structured-output modes, streaming, token accounting source, cancellation semantics, retryable failure classes, endpoint policy, and maximum limits.
Keep credentials in opaque host configuration. Deployment may select a provider identity/profile but must not contain secret bytes.
Implement deterministic scripted and recorded-response adapters first. They establish conformance without network access.
Implement one live adapter end to end, then a second provider to prove the interface is not provider-shaped. Keep both in separate packages/modules.
Build a shared hostile server/fixture that emits partial UTF-8, malformed event order, oversized chunks, slow trickle, disconnects, duplicate completion, contradictory usage, and late data after cancellation.
Generate a conformance report binding adapter version, provider profile, test corpus, observed capabilities, and nonclaims.
Make support explicit per adapter/version and fail closed on unsupported model/provider capability combinations.

Failure and security cases that must be considered

The first provider can accidentally dictate the supposedly neutral API.
Adapters can hide ambient endpoint, proxy, environment, or credential lookup.
Cancellation APIs vary and often do not prove server cancellation.
Provider-native tool calls can bypass Proposal decoding.
Usage fields may be absent or inconsistent.

Required tests and evidence

Deterministic fixture adapter passes with no network.
Recording/replay reproduces event sequence and decoded Proposal without dispatch.
At least two provider adapters pass the same core conformance corpus.
Hostile event-order, size, timeout, cancellation, and usage cases normalize correctly.
No adapter can invoke typed tools or mint authorization.
No credential, header, proxy, or ambient endpoint appears in receipts or source semantics.
Capability negotiation rejects unsupported combinations before dispatch.

Acceptance criteria

A stable provider-neutral SDK exists.
Two materially different providers can implement it without core-language changes.
Adapters are separately versioned, tested, and supported.
Credential/network authority remains explicit and host-owned.
All adapter output still passes compiler-derived Proposal decoding.

Dependencies and sequencing

[P0] Add a provider-independent model.invoke effect to the source-native Agent runtime
[P1] Enforce model call, token, context, latency, retry, failover, and cancellation budgets
[P1] Add model-call receipts, replay, redaction, and billing reconciliation

Rules for the implementing agent

Read AGENTS.md, docs/COMPLETION-MATRIX.md, docs/ROADMAP.md, docs/QUALITY-GATES.md, and every owning versioned specification named above before editing.
Prefer extending the existing semantic object, operation table, diagnostic catalogue, profile dispatcher, or evidence harness. Do not create a parallel source of truth.
Preserve canonical source, persistent identities, revision binding, fail-closed replay, deterministic rendering, and the separation between evidence and authority.
A local test is local evidence. A hosted run is hosted evidence. A generated artifact is not a support or publication decision. Record exactly what ran and do not widen claims.
Any new wire format, ABI, descriptor, carrier, evidence document, or compatibility surface requires an explicit versioned schema, independent decoder/replay, hostile-input tests, bounds, and nonclaims.
Never grant ambient filesystem, process, network, model, signing, payment, commit, publication, or registry authority merely because a declaration, receipt, test, or proof exists.
Keep changes reviewable. Split implementation into commits that each preserve the existing test suite, and explain any known-answer update field by field.
Do not weaken or delete a failing regression to obtain green CI. Fix the cause or narrow the admitted feature with a stable diagnostic.

Definition of done

The implementation, tests, documentation, status wording, diagnostics, and evidence agree on one exact scope. Targeted tests pass, the applicable repository quality gate passes, hostile cases fail with the documented stable reason, and no unrelated public surface or authority is widened.

docs(agent-runtime): specify the Live Invocation Contract v1

feat(model-budget-policy): admit model calls, retries and failover ag…

feat(provider-adapter-sdk): add adapter ABI v1 and deterministic conf…

Landed on main as bb10e81c (+ cc03e2e8 formatting): a provider adapter SDK v1 with a deterministic conformance and hostility suite. Not closing — the issue's Required outcome names "at least two live provider adapters", and no live adapter exists. See the split below.

Shipped (evidence: local re-runnable, `cargo test --locked -p semaprax --lib provider_adapter_sdk` → 30 passed; `--doc provider_adapter_sdk` → 1 passed)

New module src/provider_adapter_sdk/ (adapter, capability, conformance, fixture_adapters, hostile, report, tests) plus docs/PROVIDER-ADAPTER-SDK-V1.md.

Implementation step	State	Test
1. Adapter ABI v1 — capability declaration, start, event poll, cancel, settle, usage	done	`ProviderAdapter`/`AdapterEvent`/`AdapterPoll` in `adapter.rs`
2. Declared: structured-output modes, streaming, token-accounting source, cancellation semantics, retryable classes, endpoint policy, limits	done	`capability::tests`, 6 tests
3. Credentials stay in opaque host config	done	`a_credential_held_by_an_adapter_never_appears_in_its_conformance_report`
4. Deterministic scripted + recorded-replay adapters	done	`the_same_recording_replayed_twice_produces_byte_identical_conformance_reports`
5. Two live provider adapters	not done — HUMAN_BLOCKED	—
6. Hostile fixture corpus	done	6 named-code rejection tests + disconnect test

Negotiation refuses unsupported combinations before dispatch: the refusal tests drive a PanicsOnStartAdapter, so a regression that dispatched anyway would panic rather than merely fail an assertion. Negotiation also refuses, unconditionally, an adapter that declares its own ambient endpoint or lists an unsafe failure class as retryable — the module never imports AuthorizationGrant, EffectiveModelBudget, or ProviderPolicy, so no adapter can widen a budget or self-authorize.

Six hostile behaviours are each rejected by an exact code — ADAPTER-DUPLICATE-COMPLETION, ADAPTER-CONTRADICTORY-USAGE, ADAPTER-OVERSIZED-RESPONSE, ADAPTER-LATE-AFTER-CANCEL, ADAPTER-EVENT-AFTER-COMPLETION — and each test additionally asserts that none of the other hostility codes appear, so a catch-all rejection cannot pass the suite. A mid-stream disconnect is asserted to surface as a normalized provider error and not as a hostile rejection, which is what stops the driver from miscataloguing every failure. Partial-UTF-8 and one-byte-at-a-time delivery are asserted byte-identical to whole-chunk delivery, and the reassembled output is decoded through the real compiler-derived CompiledInteractionSchema — with a tampered-field negative control proving the refusal comes from the compiled schema, not from this SDK.

Explicitly not claimed

Offline only. No network, no provider endpoint, no key was contacted. The "two materially different adapters" that pass one corpus are two offline transport shapes (batch and streaming), which proves the interface is not shaped by one transport — it does not prove it is not shaped by one vendor.
Cancellation is best-effort; nothing here proves a server stopped work.
Usage figures are adapter-reported, not billing-verified.
A passing conformance report is local evidence. It is not a support or publication decision.

Remaining for closure

Step 5, and the parts of conformance only a live adapter can exercise (real timeouts, real disconnects, real usage accounting). That needs network egress and two sets of provider credentials — human-gated, so it is not something this run can fake or infer. Tracked follow-up to come.

Pushed 6b6d691 to main (starting at 02287bd): separate Responses and Messages provider protocol adapters over an explicitly injected HostHttpStreamTransport.

Both emit SDK events from actual vendor SSE shapes, retain partial provider token observations with unknown cost, refuse provider-native tools, normalize transport uncertainty, and require protocol completion plus host stream end. Credentials, endpoints, TLS and proxy policy remain in the host transport. Input bounds cover total wire bytes, pending frames, event inventory, decoded response size, model labels and output token configuration. Invalid zero token limits now refuse rather than silently raising the requested limit. SSE framing uses bounded scanning and a single compaction per chunk.

65 focused SDK tests passed locally, including the same neutral conformance request across both adapters, actual compiler-derived schema/bridge admission and wrong-field rejection, hostile event order/truncation/size/usage, cancellation/uncertainty, and bounds before host access. 48 policy tests also passed. No full suite, hosted CI, paid/live calls or new dependencies.

Issue remains open for concrete network-client host integration, credentialed end-to-end vendor execution and adapter support/version publication decisions. These are implemented vendor wire adapters with local scripted HTTP streams, not evidence of two live supported providers. Log: .agent-logs/0913-provider-corrected.log.

Pushed to main in 4103e23f1471fd0689c9dca01d147e42d3a53e85.

Added native HttpsBufferedTransport for the existing Responses and Messages adapters. The host supplies the HTTPS origin, TLS trust, and sensitive auth header; redirects and ambient proxies are disabled. Requests, buffered response bytes, timeout and emitted chunks are bounded. Post-dispatch failures remain conservatively uncertain.

This transport performs the network exchange synchronously during start and then exposes buffered chunks; it does not claim incremental network polling or remote cancellation. No remote provider call or credential was used. Remaining acceptance includes live-provider conformance/operational evidence and true incremental/cancellable network streaming where required. Keeping this issue open.

Local evidence: 4 native HTTP tests, including an actual loopback TLS POST and redirect/oversize/timeout refusals. The combined library/binary compile passed. No full quality profile or hosted CI run was performed for this batch, per the requested implementation-first workflow.

Accepted bbf9a95 → c02b49e on main (implementation a644d37).

Provider conformance corpus V2 now refuses concrete request/response limits before start, missing completion, inconsistent final bytes (including an empty Delta), oversized terminal bytes, and regressing final usage. Eventless non-streaming batch settlement remains supported. The report schema stays V1 and explicitly names the new corpus identity.

25 focused SDK tests passed, including the restored existing malformed-order regression and direct-driver no-start controls. Both real vendor protocol implementations passed the same neutral conformance request in a separate focused test (offline scripted transport). Logs .agent-logs/0913-sdk-final.log and .agent-logs/0913-sdk-neutral.log. No full/hosted/live-provider execution was claimed.

#181 remains open for its broader SDK/public/provider acceptance requirements and genuine opt-in provider evidence. This changes local conformance enforcement, not support/publication status.

Audit against current `main`: contract satisfied, one real gap found and closed in `9fb4f58a`

Re-audited the whole adapter contract against code and executing tests.
src/provider_adapter_sdk/ (~9.7k lines) already implements essentially all of
it. One genuine gap existed; it is now fixed.

Per-requirement audit

Requirement	Status	Evidence
Adapter ABI: capability declaration, start, poll, cancel, settlement	SATISFIED	`adapter.rs:146-181` (`ProviderAdapter`); no-ambient-authority enforced by a `compile_fail` doctest at `adapter.rs:21-28`
Capability declaration (structured output, streaming, token accounting, cancellation semantics, retryable classes, endpoint policy, limits)	SATISFIED	`capability.rs:111-134`. Notably there is no `CancellationSemantics::Guaranteed` variant, by design (`capability.rs:76-86`)
Credentials host-owned; deployment carries no secret bytes	SATISFIED	`tests/agent_runtime_v1/agent_deployment_v1.rs:357` `neither_document_can_carry_a_credential_or_an_environment_reference` — injecting `api_key`/`token`/`credentials_env` fails with `SPX-G554`/`SPX-G552`; closed-schema enforcement at `src/agent_deployment/documents.rs:268`
Deterministic scripted + recorded/replay adapters	SATISFIED	`fixture_adapters.rs` (`ScriptedBatchAdapter`, `ScriptedStreamingAdapter`, `RecordedReplayAdapter:236-273`); replay-without-dispatch proven by `source_bridge/tests/recording.rs::source_factory_records_ordered_events_and_replays_the_typed_proposal_without_dispatch`
Two materially different adapters on one corpus	SATISFIED	`vendor/openai_responses.rs`, `vendor/anthropic_messages.rs`, driven through the identical corpus at `vendor/tests.rs:276` `both_protocols_pass_the_same_neutral_conformance_request`
Conformance report (version, profile, corpus, observed capabilities, nonclaims)	SATISFIED	`report.rs:58-72`; nonclaims always attached via `conformance.rs:486-493` `standard_nonclaims()`; digest/determinism tests `report.rs:156-191`
Fail closed on unsupported capability combination	SATISFIED	`capability.rs:182-217` `negotiate()`; refusal-before-dispatch proven with `PanicsOnStartAdapter` (`fixture_adapters.rs:274-303`) at `tests.rs:199,217`
No ambient endpoint/proxy/env/credential authority	SATISFIED by test	`hostile.rs:166-172` `ambient_endpoint_capabilities()` unconditionally refused at `capability.rs:186-190` (test `capability.rs:262-278`); `tests.rs::a_credential_held_by_an_adapter_never_appears_in_its_conformance_report`
Cancellation uncertainty not normalized away	SATISFIED	`cancel()` returns nothing (`adapter.rs:174-180`); both vendor adapters classify `Cancelled` as `AttemptOutcomeClass::Uncertain` (`openai_responses.rs:294`, `anthropic_messages.rs:377`), which `retry_is_permitted` refuses to treat as safe-to-retry; `NONCLAIM_CANCELLATION_BEST_EFFORT` states it in every report
Usage absent or inconsistent	SATISFIED	`AdapterUsage` fields are `Option`; `unknown_usage()` fallbacks (`openai_responses.rs:455`, `anthropic_messages.rs:530`); contradiction detection `conformance.rs:168-188`
Adapters cannot execute tools or mint authorization	SATISFIED	structural — `adapter.rs` never imports `AuthorizationGrant`; `compile_fail` doctest `adapter.rs:21-28`; provider-native tool calls rejected pre-decode (`vendor/tests.rs:150,171`)
Version negotiation / adapter identity pinning	was PARTIAL — now SATISFIED	see below

Step 6 hostile fixture — all eight cases confirmed individually

Hostile case	Fixture	Executing test
Partial UTF-8 across a chunk boundary	inline	`tests.rs:649 a_multi_byte_utf8_character_split_across_a_chunk_boundary_assembles_identically_to_one_chunk`
Slow trickle (1 byte/poll)	inline	`tests.rs:699 one_byte_at_a_time_delivery_assembles_identically_to_one_chunk`
Malformed event order	`hostile.rs:121`	`tests.rs:561`
Oversized chunks	`hostile.rs:78`	`tests.rs:384`
Disconnect mid-stream	`hostile.rs:145`	`tests.rs:584` (negative control: classified as a normalized provider error, not a rule violation)
Duplicate completion	`hostile.rs:28`	`tests.rs:348`
Contradictory usage	`hostile.rs:49`	`tests.rs:366`
Late data after cancellation	`hostile.rs:99`	`tests.rs:402`

None is a module name masquerading as coverage; each asserts via
assert_no_other_hostile_code_appears that the refusal names exactly its own
violation code rather than a catch-all.

The one real gap, now closed

source_bridge.rs:957-972 refuses a mismatched adapter label with
source.model_adapter_identity after constructing the adapter and strictly
before start(). A repo-wide grep found that string at exactly one site —
its own definition — with no assertion anywhere. The pinning was implemented
and structurally sound but completely unguarded by a regression.

Fixed in 9fb4f58a: a_relabelled_adapter_is_refused_before_start_on_every_pinned_field
varies each pinned field of the triple on its own (adapter_identity,
adapter_version, provider_profile), so dropping any one from
adapter_matches fails the test rather than silently admitting a differently
labelled provider. Each case asserts the adapter is constructed exactly once,
never started, the recorded terminal is adapter_identity_refused, and the
diagnostic carries its stable code.

cargo test --locked -p semaprax --test agent_runtime_v1 -- a_relabelled_adapter_is_refused_before_start_on_every_pinned_field
test result: ok. 1 passed; 0 failed

Baseline unchanged: cargo test --locked -p semaprax --lib provider_adapter_sdk:: → ok. 76 passed; 0 failed.

Nonclaims — read these before treating this as production support

No live provider evidence exists anywhere in this codebase. Both vendor
adapters are exercised only against ScriptedTransport (in-memory,
vendor/tests.rs:37-51) and a local 127.0.0.1 listener (vendor/http.rs:525).
Neither has ever contacted api.openai.com or api.anthropic.com. No
credentials exist on this machine. This is local, offline, proof-only
evidence, exactly as NONCLAIM_OFFLINE_ONLY states in every generated report.
Cancellation is never claimed as server-confirmed — that is load-bearing
in the classification, not decoration.
A hosted support/publication decision has not been made, and the report
type explicitly refuses to make one (NONCLAIM_NOT_A_SUPPORT_DECISION).

Closing: every in-scope implementation item is satisfied and the one genuine
gap is fixed. Exercising an adapter against a real endpoint requires
human-supplied credentials and is an operational act, not an implementation
gap — the capability invariant forbids this crate from acquiring that authority
on its own.

Navigation Menu

Repository navigation

[P1] Create a provider adapter SDK and deterministic model-provider conformance suite #181

Summary

Why this matters competitively

Current state at the audit baseline

Required outcome

In scope

Explicitly out of scope

Starting points in the repository

Implementation sequence

Failure and security cases that must be considered

Required tests and evidence

Acceptance criteria

Dependencies and sequencing

Rules for the implementing agent

Definition of done

Shipped (evidence: local re-runnable, `cargo test --locked -p semaprax --lib provider_adapter_sdk` → 30 passed; `--doc provider_adapter_sdk` → 1 passed)

Explicitly not claimed

Remaining for closure

Audit against current `main`: contract satisfied, one real gap found and closed in `9fb4f58a`

Per-requirement audit

Step 6 hostile fixture — all eight cases confirmed individually

The one real gap, now closed

Nonclaims — read these before treating this as production support

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[P1] Create a provider adapter SDK and deterministic model-provider conformance suite #181

Description

Summary

Why this matters competitively

Current state at the audit baseline

Required outcome

In scope

Explicitly out of scope

Starting points in the repository

Implementation sequence

Failure and security cases that must be considered

Required tests and evidence

Acceptance criteria

Dependencies and sequencing

Rules for the implementing agent

Definition of done

Activity

wsdt commented on Sep 12, 2026

Shipped (evidence: local re-runnable, cargo test --locked -p semaprax --lib provider_adapter_sdk → 30 passed; --doc provider_adapter_sdk → 1 passed)

Explicitly not claimed

Remaining for closure

wsdt commented on Sep 13, 2026

wsdt commented on Sep 13, 2026

wsdt commented on Sep 13, 2026

wsdt commented on Sep 18, 2026

Audit against current main: contract satisfied, one real gap found and closed in 9fb4f58a

Per-requirement audit

Step 6 hostile fixture — all eight cases confirmed individually

The one real gap, now closed

Nonclaims — read these before treating this as production support

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Shipped (evidence: local re-runnable, `cargo test --locked -p semaprax --lib provider_adapter_sdk` → 30 passed; `--doc provider_adapter_sdk` → 1 passed)

Audit against current `main`: contract satisfied, one real gap found and closed in `9fb4f58a`