Skip to content

Commit b35139d

Browse files
committed
Add authoritative compact profiles and source streaming integration
1 parent 852aa90 commit b35139d

19 files changed

Lines changed: 1559 additions & 32 deletions

‎CHANGELOG.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ format: `Unreleased` then release buckets, grouped by impact.
88

99
## Unreleased
1010

11+
- Add authoritative task-context, Project API, candidate-diff, and Agent graph
12+
compact profiles with regeneration-bound replay (#201). Extend Rust embedding
13+
negotiation, cancellation, and the external consumer (#203). Connect the
14+
source Proposal grammar to incremental decoding and an explicit per-attempt
15+
provider adapter factory with bounded iterative context (#178).
16+
1117
- Connect the streaming Proposal decoder to the provider adapter and generic
1218
live-kernel seams (#178), and compose token/cost/call policy with the live
1319
work-budget hook (#179). Extend the embedding facade with opaque in-memory

‎docs/ARCHITECTURE.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,13 @@ Proposal decoder before the kernel's independent compiled-schema decode and
133133
authorization. `model_budget_policy/live_hook` composes explicit request-bound
134134
host token/cost estimates with the existing work hook, retaining conservative
135135
charges in memory without claiming generic-journal crash recovery.
136+
`provider_adapter_sdk/source_bridge` adapts the ordinary source `ProposalSource`
137+
with explicit fresh adapter factories, bounded canonical retained context, and
138+
`streaming_proposal_decode/source` admission against the authoritative source
139+
Proposal schema. It does not advertise durable checkpoint policy.
140+
`compact_semantic_projection/selected` wraps existing task, public API, candidate,
141+
and Agent graph producers and independently regenerates selected content on
142+
replay; encoded evidence cannot replace the owning compiler object.
136143
`embedding_api/project_session` exposes caller-owned source admission and the
137144
existing semantic service through opaque handles and canonical reports. It
138145
retains the service's staged refresh and replay boundaries; caught stateful

‎docs/COMPACT-SEMANTIC-PROJECTION-V1.md‎

Lines changed: 16 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -85,23 +85,25 @@ wraps `crate::graph::agent_context_v2_json` for one seed symbol (`profile`
8585
`"agent-context-v2"`, root the seed symbol). The wire format itself does not
8686
restrict `profile` to this list -- a future profile is one more
8787
`ProjectionSource` variant in [`encode_profile`], with no wire-format
88-
change -- but only these two are implemented and tested here.
88+
change. [`ProjectionSelection`](../src/compact_semantic_projection/selected.rs)
89+
adds authoritative profiles whose replay regenerates their owning kernel before
90+
accepting decoded content: goal-aware `semantic_task_context::compile`, the
91+
explicitly scoped Project v8 owned-data public API descriptor, a candidate's
92+
`semantic_delta_catalog(expected_candidate)`, and one compiled Agent
93+
definition's canonical graph. These profiles bind respectively to the graph
94+
revision, retained Project revision, candidate revision plus expected candidate
95+
digest, and AgentGraph digest; they do not accept a self-rehashed envelope as
96+
replay evidence.
8997

9098
## Deliberately out of scope
9199

92-
Issue #201 asks for a much larger surface: a "task context" profile wired
93-
to issue #197's goal-aware compiler, an "API surface" profile, a
94-
"candidate diff" profile, an "Agent definition" profile, byte/token
95-
benchmarks across multiple *model* tokenizer versions, version
96-
*negotiation* (as opposed to version *refusal*, which this module does
97-
implement), and CLI/MCP exposure. None of that is in this module. This is
98-
one narrow, honestly-scoped slice -- the dictionary/index encoding,
99-
canonical ordering, dual wire form, and lossless-with-detection replay
100-
bullets of #201's implementation sequence, applied to the two
101-
selected-view sources this module already had unchanged engines for --
102-
matching the precedent `semantic_task_context` and `semantic_embedding` set
103-
for shipping one demonstrable slice of a large issue rather than an
104-
unverifiable broader claim.
100+
Issue #201 additionally asks for byte/token benchmarks across multiple *model*
101+
tokenizer versions, version *negotiation* (as opposed to version *refusal*,
102+
which this module does implement), and CLI/MCP exposure. Those are not in this
103+
module. The selected profiles reuse existing authoritative producers and the
104+
unchanged bounded dictionary/text/binary codec. Selected replay adds exact
105+
regeneration equality (`SPX-Z910` on disagreement) to envelope integrity and
106+
profile/root/revision checks. It grants no candidate commit or runtime authority.
105107

106108
## Honesty bar: what "lossless" means here, exactly
107109

‎docs/EMBEDDING-API-V1.md‎

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,8 @@ Issue #203's "In scope" list, mapped to real code, as of this tranche:
4747
| Candidate validate/replay | `src/project/candidate/**` implements this for the Project workspace transaction path. | [`ProjectSession::validate_candidate_v2`](../src/embedding_api/project_session.rs) and [`ProjectSession::replay_candidate_v2`](../src/embedding_api/project_session.rs) return canonical authority-free reports over the active generation. |
4848
| Deterministic interpreter execution for admitted profiles | `interpreter`/`hosted_interpreter` exist. | [`execute_entry_source(capability, unit_name, source, options, cancellation)`](../src/embedding_api/execution.rs) executes only the existing prepared, zero-argument `i64` entrypoint profile. It requires [`ExecutionCapability`](../src/embedding_api/execution.rs), caller bounds, and a cooperative cancellation token; no source-native host effect receives a provider or ambient fallback. |
4949
| Explicit provider/capability injection | Done, for the vector-embedding effect only, by `semantic_embedding::EmbeddingCapability`/`EmbeddingProvider` (see above). | `check_source` needs no capability because checking is pure and effect-free; this bullet is satisfied for *this* operation by construction (nothing to inject authority into), not by adding an unnecessary capability type. |
50-
| Memory/resource ownership and cancellation | Not documented for any embedding surface. | Stateless analysis calls hold no resource across calls. A `ProjectSession` owns only in-memory compiler cache/service state and normal Rust drop releases it. [`ExecutionCancellation`](../src/embedding_api/execution.rs) provides monotonic cooperative cancellation for the bounded interpreter call. Project-service operations currently provide no cancellation kernel, so the session facade does not falsely advertise one. |
51-
| Version/feature negotiation | Not present for any embedding surface. | [`EMBEDDING_API_VERSION`](../src/embedding_api.rs) and `EmbeddingApiVersion::is_compatible_with` exist and are tested against both a matching and two non-matching major versions. |
50+
| Memory/resource ownership and cancellation | Not documented for any embedding surface. | Stateless analysis calls hold no resource across calls. A `ProjectSession` owns only in-memory compiler cache/service state and normal Rust drop releases it. [`EmbeddingCancellation`](../src/embedding_api.rs) pre-cancels Project opening/refresh and samples bounded context before work and before returning a successful report; it does not promise an unsafe mid-refresh interruption. [`ExecutionCancellation`](../src/embedding_api/execution.rs) cooperatively cancels the bounded interpreter call. |
51+
| Version/feature negotiation | Not present for any embedding surface. | [`EMBEDDING_API_VERSION`](../src/embedding_api.rs), `EmbeddingApiVersion::is_compatible_with`, `require_compatible`, and `negotiate_features` check major/minimum minor and exact operation/profile names. Unknown features fail closed with `SPX-EMB004`. |
5252
| Thread-safety and reentrancy contract | Not documented. | `check_source` takes no shared or mutable state. `ProjectSession` makes no `Sync`, clone, reentrancy, or simultaneous-refresh promise; refresh/candidate validation take `&mut self`. A caught stateful-operation panic poisons the handle, requiring callers to drop and reopen it from explicit bytes. |
5353

5454
## The `CheckOutcome` contract
@@ -207,15 +207,20 @@ option types or traversal state.
207207

208208
## Compatibility policy
209209

210-
`EMBEDDING_API_VERSION` (currently `1.6.0`; `1.1.0` after `format_source`,
210+
`EMBEDDING_API_VERSION` (currently `1.7.0`; `1.1.0` after `format_source`,
211211
`1.2.0` after `graph_source`, `1.3.0` after `context_source`, and `1.4.0`
212212
after `context_v2_source`, then `1.5.0` after `execute_entry_source`, and
213-
`1.6.0` after the Project session facade) names
213+
`1.6.0` after the Project session facade, and `1.7.0` after explicit
214+
version-refusal and pre-cancelled request entry points) names
214215
this Rust surface's own version,
215216
independent of any checked SEMAPRAX program's semantics.
216217
`EmbeddingApiVersion::is_compatible_with(requested_major)` returns `true`
217-
only when `requested_major` equals this build's `major`; a differing major
218-
version is refused rather than silently assumed compatible.
218+
only when `requested_major` equals this build's `major`; `require_compatible`
219+
turns a mismatch into the stable `SPX-EMB002` diagnostic rather than silently
220+
assuming compatibility. `EmbeddingCancellation` reports `SPX-EMB003` only
221+
when it was observed before a request starts, or (for pure context) before its
222+
successful report is returned; it never claims the graph/project kernels were
223+
interrupted mid-operation.
219224
`version_negotiation_accepts_matching_major_and_refuses_a_different_one`
220225
tests both a match (`1`) and two refusals (`0` and `2`). Within one major
221226
version, each existing function's accepted inputs and each existing outcome
@@ -306,3 +311,19 @@ diagnostics, retained warnings, and API-major rejection. Run it with
306311
It has its own Cargo workspace and lockfile. Its dependency is a local path
307312
to the compiler checkout; this is local consumer evidence, not validation
308313
of an installed or published registry package.
314+
315+
### Exact feature negotiation and cancellation
316+
317+
`negotiate_features(major, minimum_minor, required_names)` validates against
318+
`SUPPORTED_FEATURES`, with at most 32 names of at most 128 bytes each. A differing
319+
major or unsupported minimum minor returns `SPX-EMB002`; an unknown name or
320+
capacity violation returns `SPX-EMB004`. The returned read-only `EmbeddingFeatures`
321+
is availability information. Execution still requires `ExecutionCapability`.
322+
There is no implicit profile fallback or authority in this result.
323+
324+
Both `context_source_with_cancellation` and
325+
`context_v2_source_with_cancellation` sample the monotonic host signal before
326+
analysis and before returning a successful report. Project open and refresh
327+
sample only before starting; cancellation never reports an uncommitted state
328+
after the persistent service has adopted a generation. A pre-cancelled refresh
329+
leaves its handle usable and its old revision intact.

‎docs/PROVIDER-ADAPTER-SDK-V1.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -180,3 +180,36 @@ decode, model-policy reservation and journal receipt projection together;
180180
malformed first-chunk refusal with no further reads or authorization; and
181181
pre-dispatch/pending deadline, cancellation, schema and hostile byte-cap cases.
182182
These are offline injected adapters, not evidence for two live vendors.
183+
184+
## Ordinary source proposal streaming bridge
185+
186+
`source_bridge::StreamingSourceProposalAdapter` implements the existing source
187+
`ProposalSource` trait using a fresh, explicitly injected `SourceAdapterFactory`
188+
for each attempt. It uses the source `CompiledAgentProposalSchema`, distinct
189+
from the generic interaction schema, and refuses source revision or schema
190+
drift before adapter creation. The source driver retains authorization and
191+
retry decisions. This bridge advertises no checkpoint policy: it does not
192+
invent durable provider attempt identities or journal intents.
193+
194+
The generated `semaprax.source-adapter-prompt.v1` JSON carries exact task bytes
195+
as hex, task budget, source revision, turn, attempt, remaining iterations,
196+
canonical lifecycle state/observation, prior effect bytes as hex or null, prior
197+
rejection or null, and the compiler proposal schema. Retained carriers are
198+
checked before rendering with a conservative 65,536-byte upper bound and depth
199+
64; prior effect/rejection and complete prompt are bounded at 65,536 bytes.
200+
These bounds do not alter the lifecycle's canonical retained-value encoding.
201+
202+
An optional host cancellation token and absolute `InvocationClock` deadline
203+
are checked before factory creation, before start, and before each poll.
204+
The bridge allows at most 10,000 polls; blocking host callbacks remain
205+
cooperative. It checks each chunk before the next poll, cancels on early
206+
refusal, and requires one Completed event followed by settlement containing
207+
exactly the streamed bytes. Duplicate completion, post-completion deltas/usage,
208+
negative or regressing known usage, malformed framing, and semantic decoder
209+
refusal cannot produce a proposal. Missing usage remains unknown.
210+
211+
Local fixture tests cover real compiled-schema admission, iterative context,
212+
fresh attempts, early stream refusal/cancellation, predispatch cancellation,
213+
deadline, identity and capacity refusals, and settlement/usage contradictions.
214+
This is an in-process adapter seam, not evidence of a network provider or a
215+
durable source checkpoint implementation.

‎docs/STREAMING-PROPOSAL-DECODE-V1.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,11 @@ work, since every scanner step is a single O(1) transition over at most
153153

154154
## Determinism across chunk boundaries
155155

156+
`SourceProposalStreamDecoder` applies this same bounded framing to the
157+
source-live route's distinct `CompiledAgentProposalSchema` grammar. It does
158+
not translate that grammar into the interaction-value wire: after a terminal
159+
LF it calls the source schema's own `decode` and returns `DecodedProposal`.
160+
156161
The single most valuable property this document proves: the same total
157162
byte sequence produces the identical final outcome no matter how it is
158163
split into chunks. `ProposalStreamDecoder` has no chunk-boundary-dependent
@@ -210,9 +215,10 @@ generated-client integration.
210215

211216
`src/streaming_proposal_decode.rs` (`ProposalStreamDecoder`, `PushOutcome`,
212217
`StreamRefusal`, the `STREAM-*` closed vocabulary, and the `MAX_STREAM_*`
213-
bounds) plus its `tests` submodule
214-
(`src/streaming_proposal_decode/tests.rs`) is the complete reference
215-
implementation this document describes. Focused gate:
218+
bounds), its generic tests, and the additive `source` decoder and source tests
219+
form the reference implementation. The source decoder delegates final
220+
admission to `CompiledAgentProposalSchema` and preserves the existing source
221+
Proposal wire format. Focused gate:
216222

217223
```sh
218224
cargo test --locked -p semaprax --lib streaming_proposal_decode

‎examples/embedding-api/README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@
22

33
This standalone Rust program is an external host using only the public
44
`semaprax::embedding_api` facade. It checks caller supplied source, formats the
5-
canonical projection, renders the semantic graph, preserves a successful-check
6-
warning, rejects malformed input with `SPX-P101`, and refuses incompatible API
7-
major versions.
5+
canonical projection, renders the semantic graph and bounded context, performs
6+
explicitly authorized deterministic execution, opens an opaque in-memory
7+
Project session from embedded bytes, preserves a successful-check warning,
8+
rejects malformed input with `SPX-P101`, and refuses incompatible API majors.
89

910
Run it from this directory with:
1011

‎examples/embedding-api/src/main.rs‎

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,12 @@
44
//! the embedding API is published. It intentionally does not access compiler
55
//! internals, the CLI, or the filesystem.
66
7-
use semaprax::embedding_api::{check_source, format_source, graph_source, EMBEDDING_API_VERSION};
7+
use semaprax::embedding_api::{
8+
check_source, context_source_with_cancellation, execute_entry_source, format_source,
9+
graph_source, negotiate_features, open_project_session, EmbeddingCancellation, ExecutionCancellation,
10+
ExecutionCapability, ExecutionOptions, ProjectSessionInput, ProjectSourceInput,
11+
EMBEDDING_API_VERSION,
12+
};
813

914
const SOURCE: &str =
1015
"module host.demo;\n\n@id(\"host.demo.main\")\nfn main() -> i64\n{\n 42\n}\n";
@@ -13,6 +18,12 @@ fn main() {
1318
assert!(EMBEDDING_API_VERSION.is_compatible_with(1));
1419
assert!(!EMBEDDING_API_VERSION.is_compatible_with(0));
1520
assert!(!EMBEDDING_API_VERSION.is_compatible_with(2));
21+
EMBEDDING_API_VERSION
22+
.require_compatible(1)
23+
.expect("the example is written for embedding API v1");
24+
25+
negotiate_features(1, 7, &["context-v1", "project-session-v1", "deterministic-i64-entry-v1"])
26+
.expect("the host supports every required operation profile");
1627

1728
let checked = check_source("host-demo.spx", SOURCE);
1829
assert!(
@@ -66,4 +77,50 @@ fn main() {
6677
.diagnostics
6778
.iter()
6879
.any(|diagnostic| diagnostic.code == "SPX-S103"));
80+
81+
let cancellation = EmbeddingCancellation::new();
82+
let context = context_source_with_cancellation(
83+
"host-demo.spx",
84+
SOURCE,
85+
"host.demo.main",
86+
&Default::default(),
87+
&cancellation,
88+
);
89+
assert!(context.ok, "context failed: {:?}", context.diagnostics);
90+
assert!(context.context_json.is_some());
91+
92+
let execution = execute_entry_source(
93+
&ExecutionCapability::grant("the host admits deterministic demo evaluation"),
94+
"host-demo.spx",
95+
SOURCE,
96+
&ExecutionOptions::default(),
97+
&ExecutionCancellation::new(),
98+
);
99+
assert!(execution.ok, "execution failed: {:?}", execution.diagnostics);
100+
101+
let project_input = ProjectSessionInput::new(
102+
include_str!("../../../examples/calculator-project/semaprax.toml"),
103+
vec![
104+
ProjectSourceInput::new(
105+
"src/app.spx",
106+
include_str!("../../../examples/calculator-project/src/app.spx"),
107+
)
108+
.expect("bounded Project source"),
109+
ProjectSourceInput::new(
110+
"src/core.spx",
111+
include_str!("../../../examples/calculator-project/src/core.spx"),
112+
)
113+
.expect("bounded Project source"),
114+
ProjectSourceInput::new(
115+
"src/tests.spx",
116+
include_str!("../../../examples/calculator-project/src/tests.spx"),
117+
)
118+
.expect("bounded Project source"),
119+
],
120+
)
121+
.expect("canonical embedded Project input");
122+
let (session, opened) = open_project_session(&project_input);
123+
assert!(opened.ok, "Project open failed: {:?}", opened.diagnostics);
124+
let session = session.expect("successful Project open returns its opaque handle");
125+
assert!(session.workspace_revision().is_some());
69126
}

‎src/compact_semantic_projection.rs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,12 @@ use crate::diagnostic::Diagnostic;
114114
use crate::digest_hex::LowerHex;
115115
use crate::graph::{self, AgentContextV2Options};
116116

117+
mod selected;
118+
119+
pub use selected::{
120+
encode_selected, replay_selected_binary, replay_selected_text, ProjectionSelection,
121+
};
122+
117123
/// Schema identity of this module's compact wire envelope.
118124
pub const SCHEMA: &str = "semaprax.compact-semantic-projection.v1";
119125
/// Wire format version this build reads and writes. A decoder refuses any

0 commit comments

Comments
 (0)