in-toto profile: the eval-result predicate and the SVR export

Status: PROPOSED. Discussed at in-toto/attestation#565 and submitted as PR #575, which is open, not merged. Not standardized. The predicateType lives in a vendor namespace until (and unless) it is registered upstream. Nothing here changes the native receipt or what it proves — see NON_CLAIMS.md.

The v0.2 field table below mirrors the submitted spec as revised on 2026-09-28 (the upstream file at git blob 0996522; its schema and field sections are those of blob 46a33e7). When the two differ, the PR is the source of truth and this page is the one that is wrong; a byte-for-byte copy of the submitted file lives in docs/upstream/eval-result.md.

This page answers, for a first-time reader, three questions in a few minutes:

1. What is the predicate? A privacy-preserving in-toto Statement for an ML eval result.
2. What does it NOT prove? Authenticity and integrity of a claim, never its semantic truth.
3. Why is neither test-result nor an SVR alone enough? See the two sections below.

The problem

in-toto has a generic test-result/v0.1 predicate and a Summary Verification Result (SVR) predicate. Neither models an ML evaluation, which has three properties a generic test does not:

• a metric threshold and a pass/fail against that threshold (not just a status),
• the need to keep the model and dataset private while still proving the claim, and
• optional references to supporting evidence, such as an external signed receipt (and, later, an external time anchor).

Why test-result/v0.1 is not enough

The community test-result predicate (predicateType https://in-toto.io/attestation/test-result/v0.1) carries:

• result: one of PASSED | WARNED | FAILED,
• configuration: a list of ResourceDescriptors,
• passedTests / warnedTests / failedTests: string lists.

It has no native field for a metric, a comparator, a threshold, a sample size, or a privacy- preserving commitment. You can map a receipt onto it (proofbundle still offers that export via to_test_result_statement, stuffing the metric into a descriptor's annotations), but the eval's core facts — which metric, which threshold, met by how much — become unstructured annotations that no generic verifier understands. The threshold semantics are lost.

Why an SVR alone is not enough

An SVR (https://in-toto.io/attestation/svr/v0.1) is a verifier's summary: verifier.id, timeCreated, and a list of passing property strings. It is excellent for saying "this verifier checked these things and they held", and proofbundle emits one (see below). But an SVR is intentionally lossy: it records that properties held, not the metric, the threshold, the sample size, or the commitments. It also has no FAILED form — it lists only passing properties (a PASSED|FAILED verdict would be a VSA, which we deliberately do not implement). So an SVR is a good summary layer on top of a receipt, not a replacement for the detailed eval-result predicate.

The two compose: emit the eval-result Statement for the detail, and an SVR for the one-line "a verifier confirmed this passed" summary. Both are derived from the same verified receipt.

verify_svr_dsse and proofbundle svr --verify refuse an SVR statement whose predicate lacks one of the four members svr/v0.1 marks required (verifier, verifier.id, timeCreated, properties) or carries one of them in another JSON type, and one whose verifier.id is no URI (TypeURI) or whose timeCreated is no RFC 3339 date-time with an offset (Timestamp); the command answers such a statement as malformed, exit 2. export_svr_dsse refuses a time_created in another form. The v0.1 extension field verifier.policy stays optional, and unknown members stay unread.

The eval-result/v0.2 predicate

predicateType: https://b7n0de.com/attestation/eval-result/v0.2 (vendor namespace; the revised #575 draft, see the migration below). Statement _type is the standard https://in-toto.io/Statement/v1; the DSSE payloadType is the canonical application/vnd.in-toto+json. Written by proofbundle intoto --predicate-version v0.2 --evaluator <URI> and intoto.export_eval_result_v02_dsse; verified by verify_eval_result_dsse(..., expected_predicate_type=EVAL_RESULT_V02_PREDICATE_TYPE), which adds the shape below to the verdict (classify_eval_result_v02_predicate). Fields (lowerCamelCase; time fields are speaking RFC-3339):

field	meaning
evaluator.id	required, the URI of the party that ran the evaluation and produced the claims: the evaluation role, distinct from the verification role of an SVR. The signer is authenticated by the DSSE envelope; the same party MAY hold more than one role, so an evaluator that is also the signer is not refused. The emitter has no default: proofbundle records a result, it does not run the evaluation
evaluatedAt	when the eval ran (from the signed receipt)
suite	{name, version}, both required
claims[]	{metric, comparator, threshold, passed}. passed is the producer's signed threshold verdict, not a recomputable relation: proofbundle discards the exact score after the comparison, so without a disclosed value a generic consumer can authenticate the verdict but cannot recompute it
sampleSize	n
commitments	per private identity, model and/or dataset, each {alg, value, salted} — a salted commitment, NOT an artifact hash. Each commitment entry MUST set salted to true; false is refused
model, dataset	a public identity: a ResourceDescriptor that MUST carry digest (a DigestSet over the real content)
assuranceLevel	an issuer-declared assurance claim: self_attested | third_party | reproduced | enclave_attested. The predicate does not corroborate it; external corroboration belongs in evidence
subjectProfile	which subject profile produced the subject (below)
preRegistration	optional {alg, value} — present only if the receipt carries a prereg hash
evidence[]	optional ResourceDescriptors for supporting material. Each entry MUST carry digest, and SHOULD carry mediaType and one of uri or downloadLocation. The digest identifies the referenced artifact; with content present it identifies the decoded bytes, and the verifier refuses a content whose MD5, SHA-1, SHA-2 or SHA-3 digest differs from the one stated, or that the running Python cannot recompute. An internal Merkle root is not a substitute for the digest of that artifact
harness	optional {name, version} plus an optional digest (DigestSet) for consumers that need to bind the exact artifact that produced the result. A harness carrying only name and version stays conforming. The digest binds identity only and asserts nothing about the harness's detection performance

One identification each. For each of the model and the dataset, exactly one of commitments.<x> and the predicate-level descriptor <x> MUST be present. The verifier inspects both locations before it decides, refuses both and neither, and requires the present representation to satisfy its own rules (a commitment with salted: true, a descriptor with a usable digest). Statement subject entries and evidence references never count, in either direction. With --subject-profile public-model, the emitter writes the model as a descriptor with the subject's name and digest instead of a commitment; the dataset stays a commitment. The draft says openly that this check is not monotonic under removal of a recognized field (a statement with both representations is refused, and removing one makes it conforming); that is a question for the maintainers and is implemented as written.

What the receipt's evidence digest covers. proofbundle intoto --predicate-version v0.2 reads the receipt file once and writes one evidence entry for it: name eval-receipt, mediaType application/json, the --receipt-uri if given, and digest.sha256 over the file's exact bytes. A proofbundle receipt is one JSON object that carries the signed payload, the signature and the merkle tree together, so the digest covers the signature and changes with it. The receipt's internal Merkle root is not written into the entry at all, neither as the digest nor beside it. The library writes no evidence entry unless the caller supplies one (receipt_evidence(receipt_bytes) builds it from the bytes the caller holds); nothing is derived from a Merkle root.

Written beyond the draft, allowed by its parsing rules. subjectDigestNote stays, for the receipt profile only: that subject digest is a binder that names no file, and the note says so. anchors is not written in v0.2: the draft scoped it out, and an external anchor now travels as an evidence entry with its own digest.

The eval-result/v0.1 predicate (released, still verified)

predicateType: https://b7n0de.com/attestation/eval-result/v0.1. Emitted and signed by released versions and still the CLI default. Its predicate is verified under its own contract: signature, content root, type, and every member the table below does not mark optional, with both parts of commitments and of suite; a statement that lacks one does not verify (predicate_claim_ok is False), and neither does one whose verifier is no object carrying verifier.id as a URI, whose evaluatedAt is no RFC 3339 date-time with an offset, whose subjectProfile is not one of the profiles below, or whose claims is an empty array. The URI and the date-time are read as v0.2 reads them, and to_eval_result_predicate refuses a claim whose timestamp is in another form. No v0.2 shape applies to it (predicate_shape_ok is None). Since 2026-10-17 the three in-toto verifiers (eval-result, test-result, SVR) also require the Statement v1 layer under every predicate: _type is https://in-toto.io/Statement/v1, and subject is a non-empty list whose every entry carries a digest, because in-toto matches subject artifacts by digest alone. A digest under a hash function the package knows must have that function's length in lowercase hex; under any other algorithm name it must be a non-empty string. A statement without that layer names no artifact and does not verify (statement_ok is False). Every export function writes it, so this rule alone does not change the verdict on an eval-result, test-result or SVR attestation one of them wrote; other checks of 6.5.0 can, for a time_created, a claim timestamp or a test-result url a caller passed to an earlier exporter in another form (the CHANGELOG of 6.5.0). The public builders to_eval_result_statement and to_test_result_statement write the subject a caller hands them: a statement a caller built with them from a digest that does not have that form, and signed itself, verified before 6.3.1 and does not verify since (measured with the builders of 6.3.0: a sha256 digest in capitals, of 63 characters or with a sha256: prefix). The three verifiers read the DSSE payload in either base64 alphabet the in-toto envelope allows, standard (RFC 4648 §4) or URL-safe (§5), padded and canonical in both; a payload without its padding does not verify, and the exports write the standard alphabet. The same rule judges the Statement of a decision, outcome, relation statement, verification summary, run ledger and trust pack in the Python verifier, and the Rust verifier applies it on its relation, relation-statement and trust-pack-threshold paths, since it stands in the one Statement oracle (_statement_payload). One verdict changes there: the emitters of those six wrote a caller's subject_sha256 override verbatim, so a statement a released version wrote with an override that is not 64 lowercase hex characters, one in capitals for example, no longer verifies, and the emitters now refuse such an override. Fields:

field	meaning
verifier.id	the emitter/verifier TypeURI (replaced by evaluator.id in v0.2)
evaluatedAt	when the eval ran (from the signed receipt)
suite	{name, version}
claims[]	one or more {metric, comparator, threshold, passed}. passed is the producer's signed threshold verdict, not a recomputable relation: proofbundle discards the exact score after the comparison, so without a disclosed value a generic consumer can authenticate the verdict but cannot recompute it. In v0.1 passed is optional: a claim without it verifies and carries no verdict, and a present passed must be a boolean. A consumer that needs a verdict reads v0.2, where every claim carries passed as a boolean
sampleSize	n
commitments	{model, dataset}, each {alg, value, salted:true} — a salted commitment, NOT an artifact hash
assuranceLevel	an issuer-declared assurance claim: self_attested | third_party | reproduced | enclave_attested. The predicate does not corroborate it; external corroboration belongs in separately referenced evidence
subjectProfile	which subject profile produced the subject (below)
preRegistration	optional {alg, value} — present only if the receipt carries a prereg hash
receipt	optional {schema, merkleRootB64} — binds to the external signed receipt (replaced by evidence[] in v0.2)
harness	optional {name, version} plus an optional digest (DigestSet) for consumers that need to bind the exact artifact that produced the result. A harness carrying only name and version stays conforming. The digest binds identity only and asserts nothing about the harness's detection performance

anchors is not a field of this predicate. External time anchors were drafted in #565 and deliberately scoped out of #575: they are not eval-specific and belong as a shared optional field in their own discussion. Earlier revisions of this page listed them; that was wrong and is corrected here.

Parsing rules follow in-toto Statement v1: matching is on the subject digest alone; unknown predicate fields are ignored by consumers; and the Monotonic Principle applies — a verifier denies unless a valid attestation exists. Absence rule: unless a field says otherwise, the absence of an optional field means only that no claim is made for it — a consumer MUST NOT infer or synthesize a default from absence.

Subject profiles — what the subject IS

in-toto matches on the subject digest, so the subject must be chosen deliberately. --subject-profile:

• receipt (default): the subject is the receipt itself — the digest is a sha256 binder over the receipt's commitments + Merkle root + timestamp. It binds the attestation to the receipt without revealing the model. Use this when the model/dataset stay private.
• public-model: the subject is a disclosed public model artifact; you supply its real sha256 (--subject-sha256) and a name. Use this when the model is public and you want the attestation to match on the model's own digest.
• release-gate: the subject is a release artifact (an image, a wheel, a service digest) whose deployment is gated on the passing eval — the "deploy only if the eval passed" hook, the natural attach point for SLSA/policy. You supply the artifact's sha256.

Reading the export with foreign tools

Measured with the export of 6.1.0 against five foreign readers (Z225, tools/intoto_external on the branch claude/intoto-external, commit 13d8faaa60ec1f2d133d94bcea015965cb43a9c7): the in-toto attestation bindings for Go (v1.2.0) and Python (0.9.3), go-securesystemslib 0.11.1, securesystemslib 1.5.1, cosign 3.1.3 and GUAC 1.1.0. Every signature verified under a foreign DSSE verifier. Five findings followed; one was ours and is fixed, four are what a tool does with a conforming statement, and the format does not bend to them. The in-toto rules cited are from in-toto/attestation v1.2.0, spec/v1/README.md and spec/v1/envelope.md.

• F3, the DSSE keyid (fixed). The envelope layer says a keyid SHOULD be included for each signing key; the export wrote none, so securesystemslib raised KeyError: 'keyid' and GUAC could not find a key. By default an envelope the package signs now carries the signer's OpenSSH SHA256 fingerprint (dsse.openssh_sha256_keyid), the form go-securesystemslib's dsse.SHA256KeyID derives and sigstore's key providers compare, when the signer exposes its Ed25519 public key. Pass keyid= to write another, or keyid="" to write none. The keyid is not signed, and the single-key verifiers of the export select no key by it, so among well-formed envelopes their verdict does not change with its value; the trust pack selects each of its several keys by keyid, by design. A keyid that is not well-formed JSON text, a lone surrogate, makes the envelope malformed, as any other field would. Envelopes signed before this default keep the bytes they were signed with and carry no keyid: the archived agent-review receipts under receipts/agent_review, the fixture tests/fixtures/kette_147/receipt.json and the eval-result v0.1 statements that released versions emitted, under tests/fixtures/eval_result_v0_1. The calls the suite makes to every public function, recorded as seeds in tests/fixtures/public_surface_seeds.json, hold the test receipts as the suite passed them, and those carry no keyid either. The sdist ships all four as regression inputs, and a reader that requires a keyid refuses them. The Sigstore bundle vectors under tests/fixtures/sigstore_bundle hold envelopes as GitHub and the cli/cli test data published them, some without a keyid, and synthetic ones that test its absence; DSSE makes the keyid optional.
• F1, the top-level contentRootAlg (a tool limit). The field declares the content-root algorithm inside the signed bytes (ADR 0002). The in-toto rules say "Producers MAY add extension fields to any JSON object" and "Consumers MUST ignore unrecognized fields". The default strict parse of both bindings refuses the statement, and cosign parses strictly and then says could not parse predicate. Parsed with DiscardUnknown (Go) or ignore_unknown_fields (Python), the statement passes Validate. A reader that needs cosign can verify the DSSE signature with any DSSE verifier and parse the statement leniently.
• F2, the test-result payloadType (a tool limit). The test-result export signs with application/vnd.in-toto.test-result+json, the form spec/v1/envelope.md allows next to application/vnd.in-toto+json; SPEC section 7b pins it, and the verifier refuses any other type, as a type-confusion defense. cosign accepts only the second form. The eval-result and SVR exports use the second form and pass that check.
• F4, GUAC ingests no attestation of these predicate types (a tool limit). With a keyid, GUAC verifies the signature and stops at no document parser registered for type: ITE6: it files every in-toto statement of a predicate type it does not model as ITE6 and registers no parser for that. A SLSA provenance statement under the same key was ingested in the same run.
• F5, cosign verify-attestation matches the subject against an image digest (a tool limit). Only the release-gate profile names an image. For receipt and public-model subjects, use cosign verify-blob-attestation with the subject's own bytes or its digest.

Policy questions (for a relying party)

Before you trust an eval-result attestation for a decision, answer:

1. Whose key signed it, and do you trust that key for this claim? (The DSSE key; in v0.2 evaluator.id names who ran the eval, which need not be the signer.)
2. What is the assuranceLevel? self_attested is producer testimony; third_party / reproduced / enclave_attested are stronger — do you require one of them?
3. Which subject profile is it, and is that the thing you meant to gate on? A receipt subject binds to a private model; a release-gate subject binds to the artifact you deploy.
4. Is there a pre-registration, and is it anchored? Without an external anchor, ordering is producer-clock testimony only (external time anchors are a proposed experimental extra).
5. Does the threshold and metric match your policy? The attestation proves the signed claim; you decide whether that bar is the right one.
6. Do you require an SVR / specific passing properties (e.g. PROOFBUNDLE_SAMPLE_ROOT_VALID)?

What it does not prove

Everything in NON_CLAIMS.md applies unchanged. In short: authenticity and integrity of a claim, never its semantic truth, fairness, safety, or generalization.

Added to the submitted spec on 2026-08-07 and repeated here because it is easy to assume otherwise: the predicate does not establish that the evaluation harness or grader is fit for purpose, or that it has any particular detection performance. Binding a harness digest pins which artifact ran, not how well it detects.

The consequences of that non-claim can be asymmetric, and the direction depends on which class the detector counts as positive. Where detected positives are evidence of capability, missed positives can understate the subject. Where passed: true depends on the absence of detected failures, missed failures can instead yield a passing verdict although the failures occurred. The attestation authenticates either verdict without establishing the harness's detection capability. The submitted spec carries this as its own paragraph in ## Non-claims; it deliberately says can be rather than is, because the predicate never fixes which class a detector counts.

Migration: eval-result v0.1 → v0.2 (6.3.0)

The revised #575 draft changed the predicate in three places. The vendor type cannot keep its version while its shape changes, because v0.1 statements were emitted and signed by released versions and a consumer that keys on the type must never read them under the new rules. So the revised shape is its own type, https://b7n0de.com/attestation/eval-result/v0.2.

v0.1	v0.2	reason in the draft
verifier.id (the tool)	evaluator.id, required, set by the caller	the evaluation role; verifier stays reserved for the verification role of an SVR, the signer is authenticated by the envelope
commitments.model and commitments.dataset, both required	per identity exactly one of a commitment (private, salted: true) and a predicate-level ResourceDescriptor with digest (public)	salted commitments are only needed for a private identity
receipt: {schema, merkleRootB64}	evidence[], each entry with the digest of the artifact it names	a generic reference to supporting material instead of an emitter-specific block; an internal Merkle root is not a substitute

Compatibility is a requirement, and this is the evidence for it (gate G2). Statements of https://b7n0de.com/attestation/eval-result/v0.1 keep verifying under their original rules:

• the regression set is tests/fixtures/eval_result_v0_1/corpus.json: 14 v0.1 envelopes, 5 of them fixtures from release 6.1.0 (emitted by the published wheel) and 9 own reconstructions from the source at the release tags v2.0.0, v2.1.0, v3.0.0, v3.3.0, v3.6.0, v4.0.0, v5.0.0, v5.1.0 and v6.0.0, each with its origin; no statement from a foreign tool is known;
• for every entry, for the same entry under another key, and for six negative variants, the verdict of this version equals the verdict the released 6.1.0 verifier gives (recorded in the corpus, not recomputed); every entry is judged under the v0.1 rules (predicate_shape_ok is None) and returns its statement unchanged;
• the v0.1 emitter still writes the statement and the signature of the five wheel envelopes byte for byte; since the DSSE keyid default the envelope also names its signing key in an unsigned keyid, the signer's OpenSSH SHA256 fingerprint, and without that field each envelope equals the wheel's;
• the verifier dispatches on predicateType: the same predicate relabelled as v0.2 is refused by the revised rules, and a revised predicate labelled v0.1 is judged by the old rules, which ask for the members v0.1 requires and for no shape of v0.2;
• verify_eval_result_dsse still expects v0.1 by default, so a v0.2 statement under the default call is refused for its type and a caller that reads v0.1 fields never receives a v0.2 predicate;
• proofbundle intoto still writes v0.1 unless --predicate-version v0.2 is given, because an existing consumer's default is never switched silently (MIGRATION_EVAL_PREDICATE.md). Switching the default is a release decision, not part of this change. --verify accepts both types, each under its own rules.

Measured when the corpus was made: the receipt statement of every tag from v2.1.0 to v6.0.0 is byte identical to the 6.1.0 wheel's jcs-sha256-v1 statement, and v2.0.0's to its legacy-sortkeys-json-v0 statement; the v0.1 wire did not change across these releases for these inputs.

Readings of the draft

Where the draft text admits more than one reading, the reading implemented is named here; each is a finding for the draft, not a settled question. Line numbers are those of the upstream file (the mirror docs/upstream/eval-result.md without its header; in the mirror they are seven lines further down).

• A1, salted: false (lines 114-117). Settled by the revision: each commitment entry MUST set salted to true. Implemented: anything but true is refused.
• A2, a present null (lines 123-126). Whether "commitments": {"model": null} counts as a present representation. Readings: null is absence (the protobuf JSON mapping); null is a present value that fails its field rules. Implemented: present and failing, so it is refused, and beside a descriptor it counts as both.
• A3, "decimal string" (line 90). No grammar is given. Readings: any string; a plain decimal -?[0-9]+(\.[0-9]+)?; a decimal with exponent or sign. Implemented: the plain decimal, the grammar the claim builder enforces.
• A4, suite.version (line 105). Whether both members of {name, version} are required. Implemented: both, as non-empty strings. The v0.1 emitter wrote version: null for a claim without a suite version; the v0.2 emitter refuses such a claim.
• A5, public-model and the model descriptor (lines 120-121, 135-137). The revision settles that a subject entry never counts as an identification; it does not say whether a public-model subject requires a model descriptor, or whether its digest must equal the subject's. Implemented: no cross-check in the verifier; the CLI writes the descriptor with the subject's digest.
• A6, harness members (lines 152-156). Whether name and version are required when harness is present. Implemented: required.
• A7, empty evidence (line 141). An empty array violates no entry rule. Implemented: accepted.
• A8, sampleSize range (line 112). No range is given. Implemented: an integer from 0 to 2^53 − 1.
• A9, the encoding of content (lines 147-148). The digest identifies the decoded bytes; the JSON encoding of the bytes is the ResourceDescriptor's. Readings: standard base64 only; any alphabet or padding a protobuf JSON parser accepts. Implemented: canonical, padded standard base64, the house's one strict decoder; a digest under one of the twelve MD5, SHA-1, SHA-2 and SHA-3 names whose form the verifier checks is recomputed over the content, and refused when the running Python cannot compute it; digests under other algorithm names are not compared with the content.
• A10, uri and downloadLocation describe the same artifact (lines 148-149). Not checkable offline; the verifier does not fetch, so this is not verified.

Migration path (vendor namespace → in-toto.io)

Using a vendor predicateType for a v0.x predicate is common practice (cf. cosign.sigstore.dev/…, apko.dev/…) and is fully in-toto-spec-conform — no upstream PR is needed for a self-hosted type. If the predicate is accepted upstream (#565), the predicateType moves to an https://in-toto.io/… namespace; at that point a redirect from the vendor URI to the registered one is added and the old value is documented as an alias. The draft keeps the version v0.1 for the revised shape, so the alias of the registered type is the vendor v0.2 type; the vendor v0.1 type names the earlier draft and is not an alias of anything registered. Consumers match on the digest, so the subject binding is unaffected by a predicateType rename.