Repository navigation
Host-receipt recorder: output_sha256 digests the published excerpt only - #492
Merged
seathatflowsinourveins merged 3 commits intoSep 29, 2026
Merged
Conversation
output_sha256 was the sha256 of the raw command output, while output_excerpt is the sanitized output cut at 400 characters. When an output fits in the excerpt, a reader could put a guessed home directory or user name back in place of the sanitized token, hash the result and compare it with output_sha256, which confirms the guess offline. The digest is now the sha256 of output_excerpt itself, so it never depends on unpublished text. A digest of the sanitized but unpublished rest of a long output was rejected in review: for a command whose output is known, the sanitized text changes with the account name, so hashing a guess of it tells accounts apart. Receipts recorded before this change keep their digests (append-only). The schema's command items forbid additional properties, so no per-command basis marker is added; the code comment and the schema description record the basis change and its date. Tests: a home-path output is digested sanitized and not raw, and the offline probe no longer reproduces the digest; a long output is digested as its excerpt, not as the whole sanitized or raw output; two accounts running the same command with the same public excerpt get the same digest; an output without a home path keeps the plain digest. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…t, branch files re-registered) Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…t, branch files re-registered) Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
8 tasks done
Collaborator
Author
|
Review record, 2026-09-29 (native-agent-stack-10). Merge candidate: head
|
4 tasks done
seathatflowsinourveins
deleted the
claude/host-receipt-excerpt-digest-20260929
branch
September 29, 2026 05:15
5 tasks done
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 11, 2026
…ence, blueprints and manifests (#1010) * Re-derive 10 published digests that hashed unmasked bytes, and guard evidence, blueprints and manifests Correction #161: the 2026-10-11 masked-digest audit of main at 7b76a88 found 8 sha256 values under evidence/artifacts that hash bytes from before sanitization. Its residual list found 2 more under blueprints. Each sits beside a copy in which the home directory is replaced by ~ or the user name by <user>. Anyone could put a guessed value back, hash the result and confirm the guess offline. Each value is now the sha256 of the published bytes. That is the basis scripts/host_receipts.py record has used for output_sha256 since #492 (scripts/host_receipts.py:1012-1029 at 7b76a88). - sota-refresh-20260925 markitdown (2), mcporter (3) and rtk (1) results.json: files[].run_copy_sha256 is now the sha256 of the committed file, the same value as the entry's sha256. byte_identical_to_run_copy stays false. Each record gets a corrections entry; mcporter had no list, so one is added. - The withdrawn nemotron vLLM use receipt -2: commands[1] and commands[4] output_sha256 are now the sha256 of output_excerpt (UTF-8), and its README records the change. This removes no oracle, because 4 registered evidence/hosts receipts keep the same digests (commands[1]'s in 4, commands[4]'s in 3). - blueprints/retrieval-quality-v2/review-293-repro-20260926.json: inputs.files[0].as_run_sha256 is now the committed driver's sha256. A corrections entry in the record's own shape records the change. - The withdrawn byterover-cli install receipt: commands[0].output_sha256 is now the excerpt digest. retained-outputs.json, the byterover-cli README and the trials README record the exception to "kept byte-for-byte". The sha256 of a file before this change would confirm a guess the same way: rebuild the old file from a guess, hash it and compare. Every digest of an old file is therefore re-pinned or dropped: - the three evidence/receipts qualification receipts re-pin results.json in provenance[], as #314 did; - retained-outputs.json gives the new receipt bytes for both private_original and published; - the review-293 correction records no as_recorded_sha256; - manifests/evidence.json re-registers the 13 files through host_receipts.register_file. tests/test_published_digest_privacy.py walks every JSON file under evidence/, blueprints/ and manifests/. It flags a 64-hex digest that hashes none of the published texts or files in its own or enclosing object, but does hash one of them once a home directory or name is put back. The masks are ~, <home>, <HOME>, <host-home>, $HOME, ${HOME}, <user>, <USER> and $USER, under /home/<name> and /Users/<name>. Names come from LANDING_PROHIBITED_NAMES. Without them, as in CI, that check skips. LEGACY_SITES freezes the 106 remaining sites by path, key and 8-hex prefix, with a count assertion: - 96 output_sha256 values in evidence/hosts receipts recorded before 2026-09-29, append-only since #492; - 10 that the audit's probe could not reach, because it put back only ~ and <user> and read only files named in the digest's own object. A new site fails. A listed site that is gone also fails until it is removed, so the list only shrinks. Two checks run in CI too: every listed site is still present, and the 10 fixed sites hash the published bytes. Fail-before: on the 7b76a88 tree, the guard reports exactly the 10 fixed sites and the fixed-site check fails 10 subtests, with or without names. After this change, all 11 tests pass, or 10 pass and 1 skips without names. Not changed: git history keeps the old digests. No rewrite is proposed, because the exposure is a home directory or account name, not a credential. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Re-pin readiness to the re-pinned receipts and enforce the digest guard in pre-push (#1010 round 1) B1 (CC adjudication of #1010 round 1): catalogs/north-star/readiness.json still pinned the pre-change sha256 of the three qualification receipts this PR re-pins, in 15 places. Each receipt had five: the repo:<receipt>@<digest> key, that entry's sha256 and expected_sha256, and two nested receipt objects. Rebuilt from the new bytes and a guessed name, each old receipt matched its pin, which is the second-order oracle of correction #161. Each pin now names the receipt's current sha256: - markitdown-018: d9a1a348 -> 74d066a1; - mcporter-0141: 5379b1e3 -> 9b69956e; - rtk-050: 9de293b6 -> b47326a1. Byte counts are unchanged. tools/north-star/build_readiness.py --write was not used, because it would also fold in readiness drift that predates this PR. tests/test_published_digest_privacy.py: - ReadinessPinTests needs no names. Every readiness binding of a receipt changed since the merge base with origin/main must hash that receipt's current bytes. It fails at 7a03593 (15 stale pins) and passes after. - The guard is now enforced. scripts/git-hooks/pre-push runs it on each pushed tip with PUBLISHED_DIGEST_GUARD_HOST_NAMES=1, so the names come from scripts/host_name_scan.py host_names(), the runtime source of #934's scan (docs/decisions/2026-10-09-host-name-scan.md); NATIVE_AGENT_HOST_NAMES_JSON replaces them in fixtures. The hook's runner refuses a skip, and an unavailable source errors. CI keeps the name-free tests. - The legacy check is one-directional, so it works in a clone whose names differ. An unlisted site fails. A listed site the names do not reproduce is reported only as a count. test_the_frozen_list_only_shrinks still fails, without names, once a listed digest is gone from its file. - The mask set adds the repository's existing markers <user-home>, <SCRATCH_HOME>, %USERPROFILE% (as C:\Users\<name>, C:/Users/<name> or /mnt/c/Users/<name>), <WSL_USER> and <hostname>. Every subset of up to four present masks is tried; beyond four, all of them or one at a time. - Measured with LANDING_PROHIBITED_NAMES (3 names) and with host_names() (5 names), the guard finds exactly the 106 listed sites both times. There is no new site, so the list and LEGACY_COUNT are unchanged. - Caching texts and resolved paths halves the whole-tree walk, from 24 s to 11 s with 3 names. tests/test_pre_push_gate.py overlays the guard module on the fixture tip, as it does the scanner sources. A clean push runs the guard once. A committed receipt whose output_sha256 hashes its excerpt with the fixture home put back is refused without echoing the name. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
scripts/host_receipts.py recordnow storesoutput_sha256as the sha256 ofoutput_excerpt, the sanitized output cut at 400 characters, so the digest binds only the bytes the receipt publishes.Before this change it hashed the raw output. Anyone could put a guessed home directory or user name back where the receipt shows
~or<user>, hash the result and compare it withoutput_sha256; the digest confirmed the guess whenever the output fit in the excerpt.lane:foundation. No shared hot file is edited exceptmanifests/evidence.json(re-registration, last commit).scripts/host_receipts.py(the recording loop and its comment),adoption/host-receipt.schema.json(theoutput_sha256description),tests/test_host_receipts.py(OutputDigestBasisTests, 4 tests),manifests/evidence.json.additionalPropertiesstays false), and every other recorder (see Residuals).Why the digest covers the excerpt only
printf '%0500d\nalice bob\n' 0gives two accounts identical public excerpts, but the unpublished suffix sanitizes differently (the name equal to the account becomes<user>), so hashing the known text with either name replaced identifies the account. A digest must never depend on text the receipt does not publish.SOTA sources
hashlib:sha256over UTF-8 bytes, which is what the recorder already used.Evidence-class table
origin/main'sscripts/host_receipts.py:python3 -B -m unittest -v tests.test_host_receipts.OutputDigestBasisTestsruns 4 tests andFAILED (failures=5), all 5 in the two tests that assert the excerpt basis. The plain-digest compatibility control passes on both. The two-account probe passes on the previous recorder (its raw digests are equal); it guards the rejected whole-sanitized designpython3 -B -m unittest tests.test_host_receipts: 184 tests OK;python3 -B scripts/host_receipts.py validate: exit 0, status passedtest_component_matrix63,test_adoption_bootstrap_macos161 (32 skips, environment-gated),test_capability_gate27,test_catalog_freshness_propose80,test_record_verdicts125,test_receipt_staleness30,test_hardware_profile53,test_verdict_review_gate167,test_landscape95,test_platform_status50,test_token_e2e_receipt_checks14,test_verdict_flip_candidates10: all OK, run one module at a time withTMPDIRoutside/tmppython3 -B scripts/validate.py,scripts/evidence_manifest.py --checkandscripts/component_matrix.py --checkexit 0 (run by the merge script)Review
gpt-6-astra, effort max,codex exec -s read-only; the model is the-mrequest, not an observed resolution) reviewed the first version (digest of the whole sanitized output): needs_changes. In scope: 1 major (the suffix oracle above; fixed by digesting the excerpt only, with the reviewer's probe as a test) and 1 minor (catalog_revisiondoes not reliably mark the digest basis; the schema description no longer implies it does). Outside this change: 2 major and 1 minor, listed under Residuals. It confirmed the original short-output attack is closed and found no consumer that assumes a raw-output basis.Residuals
scripts/native_token_ci.py(raw stdout, stderr and artifact digests beside sanitized representations, around lines 718 and 744) andtools/sota-convergence/landscape-sweep/usage_record.py(a raw-output digest beside a stripped transcript-directory prefix, line 77;tests/test_landscape_sweep_harness.pyaround line 2062 enforces that raw basis).cmdfield records the command text as given, so a literal user name or home path inside a command survives. An--identityequal to a user name is confirmable through its documented identity digest.🤖 Generated with Claude Code