Repository navigation
Shared fail-closed credential guard for both paper loaders (market_research gap closed) - #219
Merged
seathatflowsinourveins merged 11 commits intoSep 25, 2026
Conversation
…closed rules market_research.py's own credentials(path) opened its env file with only O_NOFOLLOW, a regular-file check, and a size cap -- unlike runner.credentials(), it never checked owner uid, mode 0600, or outside-any-Git-worktree location (documented as an open gap in gates-20260922.json's credential-handling note and docs/decisions/2026-09-22-broker-credential-handling.md). Add credential_guard.py (stdlib-only: os, stat, pathlib) with open_verified(), a TOCTOU-safe (fstat re-checked against a pre-open lstat by device/inode, the pattern tools/credentials/alpaca_rate_limit_probe.py:read_env_file already uses) fail-closed check for owner uid, mode 0600, and outside-any-worktree location, parameterized by follow_symlinks so each loader keeps its own prior symlink behavior: runner.credentials() still resolves a symlink to its target (follow_symlinks=True), market_research.credentials() still refuses a symlink outright (follow_symlinks=False, its prior O_NOFOLLOW rule). A FIFO is refused by the pre-open lstat regular-file check before open() is ever reached, so it can never hang either loader. No raised error includes file content. Both loaders now import the same open_verified(), so the rules cannot drift. runner.credentials()'s asserted "credential_file_permissions" error-string substring is unchanged and its existing tests pass unmodified. Tests: new tests.test_adaptive_market_research.MarketResearchCredentialFilePermissions (wrong mode, wrong owner, symlink, inside a Git worktree, missing file, a FIFO that must not hang, valid file), new symlink/FIFO cases added to tests.test_adaptive_paper_runner.CredentialFilePermissions, and a new SharedCredentialGuardParity test asserting both loaders import the identical open_verified and reject the same violation the same way. Update source-hashes.json and manifests/evidence.json (sha256/bytes only, for the changed registered files) and close the gap in gates-20260922.json's credential-handling note and the 2026-09-22 decision doc, naming the tests. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
runner.py and market_research.py import it, so it is an engine source like every other adaptive-paper module. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…e hard-link/worktree/encoding gaps Independent Codex security review of 9b02cc8 returned "not ready" with 7 items; this commit addresses each. 1. MEDIUM parent-directory race: the previous fix still resolved a pathname once, checked that fd, then separately re-resolved the path (by string) for the Git-worktree walk -- a swap of an ancestor directory between those two steps was not caught. Rewritten so every check is bound to the same dir_fd-chained traversal that produces the returned descriptor: from "/", each ancestor opens with O_DIRECTORY|O_NOFOLLOW via dir_fd, the final component with O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY via dir_fd, and the Git-worktree check runs on each already-opened ancestor fd (os.stat(".git", dir_fd=fd, follow_symlinks=False)) as it is opened, not on a re-resolved pathname afterward. A symlink substituted at any position after the one-time initial resolution now fails closed with ELOOP/ENOTDIR. The immediate parent directory must be owned by the caller and not group/other-writable. gates-20260922.json's note now states exactly what this does and does not guarantee. 2. MEDIUM hard links: st_nlink != 1 is now refused (a second name for the same inode could be compliant at one path and worktree-tracked at another). Regression tests added to both loaders. 3. MEDIUM worktrees without a `.git` marker: narrowed the claim in credential_guard.py's docstring, the decision doc, and the gate note -- detection only recognizes an ancestor `.git` entry; a tree configured purely through GIT_DIR/GIT_WORK_TREE or core.worktree is not detectable by a file-local check and is explicitly called out as out of scope. 4. MEDIUM race protections were untested: added tests/test_adaptive_paper_credential_race.py. credential_guard._hook is a no-op in production and lets tests deterministically inject a filesystem mutation at an exact traversal stage (ancestor-directory symlink swap, final-component symlink/FIFO swap, metadata change between open and fstat) instead of relying on real concurrency. A naive resolve-then-open reference implementation demonstrates the same ancestor-swap silently succeeds against it while open_verified() rejects it. Every "does not hang" assertion runs under a signal.alarm deadline. A mutation-kill report disables one guard protection at a time (hard-link check, worktree check, parent-directory check, O_NOFOLLOW, O_NONBLOCK) and asserts a specific named test now fails to reject the case that protection existed for. 5. LOW encoding: both loaders now require and document ASCII explicitly (matching what every writer of this file emits) instead of silently accepting UTF-8; a non-ASCII byte is refused before any line is parsed. Regression tests added. 6. LOW error content: every rejection now raises a single fixed, interpolation-free message (credential_guard.GUARD_DENIED) -- no path, basename, or upstream OSError text is ever included. Regression tests assert this explicitly. 7. Documentation: "effective user" corrected to os.getuid() (the real uid, not an "effective user" concept); the decision doc corrected to say market_research.credentials() raises ResearchError, not SafetyError; market_research.credentials()'s size cap is now enforced by reading at most MAX_CREDENTIAL_BYTES + 1 bytes from the opened fd and rejecting a longer result, not by trusting an earlier fstat-reported size that a concurrent writer could grow past. Runner's accepted configurations are unchanged: a normal 0600 file you own, in a 0700 or 0755 directory you own, outside any worktree, still loads (explicit regression test added). runner.credentials()'s asserted "credential_file_permissions" error-string substring is unchanged. source-hashes.json and manifests/evidence.json updated surgically (sha256/ bytes only) for every changed or newly registered file. Tests: full tests/test_adaptive_*.py discovery, 826 tests, OK (skipped=1); tests.test_native_faults_min and tests.test_credential_tools, OK. All six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py --check, verdict_review_gate.py, trading_gates.py --check) pass. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…real mutation-kill evidence Two independent reviews (Codex security review r2; Claude real-mutation and attack run, 260 attack cases plus a live RENAME_EXCHANGE race, all refused) both returned "not ready" on 2e1ef68. This commit addresses every item. 1. MEDIUM (Codex) another UID can relocate an already-checked ancestor: only the immediate parent's owner/mode were checked, so an attacker-writable /shared containing a victim-owned 0700 /shared/private could have that private directory renamed under a worktree root after it was opened and checked. Every ancestor from "/" down through the immediate parent is now checked with the OpenSSH safe_path/ secure_filename rule: owned by root or getuid(), and group/other-writable only if sticky -- a writable, non-sticky ancestor is refused the moment it is opened, independent of what gets renamed into it afterward. A private directory under /tmp (sticky) still passes; an attacker-writable /shared (not sticky) never does, regardless of what it contains. Regression tests simulate ownership via per-fd fstat/open faking (never a process-wide os.getuid() patch), per the fix-round's explicit test requirement. 2. LOW (both) a symlink loop leaks the path: Path.resolve(strict=True) raises RuntimeError (not OSError) with the path on Python 3.12, and an embedded NUL raises ValueError. Both are now caught alongside OSError. Every CredentialGuardError is built and raised from code that has already left the except block that caught the original failure, so Python never attaches it as __context__ (from None alone does not achieve this -- the interpreter re-populates __context__ at the raise statement itself if one is still executing inside a handler). Tests assert __context__ and __cause__ are None, not only the message text. 3. LOW (both) ".." was normalized lexically for market_research's non-follow mode, which could select a different file through an intermediate symlink without ever refusing it. Any ".." component is now refused outright. 4. LOW (Codex) the fd leaked if fdopen() itself failed after every other check passed. Closed on any such failure now; tested by injecting a failure into fdopen. 5. LOW (Codex) GUARD_DENIED was one generic message. Replaced with fixed, path-free credential_file_permissions:<code> reason codes (missing, not_regular, symlink, owner, mode, hardlink, worktree, ancestor, parent_owner, parent_mode, encoding, size). Runner's asserted credential_file_permissions prefix is unchanged. 6. runner's read was unbounded; it now reads at most MAX_CREDENTIAL_BYTES + 1 bytes from the fd, the same shared cap market_research uses. 7. MEDIUM (both) surviving mutations and test gaps, each closed with a test that specifically fails when its protection is removed: (a) file-owner check was masked by wrong-owner tests patching os.getuid() process-wide, so the parent-owner check always fired first. New test simulates a foreign file owner via per-fd fstat faking with the parent genuinely left intact. (b) a by-name worktree check computed once, before any traversal, would miss a worktree marker added mid-walk; new test injects one via _hook exactly when a still-open ancestor is about to be opened. (c) path-in-error was only checked for a subset of failure modes; new tests cover missing, symlink-loop, NUL-byte, and permission-error paths, all asserting no path/basename and clean __context__/__cause__. (d) a static oversized fixture rejects identically whether the read is bounded or unbounded, so it never established boundedness; new tests assert the exact read() size argument each loader's call uses. (e) removing O_NONBLOCK would hang the suite instead of failing it; both FIFO tests (test_adaptive_market_research.py, test_adaptive_paper_runner.py) now run under a signal.alarm deadline. (f) fstat(fd) vs a pathname re-stat was not established; new test makes os.stat() lie about a file's mode and confirms rejection still uses the fd's own (correct) metadata. (g) added root-marker (root's own fd is genuinely walked, not skipped) and fd-leak tests (open fd count unchanged across success and every refusal path, via /proc/<pid>/fd). (h) replaced the in-process MUTATION_KILLS meta-test (which asserted a hardcoded string-to-test-name mapping without ever running those tests) with tests/_credential_mutation_driver.py: copies the real source tree to a private temp directory, applies one real exact-count textual mutation, and runs the actually-named test(s) against that copy in a subprocess. All twelve declared mutations are genuinely killed (subprocess exits non-zero); a baseline run with no mutation applied is verified to pass first. 8. Docs: the decision doc and the gate note now name the real test file (tests/test_adaptive_paper_credential_race.py, not the nonexistent tests/test_credential_guard_race.py an earlier round named), describe only what the tests actually do (no overclaimed deadline/mutation-run coverage), and state the markerless-worktree and bind-mount limits explicitly. Runner's symlink-follow acceptance of a compliant swapped-in file stays documented, unchanged. Accepted configurations are unchanged: a 0600 file in a 0700 or 0755 directory you own, under /home or /Users paths owned by root or by you, and a private directory under /tmp -- each has an explicit regression test. source-hashes.json and manifests/evidence.json updated surgically (sha256/ bytes only) for every changed file, including the driver's own self-hash. Tests: full tests/test_adaptive_*.py discovery, 849 tests, OK (skipped=1); tests.test_native_faults_min and tests.test_credential_tools, OK (45 tests). All six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py --check, verdict_review_gate.py, trading_gates.py --check) pass. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…lose, strict parent rule Both reviewers again found no access-rule bypass (268 attack cases, a live race with ~600k swaps); 31 of 33 real mutations were killed. This commit addresses the remaining items. 1. MEDIUM (both) ASCII refusal carried the credential bytes: runner.py and market_research.py raised inside a UnicodeDecodeError handler, whose `.object` attribute holds the entire input. Both now check `raw.isascii()` on the bytes before ever decoding, so no UnicodeDecodeError is constructed at all, and the refusal is raised from clean control flow (never inside an except block, so `__context__` stays None -- `from None` alone does not clear it). Both non-ASCII tests now assert `__context__`/`__cause__` are None and that no exception attribute contains a file byte. 2. MEDIUM (both) the mutation driver reported false kills: it counted any non-zero subprocess exit as a kill, so disable_worktree_check reported KILLED against completely unmutated code (the copy had no .git). The driver now git-inits every copy, runs the named test id(s) on the pristine copy first (must report a plain "ok"), and counts a kill only when a named test transitions to a genuine unittest "FAIL" (never an "ERROR", never merely a non-zero exit). New MutationDriverSelfTests proves this: an empty mutation, an unrelated-import-breaking mutation, and a mistyped test id must each report "not killed". Re-run with the corrected driver: all fourteen declared mutations are genuinely killed (true table below). 3. LOW fdopen cleanup could double-close: if `os.fdopen`'s internal `io.FileIO` construction itself failed after accepting fd ownership, CPython already closes that fd, so a second `os.close()` either raised EBADF or closed an unrelated, since-reused fd. Now builds `io.FileIO(file_fd, "rb", closefd=True)` first and, on a later failure wrapping it in a `BufferedReader`, closes only that already-constructed object, never the bare integer. Tested by making the wrapping step fail and confirming no descriptor leak. 4. Two mutations added and killed: skipping root's own ownership/mode check (a simulated foreign-owned, world-writable "/", via per-fd fstat faking) is now rejected; removing runner's size check is now rejected too (a fixture with real KEY/SECRET lines followed by a huge trailing comment line demonstrates the size check itself is what fires, independent of what a truncated read would otherwise parse). 5. Policy: tightened the immediate parent back from round 3's relaxation. The immediate parent must be owned by exactly getuid() (root does not excuse it) and never group/other-writable, with no sticky exception -- this rejects /tmp/file and /file while a private directory under /tmp still passes (checked one level down by this same rule). Ancestors above the immediate parent keep the OpenSSH safe_path/secure_filename rule with its sticky exception, unchanged. 6. LOW reason codes unified: market_research's oversized-file rejection now raises credential_file_permissions:size, the same code runner already used, instead of the older invalid_credential_file. 7. Docs: the decision doc and gate note no longer claim an atomic location guarantee for the worktree marker check -- it is a snapshot at each ancestor's own inspection time; another uid permitted to write into a sticky ancestor (e.g. /tmp) can still create a new .git entry there afterward, which does not bypass any file's own checks or relocate an already-verified descriptor. A symlinked directory component (as opposed to the final file) is now reported as :symlink, not the generic :missing -- Linux reports ENOTDIR, not ELOOP, for O_DIRECTORY|O_NOFOLLOW against a symlink, so one extra failure-path-only lstat distinguishes the two cheaply. source-hashes.json and manifests/evidence.json updated surgically (sha256/ bytes only) for every changed file. Tests: full tests/test_adaptive_*.py discovery, 855 tests, OK (skipped=1); tests.test_native_faults_min and tests.test_credential_tools, OK (45 tests). All six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py --check, verdict_review_gate.py, trading_gates.py --check) pass. True mutation table (subprocess-executed, pristine-baseline-verified, FAIL- not-ERROR-required -- all 14 killed): allow_ancestor_writable_without_sticky: KILLED disable_hardlink_check: KILLED disable_o_nofollow: KILLED disable_o_nonblock: KILLED disable_parent_directory_check: KILLED disable_root_ancestor_check: KILLED disable_runner_size_check: KILLED disable_worktree_check: KILLED leak_fd_on_fdopen_failure: KILLED normalize_dotdot_lexically: KILLED read_whole_file_market_research: KILLED skip_fstat_owner_identity: KILLED skip_fstat_use_prestat: KILLED worktree_final_path_by_name: KILLED Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…O leak, protect the untested rules Both reviewers again found no access-rule bypass (288 attack cases, a live race with ~670k swaps); all 33 earlier mutations plus round 4's 14 were killed, and the driver produced no false kills across 21 scenarios. This commit addresses the remaining small items. 1. _is_symlink_component()'s failure-path lstat caught only OSError, so a NUL byte in an ancestor directory component raised a bare ValueError, and a surrogate raised UnicodeEncodeError (whose .object attribute retains the ancestor's own name). Now catches (OSError, ValueError, UnicodeError), so the already-sanitized REASON_MISSING code applies instead. Two new tests cover both cases, asserting no path in the message, .object, __context__, or __cause__. 2. The double-close fix's own comment was wrong, and it was actually leaking a fd in one case: io.FileIO(fd, "rb", closefd=True) does NOT take ownership of fd until construction succeeds (verified directly against CPython: a failed construction on a directory fd leaves it open), so open_verified() was never closing file_fd when the FileIO constructor itself is what fails. Fixed: that specific failure now closes file_fd directly; only the already-constructed raw object (never the bare integer) is closed if the later BufferedReader wrap fails. Two new tests: one confirms no leak when FileIO's constructor fails; the other asserts raw.closed is True after a wrap failure -- the deterministic way to distinguish a proper raw.close() from a bypassing os.close(file_fd), since io.FileIO.close() is implemented in C and never calls Python's os.close() (so counting os.close() calls cannot observe this at all). 3. Four new tests protect the stricter immediate-parent rule (round 4), which reverting entirely previously still passed the suite: a real 0600 file directly in the real /tmp; a simulated root-owned 0755 parent; a simulated sticky 1777 parent; and a bare /file. Each is chosen so only the strict parent rule, not the generic sticky-excused ancestor rule, can be what rejects it. 4. A new test protects the double-close fix specifically (distinct from the existing leak test, which only proves no leak, not no double-close). 5. A new test directly asserts a symlinked directory component reports :symlink, not :missing. 6. The mutation driver's "skipped" result fell through to "survived", indistinguishable from the mutated code genuinely being exercised and passing. Now reported as "inconclusive" and treated as not-killed, so the gate fails on it the same as a real survival. Docstring corrected: a "FAIL" can come from setUp(), not only the named test method's own body. New MutationDriverSelfTests case injects a real @unittest.skip(...) and confirms the driver reports "inconclusive". Also removed one round-4 mutation (leak_fd_on_fdopen_failure) whose target text no longer matched after item 2's restructuring and whose protection is now covered more precisely by the new double_close_on_bufferedreader_failure mutation. source-hashes.json and manifests/evidence.json updated surgically (sha256/ bytes only) for every changed file. Tests: full tests/test_adaptive_*.py discovery, 865 tests, OK (skipped=1); tests.test_native_faults_min and tests.test_credential_tools, OK (45 tests). All six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py --check, verdict_review_gate.py, trading_gates.py --check) pass. True mutation table (subprocess-executed, pristine-baseline-verified, FAIL- not-ERROR-required -- all 15 killed): allow_ancestor_writable_without_sticky: KILLED disable_hardlink_check: KILLED disable_o_nofollow: KILLED disable_o_nonblock: KILLED disable_parent_directory_check: KILLED disable_root_ancestor_check: KILLED disable_runner_size_check: KILLED disable_worktree_check: KILLED double_close_on_bufferedreader_failure: KILLED normalize_dotdot_lexically: KILLED read_whole_file_market_research: KILLED revert_strict_parent_rule: KILLED skip_fstat_owner_identity: KILLED skip_fstat_use_prestat: KILLED worktree_final_path_by_name: KILLED Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…k, real high surrogate, driver exit code Claude verified the round-5 head as ready (312 attack cases, no bypass; 46 of 47 mutations killed) and Codex confirmed its round-4 findings fixed. Four small test/driver fixes remained. 1. test_file_directly_in_real_tmp_is_refused used tempfile.gettempdir(), which honors TMPDIR -- private and per-user by default on macOS, so this test would silently exercise a different, already-private parent there and never reach the code path it means to test (breaking CI validate-macos). Now creates the file exclusively in the literal /tmp via tempfile.mkstemp(dir="/tmp"), skipping with a clear reason if /tmp does not exist or is not sticky and world-writable. 2. The double-close test asserted only raw.closed, which a subtler surviving mutant (raw.close() followed by a redundant os.close(fd)) still satisfies -- the extra close's EBADF is itself an OSError, which a bare assertRaises(OSError) also silently accepted, masking that the wrong exception propagated. Now asserts the propagated exception is exactly the simulated one by identity, and separately forces the OS to reuse the exact freed fd number (opening files in a bounded loop until one receives it, rather than hoping the next unrelated open() does) and confirms that file is still open and readable afterward. A new extra_close_after_raw_close mutation is killed by this test. 3. The surrogate test used U+DCFF, which is in surrogateescape's low- surrogate range and round-trips straight back to a plain byte on encode -- os.fsencode() never raised there at all, so the test exercised only a plain OSError and passed even with the pre-fix, OSError-only exception handler restored. Now uses U+D800, a lone high surrogate with no such round-trip, which always raises UnicodeEncodeError on encode (verified directly against the reverted handler). The test also now catches broadly and calls self.fail() explicitly on the wrong exception type, since assertRaises(CredentialGuardError) would otherwise report an uncaught UnicodeEncodeError as an "ERROR", which the real mutation driver never counts as a kill. A new narrow_symlink_component_exception_handler mutation is killed by this test. 4. The mutation driver ignored the mutated subprocess's exit code once every named test id reported "ok": a tearDownClass() (or other class- scoped fixture) failure is reported by unittest as a separate pseudo- test-id that never matches anything in test_ids, so it never appeared in the collected results at all, and the driver reported "survived" even though the run did not cleanly succeed. Now requires a clean process exit for "survived"; a non-zero exit unexplained by any named test id's own FAIL/ERROR/skip is "inconclusive" instead. New MutationDriverSelfTests.test_teardownclass_error_is_inconclusive_not_survived injects a real, always-raising tearDownClass and confirms this. source-hashes.json and manifests/evidence.json updated surgically (sha256/ bytes only) for every changed file. Tests: full tests/test_adaptive_*.py discovery, 866 tests, OK (skipped=1), run once with TMPDIR unset and once with TMPDIR set to a private directory; tests.test_native_faults_min and tests.test_credential_tools, OK (45 tests). All six validators pass. The driver's own self-tests (6, including the two new ones) pass, and all 17 declared mutations are genuinely killed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…h-credential-guard-20260924 # Conflicts: # manifests/evidence.json
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…digests blueprints/us-equities/adaptive-paper/source-hashes.json maps engine file paths to their sha256. Paths that contain "credential" (credential_guard.py, tests/_credential_mutation_driver.py, tests/test_adaptive_paper_credential_race.py) made gitleaks' generic-api-key rule flag the digest on the same line. The manifest is added to the existing rule-scoped, line-anchored allowlist for reviewed checksum manifests. That allowlist's regex requires a filename-shaped key (path separator, extension or leading dot) and a 40- or 64-hex value, so a credential-shaped key name in the same file is still detected. tests.test_gitleaks_config passes with the pinned gitleaks 8.30.1 (19 OK). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…OS symlink portability Codex reviewed 1bd4e2f (which added source-hashes.json to the shared "filename-shaped keys in reviewed checksum-manifest files" allowlist) and returned "not ready": that shared regex also matches a credential-shaped key such as "credential.key", nothing enforced that every entry in source-hashes.json is a real file digest, and no gitleaks config test covered the new path. Then CI validate-macos failed on #219 with a confirmed root cause: market_research.credentials() (no-follow mode) refuses every macOS temp fixture, since /tmp and /var are themselves root-owned symlinks to /private/tmp and /private/var there. 1. Reverted 1bd4e2f's addition to the shared allowlist. Added a dedicated, rule-scoped [[rules.allowlists]] under generic-api-key for exactly blueprints/us-equities/adaptive-paper/source-hashes.json: condition = "AND", regexTarget = "line", and a stricter line-anchored regex that requires the ENTIRE key to be a real repo path under blueprints/us-equities/adaptive-paper/ or tests/, ending in .py, .json, .sh, or .txt (the manifest's own actual extension set), mapped to exactly 64 lowercase hex -- a credential-shaped key such as "credential.key" or "api_key" in this same file is not a path and still fails the regex. 2. Added tests/test_adaptive_paper_source_hashes_manifest.py: stdlib-only (hashlib, json, re, unittest, pathlib -- no nautilus_trader, no alpaca), so it runs in CI's system-python "Test validation failure modes" step (validate.yml's bare `python3 -m unittest`). For every key in source-hashes.json: the file must exist, its current sha256 must equal exactly the recorded value, no duplicate keys, and every key must match the same path shape the dedicated allowlist trusts (kept in sync deliberately) -- so a secret can never sit in this manifest without failing CI's validate job, independent of the allowlist. 3. Added 6 gitleaks config tests (tests/test_gitleaks_config.py, following its existing test_a/test_b/... pattern): a path-keyed 64-hex digest in the exact path is not detected; a credential-shaped key ("credential.key", "api_key") with 64 hex in that same path is still detected; a non-lowercase-hex value under a path-shaped key is still detected; a second secret (api_key) sharing one physical line with an allowlisted path digest is still detected; the identical path-keyed line in a different file is still detected. macOS portability (item confirmed against a full CI failure log): 4. Added tests/adaptive_paper_hermetic.py's real_tmp_root(), a shared helper both test_adaptive_market_research.py and test_adaptive_paper_credential_race.py now use for every fixture root a follow_symlinks=False test builds: os.path.realpath() resolves the macOS-only /var (hence /var/folders/..., tempfile's default root there) and /tmp symlinks before the guard ever sees the path, so no-follow mode reaches the actual scenario a test means to exercise instead of refusing the OS-provided ancestor symlink first. The literal-/tmp test now uses os.path.realpath("/tmp") explicitly as its own parent for the same reason (still refused as parent_owner/parent_mode on the resolved target, never symlink). _open_fd_count() now checks /proc/<pid>/fd or /dev/fd, whichever exists, instead of the Linux-only /proc path (macOS has no /proc at all). The NUL-byte and surrogate tests reach the actual bad component correctly once their fixture root is resolved; no platform-specific skip was needed. credential_guard.py's module docstring and the decision doc now state explicitly that no-follow mode refuses every symlink component, including OS-provided ones -- callers must pass an already symlink-free path, which the documented credential store under $HOME always is. The guard itself is unchanged: this is a fixture fix, not a weakening of the rule. Linux behavior is unchanged (os.path.realpath is a no-op there in practice). Verified locally on Linux with TMPDIR unset, and additionally with TMPDIR pointing at a symlink to a separate real directory (simulating the macOS /var/folders layout) -- both runs pass identically, including the real mutation-kill driver's own pristine-baseline runs (which exercise these same fixtures via its own copied-tree subprocess). source-hashes.json and manifests/evidence.json updated surgically (sha256/ bytes only) for every changed registered file. Tests: full tests/test_adaptive_*.py discovery, 871 tests, OK (skipped=1), run both with TMPDIR unset and with TMPDIR set to a symlinked directory; tests.test_native_faults_min and tests.test_credential_tools, OK (45 tests); the new source-hashes-manifest enforcement test, OK (5 tests); all six validators pass; tests.test_gitleaks_config with GITLEAKS_TESTS_REQUIRED=1 and the pinned gitleaks 8.30.1 on PATH, OK (24 tests); gitleaks git (--log-opts=HEAD, 511 commits, ~1.12 GB) and gitleaks dir scans both report 0 findings; all 17 real mutations still killed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The dedicated gitleaks allowlist and the manifest enforcement test both accepted keys such as tests/../scripts/validate.py, which contradicts the under adaptive-paper/ or tests/ scope. Each path segment must now start with [A-Za-z0-9_-]. Added a key-shape unit test and a gitleaks config test that a dot-segment key holding a digest is still detected (25 config tests OK). Also corrected the allowlist comment: the per-entry check is tests/test_adaptive_paper_source_hashes_manifest.py; scripts/validate.py only pins the manifest file's own hash. Found by the independent Claude review of round 7. Codex was at its usage limit, so that review was the independent check for this round. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
deleted the
claude/market-research-credential-guard-20260924
branch
September 25, 2026 00:21
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 25, 2026
…ction guard branch - runner.py: kept both import lines (corporate_actions, credential_guard); the two changes touch different regions. - source-hashes.json: took the union of both sides' keys and recomputed every digest from the merged tree. The #219 manifest test passes. - evidence.json: rebuilt from main's version. Re-registered this branch's native-check record, registered corporate_actions.py and its tests, and refreshed the six stale digests surgically. Checks, with the network blocked: 986 adaptive tests OK; native-faults, order-contract and credential-tools 56 OK; order-throughput, trading-gates, evidence-manifest and validate 163 OK; all six validators pass. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This was referenced Sep 25, 2026
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 25, 2026
…-09-24/25 merges (#225) Updates four checkpoint entries: the paper lane, the simulation lane, alpaca-paper-operational and corporate-action-data. They now reflect #198, #199, #212, #218, #219 and #222. tests.test_grand_dashboard passes, and all state strings satisfy progress.py's public-token rule. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 25, 2026
…e changes job, NUL-safe diff
Independent review found that needs: changes plus a bare if: (no status function) applies
an implicit success() to bootstrap-linux/bootstrap-macos/bootstrap-macos-brew, so once
changes itself is skipped (it only runs on pull_request), all three jobs were skipped too
on every push, schedule and workflow_dispatch run -- confirmed live by the coordinator's
dispatch run 36085789483 (changes skipped, all three bootstrap-* jobs completed as
skipped, validate-macos ran normally). Fix: if: ${{ !cancelled() && (github.event_name !=
'pull_request' || needs.changes.outputs.bootstrap != 'false') }}, following the
security-scan.yml precedent; != 'false' also makes a failed/cancelled changes job run
everything (fail safe). The changes job itself now fails safe on a missing payload SHA or
a failed git diff (echoes bootstrap=true and exits 0 instead of exiting 1 and letting
needs: skip the dependents), and switches to git diff -z with a NUL-delimited read so a
quoted non-ASCII filename still matches a PATTERNS glob. Also: SC2254-annotated case glob
comment corrected to describe over-matching (not exact paths: semantics), stale workflow
comments and decision-record claims corrected (PR #219 did run and fail validate-macos,
just wasn't required; "fourth job" -> "fifth"; measured cost 8.2-13.4 min, median 10.9 over
36 runs; decision record restructured to flat Evidence/Alternatives/Decision/Cost/Overturn/
Measured; added a rollout note about already-open PRs needing a new push after the ruleset
is applied), and tests/test_workflow_hardening.py strengthened to require a status function
and != 'false' (proven to fail against the pre-fix condition text) plus fail-safe/NUL-safety
assertions on the changes job's script.
Verified: actionlint 1.7.12 (with shellcheck 0.11.0) and zizmor --offline --no-config
--no-ignores --persona regular --strict-collection both report 0 findings on
.github/workflows. tests.test_workflow_hardening, test_adoption_bootstrap_macos,
test_adoption_bootstrap, test_adoption_docs_consistency, test_workflow_security_coverage,
test_workflow_security and the full python3 -m unittest suite (4720 tests) all pass. All
six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py
--check, verdict_review_gate.py --base origin/main, trading_gates.py --check) pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 25, 2026
…s in the record - shell: bash runs as bash -eo pipefail, so the script now runs set +e and guards mktemp with the fail-safe bootstrap=true write. The test asserts both. - Workflow comment and decision record: only push is path-filtered; schedule and workflow_dispatch run every job. - #219: validate-macos failed on earlier heads (72a15b8, 1bd4e2f) and passed at the merged head 04c867b. It was not a red run beside the merge. From the independent delta review of 8ec88c1/c19cb31d (verdict ready). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 25, 2026
Adds validate-macos (the full test suite on macOS) to the main ruleset target, and makes it report a status on every pull request. - adoption-bootstrap.yml drops the pull_request path filter. A fail-safe changes job, which diffs base and head with git, keeps bootstrap-linux, bootstrap-macos and bootstrap-macos-brew path-gated on PRs through if: !cancelled() && (event != pull_request || bootstrap != 'false'). - Push keeps its paths filter. Schedule and dispatch run every job. - Measured: before the fix, dispatch run 36085789483 skipped the bootstrap jobs. After it, dispatch run 36086815267 ran them all and they succeeded. - strict_required_status_checks_policy stays false, because its recorded overturn condition has not occurred. - Evidence for requiring macOS: #219's macOS-only failures (72a15b8, 1bd4e2f). Reviewed by Claude (Codex at its usage limit until 2026-09-30). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This closes a documented paper-lane credential gap.
market_research.pyread the Alpaca paper key file through its own loader, which had no owner, mode or outside-worktree checks. Both paper loaders,runner.credentials()andmarket_research.credentials(), now go through one shared, stdlib-only guard,blueprints/us-equities/adaptive-paper/credential_guard.py.The guard
Opening the file.
dir_fdfrom/. Every ancestor is opened withO_DIRECTORY|O_NOFOLLOW, and the file withO_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY...is refused outright.Ancestor and parent rules.
secure_filenamerule: it must be owned by root or bygetuid(). It may be group- or world-writable only if it is sticky and the next component down is owned bygetuid()or root..gitentry at the moment it is inspected.getuid()and must not be group- or world-writable. So/tmp/fileand/fileare refused, while private directories under/tmpor$HOMEload.The file itself.
getuid(), have mode exactly0600, havest_nlink == 1, and be a regular file.MAX_CREDENTIAL_BYTES + 1bytes are read.isascii()before any decode, so noUnicodeDecodeErrorcan carry file bytes.Errors and descriptors.
credential_file_permissions:<code>:missing,not_regular,symlink,owner,mode,hardlink,worktree,ancestor,parent_owner,parent_mode,encodingorsize.exceptblock, so__context__and__cause__stay empty.FileIOis built first; if that fails, the fd is closed directly, and if the later wrap fails, theFileIOis closed.Stated limits (decision doc and gate note):
GIT_DIR,GIT_WORK_TREEorcore.worktreewith no.gitancestor) cannot be detected by a check local to the file..gitcheck is a per-ancestor snapshot taken when each ancestor is inspected, not an atomic location guarantee.runner.credentials()keeps its accepted configurations and itscredential_file_permissionsprefix.Review
Six rounds, with two model families on each round.
Codex, gpt-6-astra (read-only security review), found:
Claude reviewers ran real attacks and real mutations:
RENAME_EXCHANGErace of about 660k swaps per 6 s run. Every case failed closed, and none produced an attacker read in the worktree or no-follow cases.FAIL. Skip, error and teardown outcomes are reported as inconclusive. It produced no false kills across 22 probe scenarios.Final verdicts: Claude said ready at 68ad970. Round 6 closed the last test and driver items from both reviewers. The most important was the
/tmptest, which would have failed under macOS's privateTMPDIR.Checks
TMPDIRunset and set to a private directory.tests.test_native_faults_minandtests.test_credential_toolsgive 45 OK.Later rounds (after the PR opened)
Secret-scan false positive. File paths containing "credential" made gitleaks' generic-api-key rule flag the sha256 values in
adaptive-paper/source-hashes.json. The fix:adaptive-paper/ortests/, with no.or..segments, and values must be 64 lowercase hex;tests/test_adaptive_paper_source_hashes_manifest.py) that fails unless every entry is the real sha256 of an existing file, so a secret cannot sit in that manifest;macOS portability. On macOS,
/tmpand/varare OS symlinks, which no-follow mode refuses, so the test fixtures now use realpath temporary roots, and fd counting is portable.validate-macospasses. The docs state that no-follow mode refuses OS symlinks.Review limitation. The Codex CLI reached its usage limit, which resets 2026-09-30 23:50, so round 7 and the dot-segment fix had no cross-family review. The independent Claude reviewer covered them instead:
The one Low it found (dot segments) is fixed in 04c867b.
🤖 Generated with Claude Code