Skip to content

Integrity wave M3: betterleaks v1.8.1 non-required trial job beside the gitleaks gate - #455

Merged
seathatflowsinourveins merged 12 commits into
mainfrom
claude/betterleaks-parity-20260927
Sep 28, 2026
Merged

seathatflowsinourveins merged 12 commits into
mainfrom
claude/betterleaks-parity-20260927

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

Plan move M3 (integrity wave, 2026-09-27): a non-required secret-scan-betterleaks trial job runs betterleaks v1.8.1 beside the required gitleaks 8.30.1 secret-scan gate. It uses the same .gitleaks.toml, with redacted output.

  • .github/workflows/validate.yml: the job verifies before it executes anything.
    • cosign verify-blob of checksums.txt against its Sigstore bundle, with an identity pinned to release.yml@refs/tags/v1.8.1 and GitHub's OIDC issuer.
    • A pinned archive digest.
    • Binary checks.
  • Scans. The job then scans history and the tree with --exit-code 0 (report-only for findings), and runs the ported fixture classes.
    • A scanner error still fails the step. Upstream cmd/root.go at 5eab4833: err exits 1, and findings exit with exitCode.
    • The job is not in .github/main-ruleset.json, so it never blocks a merge.
  • tests/test_gitleaks_config.py:
    • redacts the history-ancestry scan that the required job runs, and names no secret when it fails;
    • reads a null JSON report as no findings (betterleaks writes an empty report that way);
    • runs every scan through one helper that raises a unittest error for a scan that does not complete, and adds ScannerErrorTests.
  • tests/test_workflow_hardening.py: adds BetterleaksTrialJobTests (11 tests).
  • Receipt: evidence/artifacts/betterleaks-parity-20260927/ holds README.md, run-record.json, parity.json and the harness.
    • It covers verification with failing controls, parity over the full history and the fixture classes, and value-blind triage.
    • The three local restic test passphrase rows are resolved as disposable test values with no retained repository.
  • The required secret-scan job's YAML, the pre-commit hook, the pins and catalog-freshness.yml are unchanged.
  • The fixture tests that the required job runs are now stricter: a scan that does not complete is an error.
  • Base commit: ba1700ad; main merged in at 718e5620 per the docs/lanes.md hot-file protocol.
  • Lane: lane:foundation.

Review

  • Claude review round:
    • bash pipefail, archive depth 0, an explicit .gitleaksignore, and the null report.
    • A coordinator repair made the job report-only through --exit-code 0 rather than continue-on-error.
  • Cross-family GPT-6 review (codex exec -s read-only -m gpt-6-astra, effort max, on 1c029808, 2026-09-28): changes-needed, one P2.
    • The fixture step's failures-only filter would pass a scanner crash, because _run_gitleaks raised crashes as AssertionError.
    • Exits −11, 139 and 1 each produced FAILED (failures=1) and a green step.
  • Repair (11b46735, 4e5f16e9): a scan that does not complete is now a unittest error.
    • Before and after, through the workflow's own filter: those exits went from a green step to FAILED (errors=1) and a red step.
    • A finished scan that misses a value still reports as a failure.
    • Every pinned-source citation (gitleaks v8.30.1 and betterleaks v1.8.1 cmd/root.go, zerolog, Go runtime) was checked at its tag.
  • Independent Claude stack-verifier: pass.
    • It re-ran the reviewer's probe, 6 mutation controls, test_gitleaks_config (33 OK with gitleaks 8.30.1) and test_workflow_hardening (72 OK, 2 skipped).
    • A simulated fixture step with the verified binary gave FAILED (failures=3), which is the known d2/d4/d5 rule difference, and exit 0.
  • 7dae58dc fixes a pre-existing integrity issue the verifier found on main.
    • The required secret-scan job's history-ancestry test scanned the real history without --redact, and printed the findings in its failure message. A real secret in HEAD's ancestry would therefore have reached the public job log.
    • The scan now passes --redact (gitleaks v8.30.1 report/finding.go lines 78-86; betterleaks v1.8.1 cmd/root.go line 589), and a failure lists only the rule, file, line and fingerprint.
    • The new ScannerErrorTests.test_f covers both halves, and reverting either half fails it.
    • The full module with gitleaks 8.30.1: 34 tests OK, no skips.
  • CodeQL on d01b9f69: 5 new py/clear-text-logging-sensitive-data alerts (Refresh the SOTA convergence manifest from the 2026-09-22 layer-verdict lane run #66 to PR-8: verified viability probes for three targeted SDK/harness candidates (2026-09-22) #70), all in the receipt's local harness scripts. Each was dismissed as a false positive, with its SARIF taint source cited, following the Finalize portable grand catalog with independent Codex and Claude convergence #55 to Integrate PRs #55–#58 into one grand-catalog base (2026-09-22) #59 precedent. The recorded scripts stay unchanged.
  • 1573687b fixes three record details the verifier found:
    • the macOS launcher line range;
    • the last observation time;
    • the missing lock-skip behaviour change.

Residuals

  • Hosted run: the hosted run of this job is unobserved until this PR's CI. The expected result is green, with the fixture step reporting FAILED (failures=3).
  • Before any swap:
    • allowlist the three passphrase rows by fingerprint;
    • drop --exit-code 0 and the fixture-step conversion;
    • add the betterleaks and cosign pins to catalog-freshness.
  • betterleaks skips an unreadable file silently.

SOTA sources

  • betterleaks/betterleaks v1.8.1, an immutable release with a Sigstore-signed checksums.txt. The same author declared gitleaks feature-complete (gitleaks README line 12, read 2026-09-28).
  • sigstore/cosign v3.0.6, bootstrap digest pinned from sigstore/cosign-installer v4.1.2 action.yml.
  • CPython unittest (case.py, runner.py, main.py) for the failure and error semantics.

Checklist

  • Actions SHA-pinned; top-level permissions: contents: read; harden-runner; persist-credentials: false.
  • No secrets printed, uploaded or committed. Scans are redacted, the step summary shows counts only, and there's no report artifact.
  • No new required check and no new secret.
  • python3 scripts/validate.py passed; the touched test modules passed.
  • No paid hosting.

🤖 Generated with Claude Code

Scout and others added 8 commits September 27, 2026 23:20
… the gitleaks job, with a parity receipt

validate.yml gains a job that is not a required context (.github/main-ruleset.json is unchanged). It
installs cosign v3.0.6 by the digest sigstore/cosign-installer v4.1.2 pins for its own bootstrap. It then
verifies betterleaks v1.8.1's checksums.txt against its Sigstore bundle, with the identity pinned to
betterleaks/betterleaks release.yml@refs/tags/v1.8.1, GitHub's OIDC issuer, the tag commit and a push
trigger, and checks the archive digest before extraction. It runs the gitleaks allowlist fixture classes
and redacted history and tree scans that log rule, file and line only. The required secret-scan job,
the pre-commit hook and the pins files are unchanged.

tests/test_workflow_hardening.py: BetterleaksTrialJobTests (not a required context and never forced
green, read-only and hardened, binaries run only after verification, both scans redacted, and the
unredacted history class left out), with three failing controls recorded.

evidence/artifacts/betterleaks-parity-20260927: verification outputs and controls, and parity against
gitleaks 8.30.1. betterleaks is not a superset: 13 fixture detections are missing and empty reports are
written as null. It adds 16 tree and 91 history findings, triaged value-blind, with no real secret. The
receipt also records resources, lint results and deviations.

Sources: betterleaks/betterleaks v1.8.1 (tag commit 5eab4833), gitleaks/gitleaks v8.30.1,
sigstore/cosign v3.0.6, sigstore/cosign-installer v4.1.2 (6f9f1778).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…, explicit .gitleaksignore, null report)

The trial job now runs its steps under `shell: bash` (GitHub's
`bash --noprofile --norc -eo pipefail {0}`), so a failing check inside a
pipeline fails its step. Both scans pass --max-archive-depth 0 (gitleaks
8.30.1 does not open archives by default; betterleaks 1.8.1 opens eight
levels) and --gitleaks-ignore-path .gitleaksignore (betterleaks
cmd/root.go lines 100, 103, 281-291 and 465-469).

tests/test_gitleaks_config.py reads a JSON report of `null` as no
findings: betterleaks writes an empty report that way (cmd/git.go line
74, cmd/directory.go line 72, report/json.go), gitleaks writes `[]`
(detect/detect.go line 127) and is unaffected (25 of 25 ported fixture
tests and 28 of 28 module tests pass with gitleaks 8.30.1).
BetterleaksTrialJobTests grows to 7 tests: the bash default, the
verification order down to the pinned digests and certificate flags, the
exact fixture classes, both added flags, --redact without a lower value,
and no suppression channel that only betterleaks reads; 17 mutations each
fail their test.

The receipt records the review round: the live ruleset read-back, the
explorer line sizes behind gitleaks's 3,000,000-byte fragment skip, the
job-argv and one-fingerprint ignore controls, the .git skip control, the
ported fixtures (22 ok, 3 FAIL), a bash-pipefail simulation, value-blind
passphrase facts with owner confirmation pending, the first round's
retained checks, and every script that produced a recorded result.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ssertion failures reported; passphrase rows resolved

Coordinator repair before the pull request opens (integrity wave, plan move M3).

- validate.yml: both scans pass betterleaks's own --exit-code 0 (cmd/root.go
  line 82 at v1.8.1). A scan error still exits 1 (lines 640-641) and an
  unwritable report is fatal (line 636), and each step exits with the
  scanner's status. The fixture step turns unittest's exit status 1 into 0
  only when its closing line is FAILED (failures=N), optionally with
  skipped=N: unittest counts an AssertionError as a failure and any other
  exception as an error. The steps append counts to the step summary:
  findings per rule id, never a value. No continue-on-error; the precedent
  is security-scan.yml's zizmor --no-exit-codes.
- tests/test_workflow_hardening.py: BetterleaksTrialJobTests asserts the
  flag and the exit paths, and runs the steps under GitHub's bash command
  with a stand-in scanner and a stand-in fixture module. 42 mutation
  controls (17 kept, 25 new) each fail their test and only that test.
- Receipt: real_secret_status is disposable_test_values_no_retained_repository,
  from the coordinator's host observation (our-integration; no values, host
  paths or usernames) with a value-blind recheck. README section 5,
  parity.json and its generator, run-record.json coordinator_repair; harness
  exit_code_controls.sh is new, mutation_controls.py and sim_job.py updated.

Local simulation: every run step returns 0 (fixture step: Ran 25 tests;
FAILED (failures=3), reported; scans: 91 and 16 findings). zizmor 1.30.1
and actionlint 1.7.12 with shellcheck 0.11.0 are clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
… its repair-round files, and the changed workflow and tests

Re-registered through the hot-file protocol (docs/lanes.md): the manifest
is the base commit's copy plus register_file for the 25 files this branch
adds or changes (22 inserted, including the repair's new
harness/exit_code_controls.sh; 3 updated: validate.yml,
test_gitleaks_config.py, test_workflow_hardening.py). Then
component_matrix.py --write and new_host_grand_list.py --write (both left
their outputs unchanged) and scripts/validate.py (passed: 69 components,
7395 hashed files, 4 profiles, 159 receipts). This replaces the review
round's registration commit a14d1a35, which listed 24 of these files at
their earlier hashes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…error, so the fixture step fails on it

Cross-family GPT-6 review, P2 at validate.yml:425 (changes-needed):
_run_gitleaks turned an unexpected scanner exit into an AssertionError, a
unittest failure that the fixture step's failures-only filter accepts, and
it read exit status 1 without a report as no findings. Reproduced first
(harness/scanner_error_probe.py on the tree at 1c02980): scanner exits
-11, 139 and 1 mocked in memory on fixture test_a, then the step's own
status handling under bash --noprofile --norc -eo pipefail -c, each gave
FAILED (failures=1) and step exit status 0.

Exit statuses, read from the pinned sources (the local copies are
byte-identical to the files at the tags):
- gitleaks v8.30.1 cmd/root.go 463-491, betterleaks v1.8.1 cmd/root.go
  610-638: "// write report if desired / if detector.Reporter != nil {"
  writes the report before any exit; "failed to write report" is fatal
  (489, 636).
- 493-495 and 640-642: "if err != nil { os.Exit(1) }"; 497-499 and
  644-646: "if len(findings) != 0 { os.Exit(exitCode) }", and every test
  scan passes --exit-code 0.
- zerolog v1.33.0 log.go 389-398: Fatal() ends in "os.Exit(1)", also
  with the level off (482-487). 226-228 and 325-327: "unknown flag" exits
  126. Go go1.25.12 src/runtime/panic.go 1287 and 1336: a runtime crash
  is "exit(2)".
- 352-356 and 485-489: Detector() runs os.Create then os.Remove on the
  report path before the scan, so a scan that dies leaves no report;
  report/json.go 13-16 always encodes a value.
So under --exit-code 0, exit status 1 is never a findings status in
either tool, and exit status 0 always comes with a report.

tests/test_gitleaks_config.py now runs every scan through _scan_findings.
It builds the command with --exit-code 0 and a JSON report, and raises
_ScannerError (an Exception, not an AssertionError: CPython
Lib/unittest/case.py _addError records it as an error) for any nonzero
exit status and for a missing or empty report after exit status 0.
_run_gitleaks, the --exit-code 0 test, the git-mode fingerprint test
(which never read the exit status) and the history-ancestry test go
through it. A busy-lock skip now needs the guarded launcher's contract,
exit status 75 and "another scan holds the per-user lock"
(adoption/tools/gitleaks-guarded 23-32, gitleaks-guarded-macos 29 and
124-130); before, any stderr containing "lock" skipped, such as the Go
runtime's "all goroutines are asleep - deadlock!".

validate.yml: the fixture step's comment states this; its name, commands
and filter are unchanged, and so is the required secret-scan job's YAML.
That job's unittest step is stricter: the history-ancestry test now
errors on a scan that did not complete instead of reading zero findings.
On a host with the guarded launcher, "could not acquire the per-user
lock" (a status other than 75) is now an error, not a skip.

After: the probe's -11, 139, 1 with and without a report, exit 0 without
a report and the deadlock exit each give FAILED (errors=1) and step exit
1; a completed scan that misses the value still gives FAILED
(failures=1) and step exit 0. The whole fixture step over the real
module with a stand-in scanner: FAILED (errors=20, skipped=1) and step
exit 1 in six modes (before: failures=15 or 20, or OK (skipped=21), and
step exit 0).

New tests: ScannerErrorTests (5) and BetterleaksTrialJobTests.
test_fixture_step_fails_when_a_scan_does_not_complete_in_the_real_fixture_module.
harness/scanner_error_controls.py: 6 controls, each failed every named
test, module restored byte-identical. gitleaks 8.30.1 on PATH:
tests.test_gitleaks_config ran 33 tests, OK, none skipped. betterleaks
1.8.1 (verified binary) through the fixture step (harness/sim_job.py):
step exit 0, Ran 25 tests; FAILED (failures=3), d2, d4 and d5, no error,
no skip. harness/final_checks.sh: tests.test_workflow_hardening 72 tests
OK (skipped=2), freshness-pin and workflow-security modules OK, the 42
earlier controls failed their tests again, zizmor 1.30.1 "No findings to
report" (43 suppressed), actionlint 1.7.12 with shellcheck 0.11.0 and
shellcheck on the harness printed nothing.

Receipt: README.md section 6 and run-record.json scanner_error_repair
(coordinator_repair's limit marked superseded). The next commit
re-registers the files in manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Hot-file protocol (docs/lanes.md): register_file for the 7 files the
scanner-error repair (11b4673) adds or changes, on top of the branch's
registration commit 1c02980. 2 inserted: harness/scanner_error_probe.py
and harness/scanner_error_controls.py. 5 updated: validate.yml,
tests/test_gitleaks_config.py, tests/test_workflow_hardening.py, and the
receipt's README.md and run-record.json. Then component_matrix.py --write
and new_host_grand_list.py --write (both left their outputs unchanged) and
scripts/validate.py (passed: 69 components, 7397 hashed files, 4 profiles,
159 receipts). On rebase, fold this commit and 1c02980 into one last
registration commit: take main's manifest and re-register the branch's
files.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…on time, lock-skip change

Verifier residuals on the scanner-error repair:
- The gitleaks-guarded-macos busy branch is lines 125-130, not 124-130; line 124 is
  `return descriptor`.
- The repair window's last_recorded_run becomes 08:33:28Z, the real-home read-back.
- behaviour_changes gains the lock-skip change: stderr that merely contains "lock" no
  longer skips a completed scan (gitleaks v8.30.1 sources/git.go lines 251-264 forward
  git stderr as warnings or errors).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…e protocol

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 28, 2026
Scout and others added 4 commits September 28, 2026 05:32
…mes no secret

The required secret-scan job's unittest step runs GitleaksBranchAncestryHistoryTests over
the real history. That scan had no --redact, and its assertion message interpolated the
findings (Secret, Match, Line), so a real secret in HEAD's ancestry would reach the public
job log. The problem predates this branch (present on main since ba1700a); the
scanner-error repair's verifier found it.

- The scan passes --redact. Sources: gitleaks v8.30.1 report/finding.go lines 78-86
  (Finding.Redact masks Secret, Match and Line) and detect/detect.go line 367; betterleaks
  v1.8.1 cmd/root.go line 589 (RedactFindings).
- A failure lists (RuleID, File, StartLine, Fingerprint) only.
- ScannerErrorTests.test_f checks both halves with an in-memory scanner and a deliberately
  unredacted report. Reverting the message, or dropping --redact, each fails it.
- The full module, with gitleaks 8.30.1: 34 tests OK, no skips.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…e protocol

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…reason

The fixture step's comment, and the hardening test's docstring, said the history-ancestry
class scans without --redact and quotes its findings. That stopped being true when the
class started redacting (ScannerErrorTests.test_f). The class stays out because the job's
redacted history scan covers that ground. Comment and docstring only; the test name is
kept because the recorded controls cite it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…e protocol

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins
seathatflowsinourveins merged commit eb67828 into main Sep 28, 2026
26 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/betterleaks-parity-20260927 branch September 28, 2026 10:31
seathatflowsinourveins added a commit that referenced this pull request Sep 28, 2026
…-28 (#457)

* Grand dashboard: integrity-wave gate rows (M2-M6) and checkpoint 2026-09-28

Adds five gate rows for accepted work merged on 2026-09-28 (#448-#452, #455):
the open sandbox-profile gate, the security-audit skill trial, the actionlint and
betterleaks successor trials, and the CodeQL quality-suite decision. Each row links
its own receipt or decision. Re-registers state.json in manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Grand dashboard: correct the sandbox registration date and the betterleaks swap condition

Independent review: the open gate was registered 2026-09-27 (catalogs/foundation/manifest.json:254),
and the betterleaks swap waits on parity after triage (run-record.json plan_move), not triage alone.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants