Repository navigation
ci: separate frozen evidence OSV risk reporting - #845
Merged
seathatflowsinourveins merged 3 commits intoOct 8, 2026
Merged
Conversation
seathatflowsinourveins
force-pushed
the
codex/ns2604-frozen-osv-report-20261007
branch
from
October 8, 2026 07:11
de41507 to
d87f6c9
Compare
seathatflowsinourveins
marked this pull request as ready for review
October 8, 2026 07:12
|
Dependency limit exceeded — report not shown. This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report. Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard. Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account. |
seathatflowsinourveins
deleted the
codex/ns2604-frozen-osv-report-20261007
branch
October 8, 2026 10:04
5 of 7 tasks
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 8, 2026
### Scope Carry the owner's directly approved fourth rule into every live instruction surface. Requests become a starting point for evidence-based related improvements, with timing that preserves the current focus and protected windows. The exact approved text replaces the previous fourth bullet across six instruction files and the maintained top-rule decision quotations. - Base: `eb5fee4db9a494729deb29bdf38dbabac23ee643`. - Head: `efd6b717f066f20c77fe604a37932c3adee6d8e4`. - Lane: `lane:shared`; the shared core applies to both lanes, with the other lane's acknowledgement pending before any landing. - Authority: owner-selected "Tightened" option, CC `task-ns2604-coop-20261008T134651Z`. - Owned changes: six rule surfaces, the top-rule decision record, two pinned test modules and the evidence registry. The final commit contains both shared hot files, `AGENTS.md` and `manifests/evidence.json`. - Exactly one PR; landing and the host F9/fresh-session steps remain on the command center's cue through 5f. ## SOTA sources - [ECC search-first@`2b6e839771e53096d8451a213d40dc64ec8acac0`](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/skills/search-first/SKILL.md), Workflow and Decision Matrix; reuse existing carriers, renderer and tests. - [Claude Code best practices](https://code.claude.com/docs/en/best-practices#write-an-effective-claude-md) and [memory/rule organization](https://code.claude.com/docs/en/memory#organize-rules-with-clauderules), fetched October 8: broadly applicable rules load every session and stay concise. [Failure patterns](https://code.claude.com/docs/en/best-practices#avoid-common-failure-patterns) support preserving focus. - [Official OpenAI documentation: Codex AGENTS.md loading](https://developers.openai.com/codex/guides/agents-md), fetched October 8; installed `codex-cli 0.161.0` and reviewed [openai/codex rust-v0.161.0 loader](https://github.com/openai/codex/blob/rust-v0.161.0/codex-rs/core/src/agents_md.rs). - Existing supported integration at preparation base: `tools/adoption/new_wsl_client_config.py` F9, `scripts/host_receipts.py:register_file(root,relative_path)`, and the existing test helpers `template_segments()`/`startup_files()`. - Historical evidence references: `docs/decisions/2026-10-07-instruction-core.md` and `docs/decisions/2026-09-25-model-fallback-guard.md:105–115,166–175`. The October 7 record cites the September StructuredOutput comparison; it is not a new October 7 run or an experiment on this bullet. ### Evidence-class table | Claim | Evidence class | Observation | | --- | --- | --- | | Exact approved wording on all six live instruction files | `local_integration` | Native text/pin/budget tests; independent source/diff review | | Full search census and untouched historical artifacts | `source_review` + `local_integration` | All 11,538 tracked files searched: 33 matching files / 93 lines; all 25 historical files byte-equal to the base | | New core behavior matches owner direction | Owner policy decision | Exact tightened text and four dated owner excerpts in the amendment | | Historical 5/30 vs0/30 schema errors | Historical reported native result | Sonnet 5 / Claude Code 2.1.282; primary untracked captures were not independently re-read here | | New bullet improves adherence or is loaded in fresh sessions | Pending | No new model trial, host render or fresh-session check in this PR | ### Complete surface census The maintained files are `AGENTS.md`, `adoption/scaffold/AGENTS.md`, `adoption/templates/codex.AGENTS.template.md`, `adoption/new-wsl/codex-user-instructions.md`, `adoption/new-wsl/claude-user-instructions.md`, `examples/claude-native/CLAUDE.md`, and `docs/decisions/2026-09-28-top-rule-templates.md`. Thus `surfaces=7`: six instruction files plus the decision record. The searched test literal is `tests/test_install_claude_profile.py`; `tests/test_codex_worker_lane.py` changes its hash/budget-related expectation. The [amendment's full census](docs/decisions/2026-09-28-top-rule-templates.md#complete-search-census-and-frozen-files) names every matching file. These 25 historical evidence files were searched and are **untouched**: - `evidence/artifacts/prompt-audit-20260927/lane-a/adjudication/attempt2/adjudication-inputs/x5.AB.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/adjudication/attempt2/adjudication-inputs/x5.BA.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/adjudication/attempt2/judge-actions.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/adjudication/attempt2/packets/x5.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/adjudication/attempt2/prompts/judge-AB.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/adjudication/attempt2/prompts/judge-BA.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/adjudication-inputs/x5c.AB.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/adjudication-inputs/x5c.BA.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/amendment.diff` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/gpt6.BA.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/packets/x5c.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/prompts/judge-AB.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/prompts/judge-BA.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/claude.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/packets/packet.md` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/prompts/gpt6-prompt.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/proposal.diff` - `evidence/artifacts/prompt-audit-20260927/lane-a/final/tally-final.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/claude-brief.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/claude.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/gpt6-prompt.txt` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/gpt6.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/packet.md` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/proposals.json` - `evidence/artifacts/prompt-audit-20260927/lane-a/round1/user-level-top-rule.txt` The three decision quotations are labelled amended projections; the original executed bytes, measurements and adjudications remain in those frozen files. ### Changed expectations, test by test The [dated declaration](docs/decisions/2026-09-28-top-rule-templates.md#changed-expectations-test-by-test) records exact test IDs, old/new values and classification, following the CC's #845 rule. | Test | Change | | --- | --- | | `StandingRuleSurfacesTests.test_the_portable_block_is_exactly_the_core_and_the_evidence_backed_line` | `CORE` takes the exact new bullet; 1,262 → 1,486 bytes | | `StandingRuleSurfacesTests.test_every_layer_carries_the_same_core_and_no_dropped_rule` | Same exact core across all six layers; once-only/exclusion conditions retained | | `StandingRuleSurfacesTests.test_the_repository_file_keeps_its_core_and_the_trading_prerequisite` | Same required core prefix; trading prerequisite retained | | `TemplateTests.test_top_rule_is_pinned_and_rendered_rtk_is_the_unchanged_pinned_source` | `TOP_RULE_SHA256`:2618406a…c91fb62→bc9f3138…bd2aae7; RTK bytes/pin retained | | `PortableTopRuleTests.test_rendered_startup_files_fit_each_clients_fixed_byte_budget` | Claude 4,140 → 4,610; Codex 4,246 → 4,716 bytes, same 105% procedure | | `PortableTopRuleTests.test_growth_in_any_loaded_file_crosses_the_fixed_budget` | Same ceilings; negative growth condition retained | | `TemplateTests.test_agents_template_is_one_managed_block` | Comment-only measured size 2,105 → 2,329; 8,192-byte ceiling retained | No fixed word-count assertion exists in these native tests. Measured template words 286 → 323 and top-segment words 186 → 223 are recorded rather than inventing one. ### Local commands run ```sh python3 -m unittest tests.test_install_claude_profile.StandingRuleSurfacesTests tests.test_codex_worker_lane.TemplateTests tests.test_install_claude_profile.PortableTopRuleTests # unchanged main expectations after text edits: exit 1; 32 tests, 12 expected failing subtests python3 -m unittest tests.test_install_claude_profile tests.test_codex_worker_lane tests.test_new_wsl_client_config tests.test_scaffold_repo tests.test_managed_block # initial integrated run: 521 tests; 506 passed, 13 skipped, 2 failed # private-log shell mistakenly gave fixtures umask 077; the two unchanged RoleStepTests expected 0750 but got 0700 umask 022 python3 -m unittest tests.test_codex_worker_lane.RoleStepTests.test_an_existing_agents_directory_and_identical_roles_are_not_the_runs_to_remove tests.test_codex_worker_lane.RoleStepTests.test_rollback_keeps_an_agents_directory_that_existed_before_the_run # focused corrected-environment run: 2 tests, exit 0 # same two failures reproduced with base surfaces under 077; no assertion or implementation was weakened python3 scripts/validate.py # exit 0: 69 components, 10,831 hashed files, 4 profiles, 228 receipts git diff --check # exit 0 ``` Final per-test outcomes are **508 pass / 13 skip across 521 distinct test IDs**, combining the initial run with the two focused environment corrections. This is not a claimed single full-suite green run. Original failed attempts remain retained. The corrected umask affects execution only; it introduces no repository/test exception. Command-center/5f main-version tests against this exact head remain separate. ### Decision record `docs/decisions/2026-09-28-top-rule-templates.md`, dated 2026-10-08 amendment: owner approval, repeated October 4/5/8 words, exact text, evidence correction, complete census, expectation declarations, alternatives, inverse and overturn conditions. Completeness review corrected an unsupported CLI spelling of `register_file` to the actual Python helper. ### Host evidence Pending after landing: CC runs F9 for the two user-level files, then fresh native sessions in both clients that visibly load the exact bullet. Optional promptfoo adherence A/B remains a separate api-credit-sota action. No host receipt was changed. ### Checklist - [x] Exact owner wording and all maintained copies agree. - [x] Historical evidence is untouched and the whole matching set is named. - [x] Every changed expectation is declared; the shared hot files/registry are the final commit. - [x] Native existing tests and required validation are retained with their actual outcomes. - [x] Credential values and other owners' work are preserved; no new paid service. - [ ] Main-version tests, other-lane acknowledgement and landing are on the command center's cue. - [ ] F9 render and fresh-session load checks occur after landing.
5 of 7 tasks
seathatflowsinourveins
added a commit
that referenced
this pull request
Oct 8, 2026
<!-- Scope and base commit --> ### Scope - When the frozen OSV scan completes with findings (native exit 1), its advisory check currently fails on unrelated PRs. This change reports those findings successfully while keeping scanner and report errors failing. - Base commit: `89e89852161d8157e464f4bd07c2b5c747ad4706` - Lane: `lane:foundation` - Owned paths touched: frozen report workflow, its existing recording-double test, dated decision, measurement receipt and their evidence registrations. ### SOTA sources - [google/osv-scanner v2.6.0](https://github.com/google/osv-scanner/blob/e840a6e8adb14b7777c78e26cfbf6e2abc1d1fc6/cmd/osv-scanner/internal/cmd/run.go): native findings return 1; operational failures return 127–130. The workflow preserves both raw codes and normalizes only completed-report findings. - [GitHub shell and step semantics](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idstepsrun) and [job conditions](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-jobs-with-conditions): explicit exit handling and unchanged always/trusted-event guards. - [PR #845](#845), its dated frozen-report design and [native baseline run 37821865072](https://github.com/seathatflowsinourveins/native-agent-stack/actions/runs/37821865072): nine retained advisories, JSON/SARIF exits 1/1, no operational error. ### Evidence-class table | Claim | Evidence class | Command / receipt | | --- | --- | --- | | Hosted before fails only on frozen findings | `native_proven` | Retained GitHub artifact digest `07cbf215…`, native run/job and input binding in the receipt | | Finding-only report exits become 0; raw data/codes/logs stay identical | `synthetic` | Eight extracted-shell before/after cases; additional vendor 128/129 error cases | | Coverage, error and privilege controls pass | `local_integration` | 254 focused tests (one existing skip); final expanded method separately passes; actionlint 1.17.0 | | Inverse restores base bytes and forward patch restores candidate | `local_integration` | Native inverse round trip, both owned file digests verified | | Hosted after | Unobserved | Await this exact PR's native report check and explicit CC workflow landing cue | ### Local commands run ``` python3 -m unittest tests.test_frozen_evidence_reporting tests.test_osv_lockfile_coverage tests.test_frozen_macos_variant_no_use tests.test_workflow_hardening tests.test_workflow_policy tests.test_workflow_security_coverage -q # 254 tests, rc 0, one existing skip; before the final vendor-code test extension python3 -m unittest tests.test_frozen_evidence_reporting.FrozenReportWorkflowTests.test_native_findings_errors_and_both_logs_are_retained -v # final 128/129 extension: rc 0 actionlint .github/workflows/frozen-evidence-risk.yml # rc 0 python3 scripts/validate.py # rc 0: 70 components, 10878 hashed files, 4 profiles, 235 receipts git apply --check <owned inverse patch> # inverse and forward round trip passed; base/candidate file hashes matched ``` ### Decision record `docs/decisions/2026-10-08-frozen-risk-report-result.md` and `evidence/artifacts/github-ci-opt6-report-status-20261008.json`. Intentional kind 3 contract change: only `FrozenReportWorkflowTests.test_native_findings_errors_and_both_logs_are_retained`. Running all 13 unchanged base reporting methods on this candidate produces exactly its two declared finding-only subtest failures, JSON1/SARIF0 and JSON0/SARIF1; zero errors. Its original invocation/code/log assertions remain. ### Host evidence No new host receipt or status flip. The actual hosted before artifact is retained separately from synthetic after controls. The archived input, empty report config, ordinary required scan, raw serialization, artifact retention and trusted write guards keep their existing bytes/contracts. ### Checklist - [x] Action pins and version comments preserved. - [x] Empty default permissions and existing least-privilege job grants preserved. - [x] No credential values, new required secret, provider or billing surface. - [x] Peer work preserved; isolated owned worktree. - [x] Inverse tested and receipt retained. - [ ] Native hosted after recorded. - [ ] Explicit CC workflow landing cue through 5f. Revert this change through a reviewed inverse PR; findings stay visible in native JSON/SARIF/logs and review alerts. Scanner errors and unavailable/malformed reports remain failures. This PR remains draft while exact-head review and hosted checks finish.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Frozen lockfile findings currently block the required production OSV check. Keep production scanning required and report inactive frozen evidence through a separate, non-required workflow, with an owner-assigned review issue from trusted main runs. Required checks still verify frozen artifact identity, inventory coverage and inactive eligibility.
36654a8174b5cad1380bd13ed1fffc0f135aac2c(preparation base).lane:foundation.de41507bb1ea645d30c7978e111af847f05b8ede; registry-only commit last.2cad4d7407e0497d4eaa51dc3c4410ef112cd928). Landing follows the named command-center base/cue.SOTA sources
docs/configuration.md,internal/config/config.go,pkg/osvscanner/filter.goandpkg/osvscanner/vulnerability_result.go. The separate report uses an explicit empty native config and preserves frozen bytes.native-agent-stack@36654a8174b5cad1380bd13ed1fffc0f135aac2c:.github/workflows/saturation-tracking.yml:128. The issue links alerts, assignsseathatflowsinourveinsand names command-center review within 24 hours.Evidence-class table
source_reviewlocal_integrationsyntheticlocal_integrationLocal commands run
Passing evidence is retained from the unchanged prepared head. The latest per-test outcomes above combine the original run and focused repairs; they are not a single full-suite pass. Original failed attempts remain in the private receipt. Hosted acceptance remains pending.
Decision record
docs/decisions/2026-10-07-frozen-evidence-osv-reporting.mdrecords the accepted policy, changed contract IDs, alternatives and overturn conditions. Inverse after adoption: revert the source change and re-register its files, restoring the previous required frozen-scan policy.Host evidence
No host-receipt change.
Checklist