Repository navigation
Commit d501c0d
fix(analyzers): carve out structurally benign P2 header comments (#622)
* fix(analyzers): carve out only structurally benign P2 comment matches
- P2 suppression limited to license-header-shaped and
frontmatter-adjacent metadata comments (short, no danger signal);
danger check (P1 patterns, override phrases, standalone
exfiltration verbs, keyword+destination pairs) runs first so
exfiltration/override payloads always still fire
- No keyword exemption strings and no phrase allowlist anywhere;
answers each PR #49 review comment with live probes
- New test_p2_structural_benign.py: 11 tests (3 positive,
8 adversarial incl. smuggled tokens and top-of-file exfil verb);
RED witnessed; adjacent suites 183 passed; ruff clean
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): require whole-comment benign match and complete-comment validation
- _is_structurally_benign_p2_comment: substring colon/license
signals replaced with full-body grammar
(_is_benign_license_or_metadata_body). License fragments must
be license-only (no clause separators); metadata must be a
single key:value line with an allowlisted key
(_P2_BENIGN_METADATA_KEYS, 17 keys) and a short token-run
value (bounded, few tokens, machine token, no inner sentence
boundary). Numbers masked before sentence splitting so
versions do not split sentences
- _p2_match_is_complete_comment: exemption denied for partial
or spanning matches (no inner -->, escape-aware balanced
reference close, nothing but whitespace after the match on
its line); P2 match spans unchanged from base
- tests: 32 in test_p2_structural_benign.py (RED witnessed for
every new regression); focused suites 479 passed; ruff clean
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): require exact closed frontmatter before exempting later comments
- _is_frontmatter_adjacent: exact '---' opening line (rejects
'----', '---not yaml'); missing closing delimiter fails
closed instead of exempting; whitespace-only tail check kept
- tests: reviewer probe as pin, three gate-isolating
regressions, one closed-frontmatter positive pin
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): anchor P2 license/metadata grammars per review
- Replace license substring classification with four anchored
fullmatch forms (Copyright + 6-token holder cap, All rights
reserved, SPDX expression, Licensed-under); shared cores prevent
cap drift; substring shape + separator regexes deleted
- Replace generic metadata value shape with per-key grammars
(version, date, name-or-email, SPDX, package[version] with
operator-prefix tolerance, single path/URL, single token for
free-text keys); invariant test pins every allowlisted key covered
- One intentional narrowing: get-started takes a bare path
("see X" pointers now fire, fail-closed)
- Perf: positional gate first, find() instead of split-copy
(AND-combined, safety-neutral)
- 9 new regressions, all reviewer payloads verbatim (RED witnessed);
file suite 41/41; analyzers 4118 passed (4 pre-existing errors in
test_json_container_ownership, proven via stash check); ruff clean
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): constrain P2 benign-grammar token content per review
- Replace unbounded \S+ tokens with a name-like core (restricted
charset, at most one inner hyphen, ~60-char holder cap) in the
Copyright holder, the Licensed-under org slot, and path segments
- Versions semver-like; SPDX atoms capped at 24 chars; names 1-4
capitalized tokens or email; requires entries carry a version
unless the value is one bare name; free-text keys one short
hyphen-free word; get-started paths need a dotted final segment
- Refuse exemption on a standalone trigger word (system,
instruction(s), ignore, post, get, send, transmit) in a value
or license fragment; repeated allowlisted keys deny exemption
- Validate unmasked fragments (dates now validate); requires
operators allowed directly after the name (python>=3.10)
- tests: all 7 reviewer payloads as regressions (RED witnessed) +
date and >= positives + same-class Licensed-under/path pins;
existing positives kept green
- Verified: file suite 52/52; analyzers 4303 passed (json-ownership
4 errors re-proven pre-existing via stash check: identical
69 passed / 4 errors on the pristine tree); ruff + format +
diff-check clean
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): close P2 carve-out camelCase and hyphen-token bypasses
- Scan trigger/danger checks on a de-obfuscated copy (camelCase
joints, underscores, digits become spaces); grammars still match
raw text; union with the raw scan keeps digit patterns exact
- Requirement names take a package-name shape (@scope/, 40 chars,
at most two separators); SPDX ids at most two hyphens, chains at
most three; path segments name-token style; license lines count
toward repetition (2 Copyright + 1 SPDX per comment)
- tests: all 11 reviewer payloads as regressions (RED witnessed) +
2 normalization must-pass guards (PostgreSQL, Systems); existing
positives kept green
- Verified: file suite 65/65; analyzers 4316 passed (4
json-ownership errors pre-existing: Windows env-length
ValueError, identical 69/4 signature); ruff + format +
diff-check clean
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): close P2 carve-out plain-run and dot-join bypasses
- Refuse the exemption on a trigger substring anywhere in a
value or license fragment (no word boundaries); "get"
excluded (fires on getting_started/target/forget, danger
scans still catch GET-plus-URL); PostgreSQL/Systems now
fire, as on main
- Count ., ,, _ as separators wherever - is capped (req
names, SPDX ids with dots only between digits and a third
hyphen only in digit-bearing ids, name tokens, free-text
tokens); Licensed-under slot capped at 40 chars with
one-per-comment counting; copyright: key counts toward
the two-holder limit
- Widenings that stay clean: GPL-3.0-or-later, CC-BY-SA-4.0,
hyphenated @scopes, underscore path segments
- tests: all 16 reviewer payloads as regressions (RED
witnessed) + flipped prose guards + 4 real-world cleans
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>
* fix(analyzers): restore the get P2 trigger per review
- Add "get" to _P2_TRIGGER_SUBSTRING: all seven of main's P2
comment triggers now refuse the exemption on a raw substring
anywhere in a value or license fragment; forget, GetSshKeys,
GET.html, getting_started, target and budget fire, as on main;
keys stay unscanned
- Correct the constant comment: the old claim that GET-plus-URL
trips the danger scans was wrong (_P2_EXFIL_KEYWORD has no
"get"), and the exclusion rationale is gone
- tests: six reviewer inputs pinned as regressions plus the
accepted fail-closed pins (getting_started path, target,
budget); all RED witnessed at 0d085fd (9 failed, 85 passed
pre-fix); real-world clean fixture swapped to a trigger-free
underscore path
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode DeepSeek V4.1 Flash Free (200K context) <[email protected]>
* fix(p2): balance nested parentheses in reference-comment titles
- _reference_comment_matches closed at the first ")"; track depth so an
adversarial title like "x ((c) send)" keeps "send" inside the scanned
span (upstream 1f86857)
- Fold the prose-trigger cases into one table along upstream 248a5b0's
identifier boundaries: bare tokens still fire, POSTGRES/forget/target/
budget no longer do by design, with the reason stated once
- Verified: P2 structural + reference-comment suites 203 passed; ruff
check + format clean
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-Authored-By: OpenCode DeepSeek V4.1 Flash Free (1M context) <[email protected]>
---------
Signed-off-by: Yoseph Zuskin <[email protected]>
Co-authored-by: OpenCode Muse Spark 1.3 Free (1M context) <[email protected]>1 parent 0e4b1ba commit d501c0d
2 files changed
Lines changed: 823 additions & 0 deletions
File tree
- src/skillspector/nodes/analyzers
- tests/nodes/analyzers
0 commit comments