Skip to content

Commit 2c14261

Browse files
chrisknvidiarng1995claude
authored
feat(python): classify executable source surfaces (#579)
* fix(analyzer): stabilize TM1 window identity Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): track eager receiver side effects Signed-off-by: Christopher Kevin <[email protected]> * feat(python): refresh execution surfaces on current main Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve lexical TM1 when dataflow abstains Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): require a retained companion owner Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve execution scope in binding evidence Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve lexical TM1 across conservative Python analysis Signed-off-by: Christopher Kevin <[email protected]> * style(analyzer): simplify binding event deduplication Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve runtime and retained shell evidence Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve runtime and retained shell evidence Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve called-slot identity and cached method evidence Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve called-slot identity and cached method evidence Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve cached subprocess slot state across scoped imports Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve cached subprocess slot state across scoped imports Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): reconcile explicit cached-slot evidence across scopes Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): reconcile explicit cached-slot evidence across scopes Signed-off-by: Christopher Kevin <[email protected]> * fix: abstain from cached slot proof after eager effects Signed-off-by: Christopher Kevin <[email protected]> * fix: abstain from cached slot proof after eager effects Signed-off-by: Christopher Kevin <[email protected]> * test: retain lexical findings after unknown cached-slot effects Signed-off-by: Christopher Kevin <[email protected]> * test: retain lexical findings after unknown cached-slot effects Signed-off-by: Christopher Kevin <[email protected]> * fix: bound direct called-slot evidence to known execution order Signed-off-by: Christopher Kevin <[email protected]> * fix: bound direct called-slot evidence to known execution order Signed-off-by: Christopher Kevin <[email protected]> * fix: limit direct slot proof to single assignment targets Signed-off-by: Christopher Kevin <[email protected]> * fix: limit direct slot proof to single assignment targets Signed-off-by: Christopher Kevin <[email protected]> * fix: invalidate slot evidence on protocols and unsafe releases Signed-off-by: Christopher Kevin <[email protected]> * fix: invalidate slot evidence on protocols and unsafe releases Signed-off-by: Christopher Kevin <[email protected]> * fix: keep legacy shadow proof isolated from later effects Signed-off-by: Christopher Kevin <[email protected]> * fix: keep legacy shadow proof isolated from later effects Signed-off-by: Christopher Kevin <[email protected]> * fix: abstain from cached replacement proof after unknown effects Signed-off-by: Christopher Kevin <[email protected]> * fix: abstain from cached replacement proof after unknown effects Signed-off-by: Christopher Kevin <[email protected]> * fix: retain native callable detection across cached slot stores Signed-off-by: Christopher Kevin <[email protected]> * fix: retain native callable detection across cached slot stores Signed-off-by: Christopher Kevin <[email protected]> * fix: retain shell findings for native receiver assignments Signed-off-by: Christopher Kevin <[email protected]> * fix: retain shell findings for native receiver assignments Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve native Popen replacement warnings Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve native Popen replacement warnings Signed-off-by: Christopher Kevin <[email protected]> * test: retain occurrence metadata in prose shell reports Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): require proven replacement before revoking lexical TM1 Address the three remaining review findings on lexical TM1 reconciliation, where a bound shell=True call still ran natively but lost its HIGH finding: - Untaken expression stores: a walrus in a short-circuited BoolOp operand, an untaken conditional-expression arm, or a comprehension/generator body is no longer recorded as replacement evidence. Only constant operands and tests prove that the store executed. - Identity-preserving stores: unpacked targets are paired with their RHS elements. Self-stores are transparent, truthy constants keep the flag, and native receiver aliases (saved = subprocess, import subprocess as saved, Popen = subprocess.Popen) keep the native receiver. - Deferred bodies: an outer store only proves replacement when every invocation the owner scope can reach sees it. Any reference to the function, method or class (compound statements, main guards, callbacks, other bodies) can invoke it from that point on, and decorators or lambdas can run once defined. This also covers a re-import before a later invocation. The release-trigger test for a called function now matches its module-level twin: the companion abstains and the lexical HIGH remains. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(analyzer): require proven replacement before revoking lexical TM1 Address the three remaining review findings on lexical TM1 reconciliation, where a bound shell=True call still ran natively but lost its HIGH finding: - Untaken expression stores: a walrus in a short-circuited BoolOp operand, an untaken conditional-expression arm, or a comprehension/generator body is no longer recorded as replacement evidence. Only constant operands and tests prove that the store executed. - Identity-preserving stores: unpacked targets are paired with their RHS elements. Self-stores are transparent, truthy constants keep the flag, and native receiver aliases (saved = subprocess, import subprocess as saved, Popen = subprocess.Popen) keep the native receiver. - Deferred bodies: an outer store only proves replacement when every invocation the owner scope can reach sees it. Any reference to the function, method or class (compound statements, main guards, callbacks, other bodies) can invoke it from that point on, and decorators or lambdas can run once defined. This also covers a re-import before a later invocation. The release-trigger test for a called function now matches its module-level twin: the companion abstains and the lexical HIGH remains. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: account for TM1 Python parsing in marker ownership ledger Main's #784 test expects the tool-misuse ledger event for an invalid Python file to carry obfuscated_instruction_text. This PR makes tool misuse parse Python for TM1 reconciliation, so the runner's higher-precedence syntax_error now names that partial event. Keep the fail-closed contract: assert the marker reading through the lexical-only anti-refusal analyzer and keep tool misuse partial with syntax_error. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: account for TM1 Python parsing in marker ownership ledger Main's #784 test expects the tool-misuse ledger event for an invalid Python file to carry obfuscated_instruction_text. This PR makes tool misuse parse Python for TM1 reconciliation, so the runner's higher-precedence syntax_error now names that partial event. Keep the fail-closed contract: assert the marker reading through the lexical-only anti-refusal analyzer and keep tool misuse partial with syntax_error. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(analyzer): keep one TM1 owner for true-prefixed names without AST proof Port the six true-prefixed TM1 regression tests from #577. This branch already carries test_true_direct_calls_on_one_line_keep_distinct_locations unchanged, so only the other five are added. Two of them reported the same call twice on this branch: - A variable window for a name spelled `true` (in any case) duplicated the case-insensitive direct `shell=True` owner at the same call. AST reconciliation removes that window only when the dataflow companion takes the call, so a file that does not parse, or a call after an unknown receiver effect, kept both findings. coalesce_path_findings now drops such a window when a direct lexical owner starts at the same call. A window for a call whose direct candidate was folded into an identical same-line call still reports that call. - A call-anchored window for a longer true-prefixed name (`true_value`) used its raw text as identity, so the raw and normalized security views of one call did not collapse, and reconciliation then moved both to the call. The window now takes its identity from the normalized view text, as the direct owner already does. Also cover both root causes with a test for a nested `true` call after an unknown receiver effect and a normalized `true_value` call in a file that does not parse. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(analyzer): keep one TM1 owner for true-prefixed names without AST proof Port the six true-prefixed TM1 regression tests from #577. This branch already carries test_true_direct_calls_on_one_line_keep_distinct_locations as test_true_direct_calls_around_safe_comprehension_keep_distinct_locations, so only the other five are added. Two of them reported the same call twice on this branch: - A variable window for a name spelled `true` (in any case) duplicated the case-insensitive direct `shell=True` owner at the same call. AST reconciliation removes that window only when the dataflow companion takes the call, so a file that does not parse, or a call after an unknown receiver effect, kept both findings. coalesce_path_findings now drops such a window when a direct lexical owner starts at the same call. A window for a call whose direct candidate was folded into an identical same-line call still reports that call. - A call-anchored window for a longer true-prefixed name (`true_value`) used its raw text as identity, so the raw and normalized security views of one call did not collapse, and reconciliation then moved both to the call. The window now takes its identity from the normalized view text, as the direct owner already does. Also cover both root causes with a test for a nested `true` call after an unknown receiver effect and a normalized `true_value` call in a file that does not parse. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(analyzer): report each true-prefixed shell call that #577 reports The `\b`-bounded direct pattern does not match `shell=true_value`, so on this branch such a call was reported only through a variable window. That window pairs an assignment with a single call, skips an assignment inside an earlier window, and was dropped outside Python. Compared with #577 this lost: - calls in JavaScript and Markdown files, fenced or plain; - a later call that the AST companion abstains on, for example after an unknown receiver effect; - all but the last same-line call in a file that does not parse, and the second of two identical same-line `shell=true` calls outside Python, which the direct candidate key folds into one; - a call whose assignment sits inside the window of an earlier one. _variable_shell_matches now pairs every assignment of a true-prefixed name with each call its bounded window reaches, and the nearest assignment owns each call. Candidate generation, analysis and AST reconciliation share that list, and a window's candidate identity uses its full text, so windows to different calls do not collapse on a shared 200-character preview. Outside Python, call-anchored windows for true-prefixed names are kept, and coalescing still drops one when a direct owner reports the same call. With an AST, each added window passes the same visibility, counterevidence, cached replacement and companion checks as before. Windows for other names are unchanged. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(analyzer): report each true-prefixed shell call that #577 reports The `\b`-bounded direct pattern does not match `shell=true_value`, so on this branch such a call was reported only through a variable window. That window pairs an assignment with a single call, skips an assignment inside an earlier window, and was dropped outside Python. Compared with #577 this lost: - calls in JavaScript and Markdown files, fenced or plain; - a later call that the AST companion abstains on, for example after an unknown receiver effect; - all but the last same-line call in a file that does not parse, and the second of two identical same-line `shell=true` calls outside Python, which the direct candidate key folds into one; - a call whose assignment sits inside the window of an earlier one. _variable_shell_matches now pairs every assignment of a true-prefixed name with each call its bounded window reaches, and the nearest assignment owns each call. Candidate generation, analysis and AST reconciliation share that list, and a window's candidate identity uses its full text, so windows to different calls do not collapse on a shared 200-character preview. Outside Python, call-anchored windows for true-prefixed names are kept, and coalescing still drops one when a direct owner reports the same call. With an AST, each added window passes the same visibility, counterevidence, cached replacement and companion checks as before. Windows for other names are unchanged. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(supply-chain): opt into Python source-type propagation SC2's literal-XOR decoding and SC3 branch on file_type, but the supply-chain analyzer was not in the USES_PYTHON_SOURCE_TYPE set, so Python executed through an extensionless shebang file or a shebang Markdown file received its suffix type and silently lost the decoded SC2 HIGH while completeness stayed true. Opt the module in and add graph-level parity tests for .py, .pyw, extensionless and .md surfaces. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(build-context): decode PEP 263 primary Python before UTF-8 rejection A selected primary file (direct .py/.pyw or extensionless Python shebang) declaring a supported PEP 263 encoding such as latin-1 was rejected as unsupported_primary_content because the generic primary check only accepts UTF-8. The later source-decoding pass then decoded it correctly, but the artifact stayed FAILED with a fatal event and 0% coverage. Waive the generic UTF-8 rejection when the bytes classify as Python and decode under their declared encoding; an unknown codec or bytes invalid for the declared codec remain fatal, and format checks (BOMs, archives, NUL density) are unchanged. The waiver is evaluated lazily and not after the shared deadline, so no post-cache Python work starts once the budget is spent. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Signed-off-by: Christopher Kevin <[email protected]> Signed-off-by: Christopher Kevin <[email protected]> Signed-off-by: Narendran Raghavan <[email protected]> Co-authored-by: Christopher Kevin <[email protected]> Co-authored-by: Narendran Raghavan <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
1 parent 320031c commit 2c14261

32 files changed

Lines changed: 6432 additions & 626 deletions

‎src/skillspector/artifacts.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,7 @@ class _ObfuscatedIgnoreState:
230230
".markdown",
231231
".txt",
232232
".py",
233+
".pyw",
233234
".sh",
234235
".json",
235236
".yaml",
@@ -403,6 +404,20 @@ def classify_artifact(path: str, data: bytes, *, referenced: bool = False) -> Ar
403404
}
404405

405406

407+
def promote_artifact_to_decoded_text(artifact: ArtifactRecord) -> None:
408+
"""Apply a successful format-aware text decode without erasing prior limits."""
409+
generic_binary_scope = artifact["content_kind"] is ContentKind.BINARY and (
410+
artifact["disposition"] is ArtifactDisposition.OUT_OF_SCOPE
411+
or artifact["disposition"] is ArtifactDisposition.PARTIAL
412+
and "reason" not in artifact
413+
)
414+
artifact["content_kind"] = ContentKind.TEXT
415+
artifact["decodable"] = True
416+
artifact["misleading_extension"] = _suffix(artifact["path"]) in _BINARY_EXTENSIONS
417+
if generic_binary_scope:
418+
artifact["disposition"] = ArtifactDisposition.ANALYZED
419+
420+
406421
def decode_text(data: bytes) -> str:
407422
"""Return the loss-tolerant local text projection for static analyzers."""
408423
return data.decode("utf-8", errors="replace")

‎src/skillspector/cli.py‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@
5959
)
6060
from skillspector.logging_config import get_logger, set_level
6161
from skillspector.mcp_registry import scan_registry
62-
from skillspector.models import Finding
62+
from skillspector.models import OCCURRENCE_FINDING_ID_KEY, Finding
6363
from skillspector.multi_skill import (
6464
MultiSkillDetectionResult,
6565
SkillDirectory,
@@ -1314,18 +1314,32 @@ def _cache_transitive_result(
13141314
child_filtered = _coerce_findings_list(child_result.get("filtered_findings"))
13151315
child_findings = _coerce_findings_list(child_result.get("findings"))
13161316
all_ids = {finding.finding_id for finding in [*child_filtered, *child_findings]}
1317+
all_ids.update(
1318+
occurrence_id
1319+
for finding in [*child_filtered, *child_findings]
1320+
for occurrence in finding.occurrences
1321+
if isinstance((occurrence_id := occurrence.get(OCCURRENCE_FINDING_ID_KEY)), str)
1322+
)
13171323
all_ids.update(_effective_finding_ids(child_result))
13181324
finding_id_map = {
13191325
finding_id: _scoped_finding_id(source_identity, finding_id) for finding_id in all_ids
13201326
}
13211327

13221328
def _scope_finding(finding: Finding) -> Finding:
1329+
occurrences = []
1330+
for raw in finding.occurrences:
1331+
occurrence = dict(raw)
1332+
occurrence_id = occurrence.get(OCCURRENCE_FINDING_ID_KEY)
1333+
if isinstance(occurrence_id, str):
1334+
occurrence[OCCURRENCE_FINDING_ID_KEY] = finding_id_map[occurrence_id]
1335+
occurrences.append(occurrence)
13231336
return replace(
13241337
finding,
13251338
finding_id=finding_id_map[finding.finding_id],
13261339
source_url=target,
13271340
source_identity=source_identity,
13281341
source_digest=source_digest,
1342+
occurrences=occurrences,
13291343
)
13301344

13311345
scoped_filtered = [_scope_finding(item) for item in child_filtered[:_TRANSITIVE_MAX_FINDINGS]]

‎src/skillspector/input_handler.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@
8282
_DIRECT_FILE_URL_SUFFIXES = (
8383
".md",
8484
".py",
85+
".pyw",
8586
".sh",
8687
)
8788

‎src/skillspector/inspection_ledger.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,8 @@ class LedgerReason(StrEnum):
108108
OUTPUT_LIMIT = "output_limit"
109109
TRANSITIVE_CHILD_SCAN_FAILED = "transitive_child_scan_failed"
110110
STATIC_PARSE_LIMIT = "static_parse_limit"
111+
PYTHON_SOURCE_AMBIGUOUS = "python_source_ambiguous"
112+
PYTHON_SOURCE_DECODE_ERROR = "python_source_decode_error"
111113
OBFUSCATED_INSTRUCTION_TEXT = "obfuscated_instruction_text"
112114

113115

@@ -221,6 +223,12 @@ class LedgerReason(StrEnum):
221223
LedgerReason.STATIC_PARSE_LIMIT: (
222224
"A security-relevant expression exceeded a bounded static parser's span limit."
223225
),
226+
LedgerReason.PYTHON_SOURCE_AMBIGUOUS: (
227+
"Python execution intent depends on runtime or platform-specific shebang semantics."
228+
),
229+
LedgerReason.PYTHON_SOURCE_DECODE_ERROR: (
230+
"Python source bytes could not be decoded under their declared encoding."
231+
),
224232
LedgerReason.OBFUSCATED_INSTRUCTION_TEXT: (
225233
"Obfuscated instruction text could not be fully evaluated by the deterministic layer."
226234
),

‎src/skillspector/models.py‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,11 @@ def _new_finding_id() -> str:
120120
return f"finding-{uuid4().hex}"
121121

122122

123+
OCCURRENCE_FINDING_ID_KEY = "_skillspector_report_finding_id"
124+
OCCURRENCE_CODE_SNIPPET_KEY = "_skillspector_report_code_snippet"
125+
_PRIVATE_OCCURRENCE_KEYS = frozenset({OCCURRENCE_FINDING_ID_KEY, OCCURRENCE_CODE_SNIPPET_KEY})
126+
127+
123128
@dataclass
124129
class Finding:
125130
"""Finding model for graph state and report output (shape aligned with to_dict)."""
@@ -218,7 +223,9 @@ def _serialized_occurrences(self) -> list[dict[str, object]]:
218223
]
219224
serialized: list[dict[str, object]] = []
220225
for raw in occurrences:
221-
occurrence = dict(raw)
226+
occurrence = {
227+
key: value for key, value in raw.items() if key not in _PRIVATE_OCCURRENCE_KEYS
228+
}
222229
if self.source_identity:
223230
occurrence.setdefault("source_identity", self.source_identity)
224231
if self.source_digest:

‎src/skillspector/nested_artifacts.py‎

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@
4141
LedgerRecordType,
4242
ledger_event,
4343
)
44+
from skillspector.python_ast import PythonSourceClassification, classify_python_source
4445

4546
ARCHIVE_MAX_DEPTH = 3
4647
ARCHIVE_MAX_MEMBERS = 1_000
@@ -80,6 +81,7 @@
8081
".phtml",
8182
".ps1",
8283
".py",
84+
".pyw",
8385
".pyc",
8486
".pyo",
8587
".rb",
@@ -114,6 +116,11 @@ class NestedInspectionResult:
114116
components: list[str] = field(default_factory=list)
115117
file_cache: dict[str, str] = field(default_factory=dict)
116118
raw_file_cache: dict[str, bytes] = field(default_factory=dict)
119+
# Classify with the archive member's execution path, while retaining the
120+
# virtual path as the stable cache/report key used by downstream analyzers.
121+
python_source_classifications: dict[str, PythonSourceClassification] = field(
122+
default_factory=dict
123+
)
117124
artifact_inventory: list[ArtifactRecord] = field(default_factory=list)
118125
metadata: list[dict[str, object]] = field(default_factory=list)
119126
outer_metadata: dict[str, dict[str, object]] = field(default_factory=dict)
@@ -539,14 +546,22 @@ def _record_outer_metadata(
539546
}
540547

541548

542-
def _virtual_type(path: str, data: bytes, nested_type: str | None) -> str:
549+
def _virtual_type(
550+
path: str,
551+
data: bytes,
552+
nested_type: str | None,
553+
source_classification: PythonSourceClassification,
554+
) -> str:
543555
if nested_type is not None:
544556
return nested_type
557+
if source_classification is PythonSourceClassification.PYTHON:
558+
return "python"
545559
suffix = Path(path).suffix.lower()
546560
return {
547561
".md": "markdown",
548562
".markdown": "markdown",
549563
".py": "python",
564+
".pyw": "python",
550565
".sh": "shell",
551566
".bash": "shell",
552567
".zsh": "shell",
@@ -1098,10 +1113,17 @@ def _inspect_zip_bytes(
10981113
executable = _member_executable(info, safe_name, member_data)
10991114
member_hidden = _is_hidden_path(safe_name)
11001115
concealed = executable and bool(concealment_reasons)
1101-
virtual_type = _virtual_type(safe_name, member_data, nested_type)
1116+
source_classification = classify_python_source(safe_name, member_data)
1117+
virtual_type = _virtual_type(
1118+
safe_name,
1119+
member_data,
1120+
nested_type,
1121+
source_classification,
1122+
)
11021123
result.components.append(virtual_path)
11031124
result.file_cache[virtual_path] = member_data.decode("utf-8", errors="replace")
11041125
result.raw_file_cache[virtual_path] = member_data
1126+
result.python_source_classifications[virtual_path] = source_classification
11051127
artifact = classify_artifact(virtual_path, member_data)
11061128
result.artifact_inventory.append(artifact)
11071129
result.metadata.append(

‎src/skillspector/nodes/analyzers/behavioral_ast.py‎

Lines changed: 76 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,12 @@
3232
)
3333
from skillspector.logging_config import get_logger
3434
from skillspector.models import AnalyzerFinding, Finding, Location, Severity
35-
from skillspector.python_ast import ParsedPythonFile, get_python_ast
35+
from skillspector.python_ast import (
36+
ParsedPythonFile,
37+
PythonSourceClassification,
38+
get_python_ast,
39+
resolve_python_source_classification,
40+
)
3641
from skillspector.state import (
3742
AnalyzerNodeResponse,
3843
SkillspectorState,
@@ -648,14 +653,42 @@ def node(state: SkillspectorState) -> AnalyzerNodeResponse:
648653
"""Parse Python files via AST and detect dangerous execution patterns."""
649654
components: list[str] = state.get("components") or []
650655
file_cache: dict[str, str] = state.get("local_file_cache") or state.get("file_cache") or {}
656+
raw_file_cache = state.get("raw_file_cache")
657+
source_classifications = (
658+
state.get("python_source_classifications")
659+
if "python_source_classifications" in state
660+
else None
661+
)
662+
source_classification_limitations = state.get("python_source_classification_limitations") or {}
663+
source_decode_failures = state.get("python_source_decode_failures") or {}
651664
python_ast_cache_key = state.get("python_ast_cache_key")
652665
all_findings: list[Finding] = []
653666
ledger_events: list[InspectionLedgerEvent] = []
654667
budget = _BehavioralBudget(state)
655668
terminal_limit: _BehavioralResourceLimitError | None = None
656669

657670
for path in components:
658-
if not path.endswith(".py"):
671+
content = file_cache.get(path)
672+
source_classification: PythonSourceClassification | None = None
673+
if source_classifications is not None and path in source_classifications:
674+
source_classification = resolve_python_source_classification(
675+
path,
676+
content,
677+
source_classifications=source_classifications,
678+
raw_file_cache=raw_file_cache,
679+
)
680+
if source_classification is PythonSourceClassification.NON_PYTHON:
681+
continue
682+
if path in source_classification_limitations:
683+
ledger_events.append(
684+
ledger_event(
685+
outcome=LedgerOutcome.PARTIAL,
686+
phase="behavioral",
687+
analyzer_id=ANALYZER_ID,
688+
path=path,
689+
reason=LedgerReason.RUNTIME_LIMIT,
690+
)
691+
)
659692
continue
660693
if terminal_limit is None and budget.analyzer_exhausted():
661694
terminal_limit = _BehavioralResourceLimitError(
@@ -665,11 +698,41 @@ def node(state: SkillspectorState) -> AnalyzerNodeResponse:
665698
"limit_findings": MAX_FINDINGS_PER_ANALYZER,
666699
},
667700
)
701+
if terminal_limit is None:
702+
try:
703+
budget.check_runtime()
704+
except _BehavioralResourceLimitError as exc:
705+
terminal_limit = exc
668706
if terminal_limit is not None:
669707
event = _partial_limit_event(path, terminal_limit)
670708
ledger_events.append(event)
671709
continue
672-
content = file_cache.get(path)
710+
if path in source_decode_failures:
711+
ledger_events.append(
712+
ledger_event(
713+
outcome=LedgerOutcome.PARTIAL,
714+
phase="behavioral",
715+
analyzer_id=ANALYZER_ID,
716+
path=path,
717+
reason=LedgerReason.PYTHON_SOURCE_DECODE_ERROR,
718+
)
719+
)
720+
continue
721+
if source_classification is None:
722+
source_classification = resolve_python_source_classification(
723+
path,
724+
content,
725+
source_classifications=source_classifications,
726+
raw_file_cache=raw_file_cache,
727+
)
728+
try:
729+
budget.check_runtime()
730+
except _BehavioralResourceLimitError as exc:
731+
terminal_limit = exc
732+
ledger_events.append(_partial_limit_event(path, exc))
733+
continue
734+
if source_classification is PythonSourceClassification.NON_PYTHON:
735+
continue
673736
if content is None:
674737
event = ledger_event(
675738
outcome=LedgerOutcome.FAILED,
@@ -725,10 +788,19 @@ def node(state: SkillspectorState) -> AnalyzerNodeResponse:
725788
)
726789
else:
727790
event = ledger_event(
728-
outcome=LedgerOutcome.COMPLETED,
791+
outcome=(
792+
LedgerOutcome.PARTIAL
793+
if source_classification is PythonSourceClassification.AMBIGUOUS
794+
else LedgerOutcome.COMPLETED
795+
),
729796
phase="behavioral",
730797
analyzer_id=ANALYZER_ID,
731798
path=path,
799+
reason=(
800+
LedgerReason.PYTHON_SOURCE_AMBIGUOUS
801+
if source_classification is PythonSourceClassification.AMBIGUOUS
802+
else None
803+
),
732804
emitted_finding_ids=[finding.finding_id for finding in path_findings],
733805
)
734806
ledger_events.append(event)

0 commit comments

Comments
 (0)