Repository navigation
Commit 2c14261
feat(python): classify executable source surfaces (#579)
* fix(analyzer): stabilize TM1 window identity
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): track eager receiver side effects
Signed-off-by: Christopher Kevin <[email protected]>
* feat(python): refresh execution surfaces on current main
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): preserve lexical TM1 when dataflow abstains
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): require a retained companion owner
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): preserve execution scope in binding evidence
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): preserve lexical TM1 across conservative Python analysis
Signed-off-by: Christopher Kevin <[email protected]>
* style(analyzer): simplify binding event deduplication
Signed-off-by: Christopher Kevin <[email protected]>
* fix: preserve runtime and retained shell evidence
Signed-off-by: Christopher Kevin <[email protected]>
* fix: preserve runtime and retained shell evidence
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): preserve called-slot identity and cached method evidence
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): preserve called-slot identity and cached method evidence
Signed-off-by: Christopher Kevin <[email protected]>
* fix: preserve cached subprocess slot state across scoped imports
Signed-off-by: Christopher Kevin <[email protected]>
* fix: preserve cached subprocess slot state across scoped imports
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): reconcile explicit cached-slot evidence across scopes
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): reconcile explicit cached-slot evidence across scopes
Signed-off-by: Christopher Kevin <[email protected]>
* fix: abstain from cached slot proof after eager effects
Signed-off-by: Christopher Kevin <[email protected]>
* fix: abstain from cached slot proof after eager effects
Signed-off-by: Christopher Kevin <[email protected]>
* test: retain lexical findings after unknown cached-slot effects
Signed-off-by: Christopher Kevin <[email protected]>
* test: retain lexical findings after unknown cached-slot effects
Signed-off-by: Christopher Kevin <[email protected]>
* fix: bound direct called-slot evidence to known execution order
Signed-off-by: Christopher Kevin <[email protected]>
* fix: bound direct called-slot evidence to known execution order
Signed-off-by: Christopher Kevin <[email protected]>
* fix: limit direct slot proof to single assignment targets
Signed-off-by: Christopher Kevin <[email protected]>
* fix: limit direct slot proof to single assignment targets
Signed-off-by: Christopher Kevin <[email protected]>
* fix: invalidate slot evidence on protocols and unsafe releases
Signed-off-by: Christopher Kevin <[email protected]>
* fix: invalidate slot evidence on protocols and unsafe releases
Signed-off-by: Christopher Kevin <[email protected]>
* fix: keep legacy shadow proof isolated from later effects
Signed-off-by: Christopher Kevin <[email protected]>
* fix: keep legacy shadow proof isolated from later effects
Signed-off-by: Christopher Kevin <[email protected]>
* fix: abstain from cached replacement proof after unknown effects
Signed-off-by: Christopher Kevin <[email protected]>
* fix: abstain from cached replacement proof after unknown effects
Signed-off-by: Christopher Kevin <[email protected]>
* fix: retain native callable detection across cached slot stores
Signed-off-by: Christopher Kevin <[email protected]>
* fix: retain native callable detection across cached slot stores
Signed-off-by: Christopher Kevin <[email protected]>
* fix: retain shell findings for native receiver assignments
Signed-off-by: Christopher Kevin <[email protected]>
* fix: retain shell findings for native receiver assignments
Signed-off-by: Christopher Kevin <[email protected]>
* fix: preserve native Popen replacement warnings
Signed-off-by: Christopher Kevin <[email protected]>
* fix: preserve native Popen replacement warnings
Signed-off-by: Christopher Kevin <[email protected]>
* test: retain occurrence metadata in prose shell reports
Signed-off-by: Christopher Kevin <[email protected]>
* fix(analyzer): require proven replacement before revoking lexical TM1
Address the three remaining review findings on lexical TM1 reconciliation,
where a bound shell=True call still ran natively but lost its HIGH finding:
- Untaken expression stores: a walrus in a short-circuited BoolOp operand,
an untaken conditional-expression arm, or a comprehension/generator body
is no longer recorded as replacement evidence. Only constant operands
and tests prove that the store executed.
- Identity-preserving stores: unpacked targets are paired with their RHS
elements. Self-stores are transparent, truthy constants keep the flag,
and native receiver aliases (saved = subprocess, import subprocess as
saved, Popen = subprocess.Popen) keep the native receiver.
- Deferred bodies: an outer store only proves replacement when every
invocation the owner scope can reach sees it. Any reference to the
function, method or class (compound statements, main guards, callbacks,
other bodies) can invoke it from that point on, and decorators or
lambdas can run once defined. This also covers a re-import before a
later invocation.
The release-trigger test for a called function now matches its
module-level twin: the companion abstains and the lexical HIGH remains.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(analyzer): require proven replacement before revoking lexical TM1
Address the three remaining review findings on lexical TM1 reconciliation,
where a bound shell=True call still ran natively but lost its HIGH finding:
- Untaken expression stores: a walrus in a short-circuited BoolOp operand,
an untaken conditional-expression arm, or a comprehension/generator body
is no longer recorded as replacement evidence. Only constant operands
and tests prove that the store executed.
- Identity-preserving stores: unpacked targets are paired with their RHS
elements. Self-stores are transparent, truthy constants keep the flag,
and native receiver aliases (saved = subprocess, import subprocess as
saved, Popen = subprocess.Popen) keep the native receiver.
- Deferred bodies: an outer store only proves replacement when every
invocation the owner scope can reach sees it. Any reference to the
function, method or class (compound statements, main guards, callbacks,
other bodies) can invoke it from that point on, and decorators or
lambdas can run once defined. This also covers a re-import before a
later invocation.
The release-trigger test for a called function now matches its
module-level twin: the companion abstains and the lexical HIGH remains.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* test: account for TM1 Python parsing in marker ownership ledger
Main's #784 test expects the tool-misuse ledger event for an invalid
Python file to carry obfuscated_instruction_text. This PR makes tool
misuse parse Python for TM1 reconciliation, so the runner's
higher-precedence syntax_error now names that partial event. Keep the
fail-closed contract: assert the marker reading through the lexical-only
anti-refusal analyzer and keep tool misuse partial with syntax_error.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* test: account for TM1 Python parsing in marker ownership ledger
Main's #784 test expects the tool-misuse ledger event for an invalid
Python file to carry obfuscated_instruction_text. This PR makes tool
misuse parse Python for TM1 reconciliation, so the runner's
higher-precedence syntax_error now names that partial event. Keep the
fail-closed contract: assert the marker reading through the lexical-only
anti-refusal analyzer and keep tool misuse partial with syntax_error.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(analyzer): keep one TM1 owner for true-prefixed names without AST proof
Port the six true-prefixed TM1 regression tests from #577. This branch
already carries test_true_direct_calls_on_one_line_keep_distinct_locations
unchanged, so only the other five are added. Two of them reported the
same call twice on this branch:
- A variable window for a name spelled `true` (in any case) duplicated
the case-insensitive direct `shell=True` owner at the same call. AST
reconciliation removes that window only when the dataflow companion
takes the call, so a file that does not parse, or a call after an
unknown receiver effect, kept both findings. coalesce_path_findings
now drops such a window when a direct lexical owner starts at the
same call. A window for a call whose direct candidate was folded into
an identical same-line call still reports that call.
- A call-anchored window for a longer true-prefixed name (`true_value`)
used its raw text as identity, so the raw and normalized security
views of one call did not collapse, and reconciliation then moved both
to the call. The window now takes its identity from the normalized
view text, as the direct owner already does.
Also cover both root causes with a test for a nested `true` call after
an unknown receiver effect and a normalized `true_value` call in a file
that does not parse.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(analyzer): keep one TM1 owner for true-prefixed names without AST proof
Port the six true-prefixed TM1 regression tests from #577. This branch
already carries test_true_direct_calls_on_one_line_keep_distinct_locations
as test_true_direct_calls_around_safe_comprehension_keep_distinct_locations,
so only the other five are added. Two of them reported the same call twice
on this branch:
- A variable window for a name spelled `true` (in any case) duplicated
the case-insensitive direct `shell=True` owner at the same call. AST
reconciliation removes that window only when the dataflow companion
takes the call, so a file that does not parse, or a call after an
unknown receiver effect, kept both findings. coalesce_path_findings
now drops such a window when a direct lexical owner starts at the
same call. A window for a call whose direct candidate was folded into
an identical same-line call still reports that call.
- A call-anchored window for a longer true-prefixed name (`true_value`)
used its raw text as identity, so the raw and normalized security
views of one call did not collapse, and reconciliation then moved both
to the call. The window now takes its identity from the normalized
view text, as the direct owner already does.
Also cover both root causes with a test for a nested `true` call after
an unknown receiver effect and a normalized `true_value` call in a file
that does not parse.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(analyzer): report each true-prefixed shell call that #577 reports
The `\b`-bounded direct pattern does not match `shell=true_value`, so on
this branch such a call was reported only through a variable window. That
window pairs an assignment with a single call, skips an assignment
inside an earlier window, and was dropped outside Python. Compared with
#577 this lost:
- calls in JavaScript and Markdown files, fenced or plain;
- a later call that the AST companion abstains on, for example after an
unknown receiver effect;
- all but the last same-line call in a file that does not parse, and the
second of two identical same-line `shell=true` calls outside Python,
which the direct candidate key folds into one;
- a call whose assignment sits inside the window of an earlier one.
_variable_shell_matches now pairs every assignment of a true-prefixed
name with each call its bounded window reaches, and the nearest
assignment owns each call. Candidate generation, analysis and AST
reconciliation share that list, and a window's candidate identity uses
its full text, so windows to different calls do not collapse on a shared
200-character preview. Outside Python, call-anchored windows for
true-prefixed names are kept, and coalescing still drops one when a
direct owner reports the same call. With an AST, each added window
passes the same visibility, counterevidence, cached replacement and
companion checks as before. Windows for other names are unchanged.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(analyzer): report each true-prefixed shell call that #577 reports
The `\b`-bounded direct pattern does not match `shell=true_value`, so on
this branch such a call was reported only through a variable window. That
window pairs an assignment with a single call, skips an assignment
inside an earlier window, and was dropped outside Python. Compared with
#577 this lost:
- calls in JavaScript and Markdown files, fenced or plain;
- a later call that the AST companion abstains on, for example after an
unknown receiver effect;
- all but the last same-line call in a file that does not parse, and the
second of two identical same-line `shell=true` calls outside Python,
which the direct candidate key folds into one;
- a call whose assignment sits inside the window of an earlier one.
_variable_shell_matches now pairs every assignment of a true-prefixed
name with each call its bounded window reaches, and the nearest
assignment owns each call. Candidate generation, analysis and AST
reconciliation share that list, and a window's candidate identity uses
its full text, so windows to different calls do not collapse on a shared
200-character preview. Outside Python, call-anchored windows for
true-prefixed names are kept, and coalescing still drops one when a
direct owner reports the same call. With an AST, each added window
passes the same visibility, counterevidence, cached replacement and
companion checks as before. Windows for other names are unchanged.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(supply-chain): opt into Python source-type propagation
SC2's literal-XOR decoding and SC3 branch on file_type, but the
supply-chain analyzer was not in the USES_PYTHON_SOURCE_TYPE set, so
Python executed through an extensionless shebang file or a shebang
Markdown file received its suffix type and silently lost the decoded
SC2 HIGH while completeness stayed true. Opt the module in and add
graph-level parity tests for .py, .pyw, extensionless and .md surfaces.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(build-context): decode PEP 263 primary Python before UTF-8 rejection
A selected primary file (direct .py/.pyw or extensionless Python
shebang) declaring a supported PEP 263 encoding such as latin-1 was
rejected as unsupported_primary_content because the generic primary
check only accepts UTF-8. The later source-decoding pass then decoded
it correctly, but the artifact stayed FAILED with a fatal event and 0%
coverage. Waive the generic UTF-8 rejection when the bytes classify as
Python and decode under their declared encoding; an unknown codec or
bytes invalid for the declared codec remain fatal, and format checks
(BOMs, archives, NUL density) are unchanged. The waiver is evaluated
lazily and not after the shared deadline, so no post-cache Python work
starts once the budget is spent.
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Signed-off-by: Christopher Kevin <[email protected]>
Signed-off-by: Christopher Kevin <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>
Co-authored-by: Christopher Kevin <[email protected]>
Co-authored-by: Narendran Raghavan <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent 320031c commit 2c14261
32 files changed
Lines changed: 6432 additions & 626 deletions
File tree
- src/skillspector
- nodes
- analyzers
- tests
- nodes
- analyzers
- unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
230 | 230 | | |
231 | 231 | | |
232 | 232 | | |
| 233 | + | |
233 | 234 | | |
234 | 235 | | |
235 | 236 | | |
| |||
403 | 404 | | |
404 | 405 | | |
405 | 406 | | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
406 | 421 | | |
407 | 422 | | |
408 | 423 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
1314 | 1314 | | |
1315 | 1315 | | |
1316 | 1316 | | |
| 1317 | + | |
| 1318 | + | |
| 1319 | + | |
| 1320 | + | |
| 1321 | + | |
| 1322 | + | |
1317 | 1323 | | |
1318 | 1324 | | |
1319 | 1325 | | |
1320 | 1326 | | |
1321 | 1327 | | |
1322 | 1328 | | |
| 1329 | + | |
| 1330 | + | |
| 1331 | + | |
| 1332 | + | |
| 1333 | + | |
| 1334 | + | |
| 1335 | + | |
1323 | 1336 | | |
1324 | 1337 | | |
1325 | 1338 | | |
1326 | 1339 | | |
1327 | 1340 | | |
1328 | 1341 | | |
| 1342 | + | |
1329 | 1343 | | |
1330 | 1344 | | |
1331 | 1345 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| 85 | + | |
85 | 86 | | |
86 | 87 | | |
87 | 88 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
| 111 | + | |
| 112 | + | |
111 | 113 | | |
112 | 114 | | |
113 | 115 | | |
| |||
221 | 223 | | |
222 | 224 | | |
223 | 225 | | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
224 | 232 | | |
225 | 233 | | |
226 | 234 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
123 | 128 | | |
124 | 129 | | |
125 | 130 | | |
| |||
218 | 223 | | |
219 | 224 | | |
220 | 225 | | |
221 | | - | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
222 | 229 | | |
223 | 230 | | |
224 | 231 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| 44 | + | |
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| |||
80 | 81 | | |
81 | 82 | | |
82 | 83 | | |
| 84 | + | |
83 | 85 | | |
84 | 86 | | |
85 | 87 | | |
| |||
114 | 116 | | |
115 | 117 | | |
116 | 118 | | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
117 | 124 | | |
118 | 125 | | |
119 | 126 | | |
| |||
539 | 546 | | |
540 | 547 | | |
541 | 548 | | |
542 | | - | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
543 | 555 | | |
544 | 556 | | |
| 557 | + | |
| 558 | + | |
545 | 559 | | |
546 | 560 | | |
547 | 561 | | |
548 | 562 | | |
549 | 563 | | |
| 564 | + | |
550 | 565 | | |
551 | 566 | | |
552 | 567 | | |
| |||
1098 | 1113 | | |
1099 | 1114 | | |
1100 | 1115 | | |
1101 | | - | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
1102 | 1123 | | |
1103 | 1124 | | |
1104 | 1125 | | |
| 1126 | + | |
1105 | 1127 | | |
1106 | 1128 | | |
1107 | 1129 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
36 | 41 | | |
37 | 42 | | |
38 | 43 | | |
| |||
648 | 653 | | |
649 | 654 | | |
650 | 655 | | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
651 | 664 | | |
652 | 665 | | |
653 | 666 | | |
654 | 667 | | |
655 | 668 | | |
656 | 669 | | |
657 | 670 | | |
658 | | - | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
659 | 692 | | |
660 | 693 | | |
661 | 694 | | |
| |||
665 | 698 | | |
666 | 699 | | |
667 | 700 | | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
668 | 706 | | |
669 | 707 | | |
670 | 708 | | |
671 | 709 | | |
672 | | - | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
673 | 736 | | |
674 | 737 | | |
675 | 738 | | |
| |||
725 | 788 | | |
726 | 789 | | |
727 | 790 | | |
728 | | - | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
729 | 796 | | |
730 | 797 | | |
731 | 798 | | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
732 | 804 | | |
733 | 805 | | |
734 | 806 | | |
| |||
0 commit comments