Skip to content

Commit 5ee48ef

Browse files
chrisknvidiarng1995claude
authored
fix(analyzer): detect bound shell truthiness (#577)
* test(analyzer): define shell truthiness core contract Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): detect bound shell truthiness Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): harden bound shell facts Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): honor shell argument evaluation order Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): invalidate effectful subprocess receivers Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): invalidate receiver trust after generic calls Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): reconcile bound shell ownership Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): reconcile true-prefixed shell ownership Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): invalidate nested RHS receiver effects Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): invalidate unsupported eager receiver effects Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): track eager receiver state Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve lexical TM1 when dataflow abstains Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): require a retained companion owner Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve execution scope in binding evidence Signed-off-by: Christopher Kevin <[email protected]> * test(analyzer): distinguish deferred abstention from lexical signal Signed-off-by: Christopher Kevin <[email protected]> * style(analyzer): simplify binding event deduplication Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve runtime and retained shell evidence Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): preserve called-slot identity and cached method evidence Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve cached subprocess slot state across scoped imports Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): reconcile explicit cached-slot evidence across scopes Signed-off-by: Christopher Kevin <[email protected]> * fix: abstain from cached slot proof after eager effects Signed-off-by: Christopher Kevin <[email protected]> * test: retain lexical findings after unknown cached-slot effects Signed-off-by: Christopher Kevin <[email protected]> * fix: bound direct called-slot evidence to known execution order Signed-off-by: Christopher Kevin <[email protected]> * fix: limit direct slot proof to single assignment targets Signed-off-by: Christopher Kevin <[email protected]> * fix: invalidate slot evidence on protocols and unsafe releases Signed-off-by: Christopher Kevin <[email protected]> * fix: keep legacy shadow proof isolated from later effects Signed-off-by: Christopher Kevin <[email protected]> * fix: abstain from cached replacement proof after unknown effects Signed-off-by: Christopher Kevin <[email protected]> * fix: retain native callable detection across cached slot stores Signed-off-by: Christopher Kevin <[email protected]> * fix: retain shell findings for native receiver assignments Signed-off-by: Christopher Kevin <[email protected]> * fix: preserve native Popen replacement warnings Signed-off-by: Christopher Kevin <[email protected]> * fix(analyzer): require proven replacement before revoking lexical TM1 Address the three remaining review findings on lexical TM1 reconciliation, where a bound shell=True call still ran natively but lost its HIGH finding: - Untaken expression stores: a walrus in a short-circuited BoolOp operand, an untaken conditional-expression arm, or a comprehension/generator body is no longer recorded as replacement evidence. Only constant operands and tests prove that the store executed. - Identity-preserving stores: unpacked targets are paired with their RHS elements. Self-stores are transparent, truthy constants keep the flag, and native receiver aliases (saved = subprocess, import subprocess as saved, Popen = subprocess.Popen) keep the native receiver. - Deferred bodies: an outer store only proves replacement when every invocation the owner scope can reach sees it. Any reference to the function, method or class (compound statements, main guards, callbacks, other bodies) can invoke it from that point on, and decorators or lambdas can run once defined. This also covers a re-import before a later invocation. The release-trigger test for a called function now matches its module-level twin: the companion abstains and the lexical HIGH remains. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: account for TM1 Python parsing in marker ownership ledger Main's #784 test expects the tool-misuse ledger event for an invalid Python file to carry obfuscated_instruction_text. This PR makes tool misuse parse Python for TM1 reconciliation, so the runner's higher-precedence syntax_error now names that partial event. Keep the fail-closed contract: assert the marker reading through the lexical-only anti-refusal analyzer and keep tool misuse partial with syntax_error. Signed-off-by: Narendran Raghavan <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Signed-off-by: Christopher Kevin <[email protected]> Signed-off-by: Christopher Kevin <[email protected]> Signed-off-by: Narendran Raghavan <[email protected]> Co-authored-by: Christopher Kevin <[email protected]> Co-authored-by: Narendran Raghavan <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
1 parent 8777636 commit 5ee48ef

9 files changed

Lines changed: 5565 additions & 172 deletions

‎src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py‎

Lines changed: 1127 additions & 170 deletions
Large diffs are not rendered by default.

‎src/skillspector/nodes/analyzers/static_python_shell_truthiness.py‎

Lines changed: 1967 additions & 0 deletions
Large diffs are not rendered by default.

‎src/skillspector/nodes/analyzers/static_runner.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -916,6 +916,8 @@ def _scan_path(
916916
analyze_kwargs["python_ast"] = python_ast
917917
if _uses_runtime_check(module):
918918
analyze_kwargs["check_runtime"] = finding_budget.check_runtime
919+
if _explicit_module_hook(module, "ANALYZE_USES_POSTPROCESS") is True:
920+
analyze_kwargs["defer_variable_reconciliation"] = True
919921
raw = module.analyze(**analyze_kwargs)
920922
finding_budget.check_runtime()
921923
for af in raw:
Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""Cached module slots retain their state across imports in deferred scopes."""
5+
6+
from __future__ import annotations
7+
8+
import pytest
9+
10+
from skillspector.nodes.analyzers import static_python_shell_truthiness as truthiness
11+
from skillspector.python_ast import parse_python_source
12+
13+
14+
@pytest.mark.parametrize(
15+
"source",
16+
[
17+
"import subprocess\nsubprocess.run = proxy\n"
18+
"def run():\n import subprocess\n enabled = True\n"
19+
" subprocess.run(command, shell=enabled)\nrun()\n",
20+
"import subprocess\nsubprocess.Popen = proxy\n"
21+
"def run():\n from subprocess import Popen\n enabled = True\n"
22+
" Popen(command, shell=enabled)\nrun()\n",
23+
"import subprocess\ndef outer():\n subprocess.run = proxy\n"
24+
" def inner():\n import subprocess\n enabled = True\n"
25+
" subprocess.run(command, shell=enabled)\n inner()\nouter()\n",
26+
"import subprocess\nsubprocess.run = proxy\nclass Tool:\n"
27+
" def run(self):\n import subprocess\n enabled = True\n"
28+
" subprocess.run(command, shell=enabled)\n",
29+
"import subprocess\nclass Tool:\n def alter(self):\n"
30+
" global subprocess\n subprocess = proxy\n"
31+
"subprocess.run = handler\ndef run():\n import subprocess\n enabled = 1\n"
32+
" subprocess.run(command, shell=enabled)\n",
33+
],
34+
)
35+
def test_local_import_does_not_restore_changed_cached_module_slot(source: str) -> None:
36+
assert truthiness.analyze(source, "run.py", "python") == []
37+
38+
39+
@pytest.mark.parametrize(
40+
"source",
41+
[
42+
"import subprocess\ndef run():\n import subprocess\n enabled = True\n"
43+
" subprocess.run(command, shell=enabled)\nrun()\nsubprocess.run = proxy\n",
44+
"import subprocess\ndef alter():\n subprocess.run = proxy\n"
45+
"def run():\n import subprocess\n enabled = True\n"
46+
" subprocess.run(command, shell=enabled)\nrun()\n",
47+
"subprocess = proxy\nsubprocess.run = proxy.run\ndef run():\n"
48+
" import subprocess\n enabled = True\n"
49+
" subprocess.run(command, shell=enabled)\nrun()\n",
50+
"import subprocess\nsubprocess.Popen = proxy\ndef run():\n"
51+
" import subprocess\n enabled = True\n"
52+
" subprocess.run(command, shell=enabled)\nrun()\n",
53+
"import subprocess\ndef outer(subprocess):\n subprocess.run = proxy\n"
54+
" def inner():\n import subprocess\n enabled = 1\n"
55+
" subprocess.run(command, shell=enabled)\n inner()\n",
56+
"import subprocess\nclass Tool:\n global subprocess\n subprocess = proxy\n"
57+
"subprocess.run = handler\ndef run():\n import subprocess\n enabled = 1\n"
58+
" subprocess.run(command, shell=enabled)\n",
59+
"import subprocess\nclass Outer:\n class Inner:\n"
60+
" global subprocess\n subprocess = proxy\n"
61+
"subprocess.run = handler\ndef run():\n import subprocess\n enabled = 1\n"
62+
" subprocess.run(command, shell=enabled)\n",
63+
"import subprocess\nsubprocess.run = (subprocess := proxy)\nimport subprocess\n"
64+
"def run():\n import subprocess\n enabled = 1\n"
65+
" subprocess.run(command, shell=enabled)\nrun()\n",
66+
"import subprocess\ndef replace_receiver():\n global subprocess\n"
67+
" subprocess = proxy\n return handler\n"
68+
"subprocess.run = replace_receiver()\nimport subprocess\ndef run():\n"
69+
" import subprocess\n enabled = 1\n"
70+
" subprocess.run(command, shell=enabled)\nrun()\n",
71+
],
72+
)
73+
def test_cached_module_tracking_preserves_observed_and_unaffected_calls(source: str) -> None:
74+
findings = truthiness.analyze(source, "run.py", "python")
75+
assert len(findings) == 1
76+
assert findings[0].severity == "HIGH"
77+
78+
79+
@pytest.mark.parametrize(
80+
("source", "replacement"),
81+
[
82+
(
83+
"import subprocess\nsubprocess.run = proxy\nenabled = True\n"
84+
"subprocess.run(command, shell=enabled)\n",
85+
True,
86+
),
87+
(
88+
"import subprocess\nsubprocess.Popen = proxy\nfrom subprocess import Popen\n"
89+
"enabled = True\nPopen(command, shell=enabled)\n",
90+
True,
91+
),
92+
(
93+
"from subprocess import Popen\nimport subprocess\nsubprocess.Popen = proxy\n"
94+
"enabled = True\nPopen(command, shell=enabled)\n",
95+
False,
96+
),
97+
(
98+
"import subprocess\nhelper()\nenabled = True\nsubprocess.run(command, shell=enabled)\n",
99+
False,
100+
),
101+
(
102+
"import subprocess\ndef run(subprocess, command):\n"
103+
" subprocess.run = proxy\n import subprocess\n enabled = 1\n"
104+
" subprocess.run(command, shell=enabled)\n",
105+
False,
106+
),
107+
(
108+
"import subprocess\nsubprocess.run = proxy\ntrue_flag = True\n"
109+
"subprocess.run(command, shell=true_flag)\n",
110+
True,
111+
),
112+
(
113+
"import subprocess\ndef run():\n import subprocess\n enabled = True\n"
114+
" subprocess.run(command, shell=enabled)\nrun()\n"
115+
"subprocess.run = proxy\nrun()\n",
116+
False,
117+
),
118+
],
119+
)
120+
def test_cached_replacement_is_separate_from_unknown_receiver_trust(
121+
source: str, replacement: bool
122+
) -> None:
123+
parsed = parse_python_source(source, "run.py")
124+
ownership, emitted, replacements = truthiness.bound_shell_call_analysis("run.py", parsed)
125+
assert len(ownership) == 1
126+
assert bool(replacements) is replacement
127+
assert replacements.issubset(ownership)
128+
assert truthiness.bound_shell_call_state("run.py", parsed) == (ownership, emitted)
129+
130+
131+
def test_cached_replacement_analysis_keeps_invalid_python_empty() -> None:
132+
parsed = parse_python_source("def incomplete(", "run.py")
133+
assert truthiness.bound_shell_call_analysis("run.py", parsed) == ({}, set(), set())
134+
135+
136+
@pytest.mark.parametrize(
137+
"prefix",
138+
[
139+
"box[redirect()] = subprocess.run = handler\n",
140+
"def unused(default=(subprocess := proxy)):\n pass\nsubprocess.run = handler\n",
141+
"class Tool((subprocess := proxy)):\n pass\nsubprocess.run = handler\n",
142+
"original = subprocess.run\ndef restore():\n subprocess.run = original\n"
143+
"subprocess.run = handler\nrestore()\n",
144+
"original = subprocess.run\ndef restore():\n subprocess.run = original\n"
145+
"subprocess.run = handler\ndef unused(default=restore()):\n pass\n",
146+
"subprocess.run = proxy\nsubprocess.__class__ = Restoring\n",
147+
"subprocess.__class__ = Restoring\nimport subprocess\nsubprocess.run = proxy\n",
148+
"restorer = Untrusted()\nimport subprocess\nsubprocess.run = proxy\ndel restorer\n",
149+
"subprocess.run = proxy\ndel subprocess.run\n",
150+
"subprocess.run = proxy\nsubprocess.run = handler\n",
151+
"subprocess.run = proxy\nimport restore_module\n",
152+
"subprocess.run = proxy\nfrom restore_module import restored\n",
153+
"subprocess.run = proxy\nimport subprocess, restore_module\n",
154+
"restorer = Untrusted()\nimport subprocess\nsubprocess.run = proxy\nrestorer = 0\n",
155+
],
156+
)
157+
def test_unknown_eager_effect_invalidates_prior_replacement_proof(prefix: str) -> None:
158+
source = (
159+
"import subprocess\n" + prefix + "import subprocess\ndef work():\n"
160+
" import subprocess\n enabled = 1\n"
161+
" subprocess.run(command, shell=enabled)\nwork()\n"
162+
)
163+
assert len(truthiness.analyze(source, "run.py", "python")) == 1
164+
_, _, replacements = truthiness.bound_shell_call_analysis(
165+
"run.py", parse_python_source(source, "run.py")
166+
)
167+
assert not replacements
168+
169+
170+
@pytest.mark.parametrize(
171+
("expression", "replacement"),
172+
[
173+
("return Popen(command, shell=enabled)", True),
174+
("wrapper(Popen(command, shell=enabled))", True),
175+
("return [Popen(command, shell=enabled)]", True),
176+
("return wrapper(helper(), Popen(command, shell=enabled))", False),
177+
("return lambda: Popen(command, shell=enabled)", False),
178+
("return (Popen(command, shell=enabled) for item in items)", False),
179+
],
180+
)
181+
def test_cached_replacement_proof_respects_eager_expression_order(
182+
expression: str, replacement: bool
183+
) -> None:
184+
source = (
185+
"import subprocess\nsubprocess.Popen = proxy\ndef run():\n"
186+
" from subprocess import Popen\n enabled = True\n"
187+
f" {expression}\n"
188+
)
189+
_, emitted, replacements = truthiness.bound_shell_call_analysis(
190+
"run.py", parse_python_source(source, "run.py")
191+
)
192+
assert bool(replacements) is replacement
193+
assert not emitted
194+
195+
196+
@pytest.mark.parametrize(
197+
"source",
198+
[
199+
"restorer = 0\nimport subprocess\nclass Tool:\n"
200+
" helper()\n import subprocess\n subprocess.run = proxy\n"
201+
"restorer = 0\nenabled = True\nsubprocess.run(command, shell=enabled)\n",
202+
"helper()\nimport subprocess\nsubprocess.run = proxy\nrestorer = 0\n"
203+
"enabled = True\nsubprocess.run(command, shell=enabled)\n",
204+
"import subprocess\nenabled = True\nhelper()\nimport subprocess\n"
205+
"subprocess.run = proxy\nsubprocess.run(command, shell=enabled)\n",
206+
"import subprocess\nsubprocess.run = proxy\ndef work():\n"
207+
" import subprocess\n enabled = 1\n"
208+
" subprocess.run(command, shell=enabled)\nwork()\nhelper()\n"
209+
"import subprocess\nsubprocess.run = proxy\nwork()\n",
210+
],
211+
ids=["class-outer-finalizer", "unknown-new-binding", "unknown-protocol", "later-observation"],
212+
)
213+
def test_unknown_effect_cannot_reestablish_cached_replacement_proof(source: str) -> None:
214+
# Reimporting the same cached module does not establish that an unknown
215+
# earlier effect left its lookup protocol or other finalizer bindings intact.
216+
ownership, _, replacements = truthiness.bound_shell_call_analysis(
217+
"run.py", parse_python_source(source, "run.py")
218+
)
219+
assert len(ownership) == 1
220+
assert not replacements
221+
222+
223+
@pytest.mark.parametrize(
224+
("prefix", "method"),
225+
[
226+
("original = subprocess.run\nsubprocess.run = original\n", "run"),
227+
("subprocess.run = subprocess.check_call\n", "run"),
228+
(
229+
"native_module = subprocess\noriginal = native_module.run\nsubprocess.run = original\n",
230+
"run",
231+
),
232+
(
233+
"import subprocess as native_module\nsubprocess.run = native_module.check_output\n",
234+
"run",
235+
),
236+
("original, = (subprocess.run,)\nsubprocess.run = original\n", "run"),
237+
("[original] = [subprocess.run]\nsubprocess.run = original\n", "run"),
238+
("from subprocess import Popen\nsubprocess.run = Popen\n", "run"),
239+
("from subprocess import Popen\noriginal = Popen\nsubprocess.run = original\n", "run"),
240+
("subprocess.Popen = subprocess.run\n", "Popen"),
241+
],
242+
)
243+
def test_native_callable_value_does_not_prove_a_custom_replacement(
244+
prefix: str, method: str
245+
) -> None:
246+
source = (
247+
"import subprocess\n" + prefix + "def work():\n"
248+
" import subprocess\n enabled = True\n"
249+
f" subprocess.{method}(command, shell=enabled)\nwork()\n"
250+
)
251+
assert len(truthiness.analyze(source, "run.py", "python")) == 1
252+
_, _, replacements = truthiness.bound_shell_call_analysis(
253+
"run.py", parse_python_source(source, "run.py")
254+
)
255+
assert not replacements

‎tests/nodes/analyzers/test_python_string_marker_ownership.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -432,6 +432,15 @@ def test_marker_declared_inside_a_python_token_still_fails_closed(content: str,
432432
],
433433
)
434434
def test_glued_marker_outside_proven_python_still_fails_closed(path: str, content: str) -> None:
435+
if path.endswith(".py"):
436+
# Tool misuse parses Python for TM1 reconciliation, so the
437+
# higher-precedence syntax error names its partial event. A lexical-only
438+
# analyzer still reports the unproven marker reading.
439+
_assert_partial_marker_text(_ledger(path, content, ar_module))
440+
event = _ledger(path, content)["inspection_ledger"][0]
441+
assert event["outcome"] is LedgerOutcome.PARTIAL
442+
assert event["reason_code"] is LedgerReason.SYNTAX_ERROR
443+
return
435444
_assert_partial_marker_text(_ledger(path, content))
436445

437446

‎tests/nodes/analyzers/test_shared_python_ast.py‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
behavioral_taint_tracking,
1717
static_patterns_data_exfiltration,
1818
static_patterns_output_handling,
19+
static_patterns_tool_misuse,
1920
static_runner,
2021
)
2122
from skillspector.nodes.build_context import build_context
@@ -59,11 +60,12 @@ def code(tail: str) -> str:
5960
def test_preparsed_python_is_reused_by_all_ast_analyzers(tmp_path, monkeypatch) -> None:
6061
"""One scan parses each eligible Python file once before analyzer fan-out."""
6162
(tmp_path / "script.py").write_text(
62-
"import os\n"
6363
"import subprocess\n"
64+
"enabled = True\n"
65+
"subprocess.run(output, shell=enabled)\n"
66+
"import os\n"
6467
"payload = input()\n"
6568
"environment = os.environ.copy()\n"
66-
"subprocess.run(output)\n"
6769
"exec(payload)\n",
6870
encoding="utf-8",
6971
)
@@ -91,11 +93,13 @@ def count_parse(*args, **kwargs):
9193

9294
data_findings = static_patterns_data_exfiltration.node(state)["findings"]
9395
output_findings = static_patterns_output_handling.node(state)["findings"]
96+
tool_misuse_findings = static_patterns_tool_misuse.node(state)["findings"]
9497
ast_findings = behavioral_ast.node(state)["findings"]
9598
taint_findings = behavioral_taint_tracking.node(state)["findings"]
9699

97100
assert any(finding.rule_id == "E2" for finding in data_findings)
98101
assert any(finding.rule_id == "OH1" for finding in output_findings)
102+
assert any(finding.rule_id == "TM1" for finding in tool_misuse_findings)
99103
assert any(finding.rule_id == "AST1" for finding in ast_findings)
100104
assert any(finding.rule_id == "TT5" for finding in taint_findings)
101105
assert parse_calls == 1

0 commit comments

Comments
 (0)