Repository navigation
Proposal: canonical dangerous-callable resolver + shared evasion corpus as a fitness gate #181
Description
Problem
The behavioral analyzers detect dangerous primitives by matching a set of name spellings, and each new way to spell the same primitive has needed its own branch: #114/#115 (import alias), #166 (getattr), and the builtins/importlib branch (#180). This enumerate-instead-of-canonicalize pattern leaks a fresh blind spot per missed spelling — e.g. subscript __builtins__["exec"] / os.__dict__["system"], and importlib.util / runpy / code machinery, which are currently undetected.
Proposal
One chokepoint resolve_to_canonical_sink(node, aliases, type_map) reducing any spelling — bare/alias/from/as, builtins.*, importlib.import_module, getattr("lit"), subscript __builtins__["lit"] / <mod>.__dict__["lit"], and dynamic-import / code-exec siblings — to one canonical sink id, reusing the existing alias/type machinery and subsuming the per-idiom branches. Paired with a shared, parametrized evasion corpus (tests/evasion_corpus/): several equivalent spellings per primitive plus FP-neighbors that must NOT match, wired as a fitness test so a future missed spelling fails a shared gate instead of shipping silently.
Scope / non-goals
In scope: statically-resolvable spellings. Explicit non-goals (def-use / taint territory, kept residual): non-literal / kwarg getattr(os, name), variable subscript keys, computed attribute strings, module-name-from-variable, and local indirection (f = os.system; f(x)). Receiver-gated instance methods (spec.loader.exec_module, code.InteractiveInterpreter().runsource) fire only when the receiver resolves to the real machinery.
Status
I have a working prototype (chokepoint + corpus + fitness test) that reproduces all current detections with no regression, adds the subscript / __dict__ / sibling spellings, and composes with #166. Looking for a maintainer yes/no on the direction before opening the PR.
Prepared with AI assistance; reviewed and validated by the author.
Activity
zied-jlassi commentedon Jun 23, 2026
I've pushed a complete working prototype as a branch — no PR yet, happy to open one if the direction is welcome:
Branch:
zied-jlassi/SkillSpector:feature/canonical-sink-evasion-corpusWhat it contains
resolve_to_canonical_sink— the single chokepoint described above, reusing the existing alias / type-map / dynamic-import machinery (no parallel resolution scheme).behavioral_astandbehavioral_taint_trackingwired to call it, so the per-idiom ladders stay simple. Reflectivesubprocessinvocations grade AST9-HIGH for parity with reflectiveos.system; a directsubprocess.*call keeps baseline AST4-MEDIUM; a small guard keeps AST9 and the canonical fallback complementary so they never double-fire.tests/evasion_corpus/— the shared parametrized corpus, wired as a fitness gate (every spelling → its canonical id and a real sink rule; benign neighbours → no sink).Validation (run on the real analyzers in an isolated container, not a bespoke harness)
test_static_patterns::TestRunStaticPatternsAgentSnooping::test_no_same_line_duplicatefailure is unrelated and present on the base branch too)ruff==0.15.2check +ruff format+mypyall cleanStacking / rebase note
The branch is currently stacked on #180 (the
builtins.*/importlib.import_modulesink-evasion fix) because the resolver reusesresolve_dynamic_import_callintroduced there. Once #180 lands I'll rebase this ontomainso the diff shows only the canonical layer. (If you'd rather land the canonical layer first and fold #180's helpers into it, that ordering works too — your call.)AI disclosure: prepared with AI assistance; every line was reviewed and validated by me, and the results above come from running the repo's real analyzers + test toolchain.
Wanted a yes/no on the approach before opening a PR.
zied-jlassi commentedon Jun 23, 2026
Update: I've gone ahead and opened the concrete PR for this — #182 — so the approach can be reviewed on real, tested code rather than just a branch link.
Same content as the prototype branch above (canonical resolver + analyzers wired + evasion-corpus fitness gate; 108/108 corpus, +108 suite / zero regressions, ruff + mypy clean). The PR is stacked on #180 and discloses that up front: its diff currently includes #180's commit until #180 lands, after which I'll rebase onto
main. Suggested merge order is #180 first, then #182.Still very happy to adjust the approach (or close) based on your feedback here — opening the PR just makes the diff tangible.
rng1995 commentedon Sep 16, 2026
Implementation is in progress in PR #182, adding a canonical dangerous-callable resolver and shared evasion corpus. Keeping this proposal open until the PR merges and the requested fitness-gate behavior is verified.
rng1995 commentedon Oct 4, 2026
PR-state snapshot checked on 2026-10-04:
Keeping this issue open: the relevant implementation is not merged. Check/review status is a point-in-time snapshot, not a claim of merge readiness.
feat(behavioral): canonical dangerous-callable resolver + shared evas…