Repository navigation
Commit 5d370aa
authored
fix(patterns): detect constructed sensitive paths (#545)
* fix(patterns): detect constructed sensitive paths
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): preserve shared Python AST cache
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): resolve constructed joins via AST with per-line PE3 dedup
The PE3 constructed-path recognizer missed multiline and imported/aliased
os.path.join spellings, and emitted a second PE3 for lines the pattern loop
already covered. Resolve literal join calls from the Python AST (handling
from/import aliases), and keep the best-confidence PE3 per line.
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): gate constructed-path AST parse behind join pre-check
The AST-based os.path.join resolver parsed every Python file, defeating
the runner's parse-once invariant (one shared parse per file across the
analyzer fan-out). Gate the local parse behind a cheap textual pre-check
so files without a plausible join() call never pay for it. The module
stays lexical, preserving windowed and normalized-view scans for all
file types.
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): alias-aware join gate, shared AST reuse, span dedup
Address rng1995's review on constructed sensitive-path detection:
- Broaden the join pre-check to cover imported local names: the gate now
matches every name the file binds to os.path or os.path.join, so
'from os.path import join as j' followed by j('/etc', 'passwd') reaches
alias resolution instead of returning early on the missing literal join(.
- Route the supplemental analysis through the scan's shared AST: node()
publishes the runner's python_ast_cache_key via a contextvar and
_constructed_sensitive_paths consults it with the new peek_python_ast
helper (exact-content hit only, never parses or writes). The module stays
lexical so windowed and normalized views keep running; windowed view
fragments are skipped and standalone callers keep one on-demand parse.
- Deduplicate across the call's whole source span: hits now carry
(start_line, end_line), so a join call wrapped over several lines no
longer produces a second PE3 when the lexical loop fires on a wrapped
argument line.
Tests: extend test_pe3_aliased_join_imports_are_detected with the renamed
import case, add test_pe3_multiline_constructed_join_is_deduplicated, and
extend test_graph_scan_parses_python_once_before_parallel_analyzers with a
literal os.path.join call to verify the graph still performs one parse.
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): restore constructed-path findings on windowed views
Above SECURITY_VIEW_WINDOW_CHARS the runner hands lexical modules window
slices instead of the whole file, so peek_python_ast never hits the scan's
whole-file cache entry and the constructed-path analysis silently dropped
its findings (regression introduced in 56d121c; 40cea88 reported them).
On a peek miss, parse the provided content directly behind the same plain
textual join gate, restoring the 40cea88 behavior for window slices while
keeping the 17:45 improvements for whole files: alias-aware gate, shared
AST reuse, multiline span dedup, and scan-count discipline (fragments
without a plausible join call never parse).
Regression tests:
- unit: a windowed fragment under a scan keeps PE3 coverage, and a
join-free fragment triggers no parse
- graph: a ~376k-char file keeps its constructed-path PE3 through the
full scan
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): recognize renamed join imports on windowed cache misses
The windowed-fragment fallback gated on the plain join( spelling, so a
fragment calling j( bound by from os.path import join as j never parsed
and silently dropped its constructed-path PE3 (review P1).
The parse gate now also fires on from os.path import spellings: the
fragment is parsed to collect its import aliases, and the existing
alias-aware join gate filters import-only fragments. Fragments without
a plausible join call or join import still never parse.
Regression tests:
- graph: the ~376k-char large-file regression now spells the call
through the renamed import with trailing comment padding
- unit: a windowed fragment with from os.path import join as j keeps
its PE3 coverage under a scan
Signed-off-by: Deepak Jain <[email protected]>
* fix(patterns): carry whole-file join aliases into windowed fragments
A renamed os.path.join import (from os.path import join as j) and its call
site can fall in different runner windows. The call's fragment carries no
import aliases of its own and never spells join(, so the textual gate
skipped it before alias resolution and the constructed sensitive path was
silently dropped.
Carry the whole file's import-alias map from the scan cache into the
fragment analysis: peek the whole-file parse by filename (no content
match), use its aliases for the join pre-check gate, the alias-aware gate,
and call resolution. Fragments of the same scanned file share its import
bindings, so the renamed call resolves to os.path.join in the later
window. Standalone and exact-cache-hit paths are unchanged, and fragments
without a plausible join call or join import still never parse.
Adds two regression tests: an analyze-level fragment whose import lives in
another window, and a full node() graph regression with import and call
separated across real runner windows.
Signed-off-by: deepujain <[email protected]>
* fix(patterns): evaluate windowed join calls on the whole-file AST
The windowed fallback parsed each fragment as a standalone module, so a
slice starting mid-block (for example inside a function body) failed
parsing and its constructed-path PE3 findings were silently dropped.
Evaluate the supplemental constructed calls against the whole file's
cached tree instead, reusing the existing cache peek: call spans map
onto fragment-relative lines and only calls starting inside the
fragment are owned by it, preserving full-file syntax, scope and
import aliases as well as the runner's source-coordinate and dedupe
behavior. Fragments that are not slices of the whole file (normalized
views) keep the standalone-parse fallback.
Add direct and graph regressions using the enclosing-function fixture:
a mid-function window now reports exactly one PE3 at the call site.
Signed-off-by: Deepak Jain <[email protected]>
---------
Signed-off-by: Deepak Jain <[email protected]>
Signed-off-by: deepujain <[email protected]>1 parent a8a4e9a commit 5d370aa
4 files changed
Lines changed: 614 additions & 3 deletions
File tree
- src/skillspector
- nodes/analyzers
- tests
- nodes/analyzers
- unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| 25 | + | |
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
26 | 35 | | |
27 | 36 | | |
28 | 37 | | |
| |||
34 | 43 | | |
35 | 44 | | |
36 | 45 | | |
| 46 | + | |
37 | 47 | | |
38 | 48 | | |
39 | 49 | | |
40 | 50 | | |
41 | 51 | | |
42 | 52 | | |
43 | 53 | | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
44 | 63 | | |
45 | 64 | | |
46 | 65 | | |
| |||
659 | 678 | | |
660 | 679 | | |
661 | 680 | | |
662 | | - | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
663 | 854 | | |
664 | 855 | | |
665 | 856 | | |
| |||
808 | 999 | | |
809 | 1000 | | |
810 | 1001 | | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
811 | 1036 | | |
812 | 1037 | | |
813 | 1038 | | |
| |||
955 | 1180 | | |
956 | 1181 | | |
957 | 1182 | | |
958 | | - | |
| 1183 | + | |
| 1184 | + | |
| 1185 | + | |
| 1186 | + | |
| 1187 | + | |
959 | 1188 | | |
960 | 1189 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
337 | 337 | | |
338 | 338 | | |
339 | 339 | | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
340 | 380 | | |
341 | 381 | | |
342 | 382 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
| |||
128 | 129 | | |
129 | 130 | | |
130 | 131 | | |
131 | | - | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
132 | 138 | | |
133 | 139 | | |
134 | 140 | | |
135 | 141 | | |
136 | 142 | | |
| 143 | + | |
137 | 144 | | |
138 | 145 | | |
139 | 146 | | |
| |||
151 | 158 | | |
152 | 159 | | |
153 | 160 | | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
154 | 165 | | |
155 | 166 | | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
0 commit comments