Repository navigation
Commit 025c65e
fix(analyzer): keep Python and Perl syntax from exhausting the shell parser (#778)
* fix(analyzer): keep Python and Perl syntax from exhausting the shell parser
Valid host-language source could make has_bounded_parse_exhaustion report
static_parse_limit. The file was then recorded as partially inspected and
every SKILL.md reference to it became a HIGH AE1 finding. These triggers
are new in 2.12.0:
- Perl `eval BLOCK` (`eval { require $f; 1 } or die $@;`) reached the
shell `eval` command-string reconstruction, and the dynamic `$f`
operand made the reparsed string unresolvable.
- In Python, a string, docstring or comment containing a Markdown fence,
an unpaired backtick or an apostrophe opened a shell quote or command
substitution that ran to the end of the view. Any file with more than
4096 characters after it was partial.
- Python code that uses a shell wrapper name (`signal.alarm(timeout)`,
`if (timeout > 0 ...`) was reparsed as a `timeout` command string, and
a `$0` in a comment was treated as a runtime-selected command whose
operands ran on into later code.
Recognize Perl `eval BLOCK` in .pl files: a clause-initial `eval` followed
by `{` is not a string reparse. `{` already starts a new clause, so the
statements in the block are still scanned. String eval (`eval $code`,
`eval "..."`, `eval qq{...}`) and shell `eval` are unchanged.
For a complete .py module that ast.parse accepts (no non-Python shebang,
no lone CR, within the AST size bound), compute string, f-string and
comment token spans with tokenize and verify them against the source.
Valid Python code outside those tokens contains no shell quote or
expansion opener, so:
- an unresolved shell word is charged only up to the first code-level
word break after it, which ends the string or comment token that
opened the quote, instead of the end of the view;
- a command-string clause (eval, sh -c, wrappers) that begins in Python
code is not reparsed as shell;
- a runtime-selected command inside a comment is rechecked against that
comment alone, because no shell ever receives a comment.
Spans are computed lazily, only when one of these checks would otherwise
fail closed, and they honor the runtime deadline. Fragments, malformed
source, shell-shebang polyglots, a string or comment that itself exceeds
the command-word budget, the destructive rm, brace-expansion and printf
checks, runtime commands built from strings, and all shell and Markdown
behavior keep their existing conservative results. Rule findings are
unchanged.
Refs #694
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(analyzer): keep the Python ownership parse from re-emitting warnings
The module parse that proves Python token ownership repeated every
compiler warning the runner's own parse had already reported, such as an
invalid `"\d"` escape. Ordinary skill code printed duplicate warnings,
and under `-W error::SyntaxWarning` the parse raised and silently
dropped ownership, so the file fell back to static_parse_limit.
Run that parse with warnings ignored. `warnings.catch_warnings` swaps the
process-global filter list, and analyzer nodes run on concurrent graph
worker threads, where two such blocks that exit out of order leave
their `ignore` filter installed for the whole process. Serialize the
block with a module lock.
Pin the existing result for a module that starts with a UTF-8 BOM: the
file cache decodes with utf-8, not utf-8-sig, so a leading U+FEFF fails
the parse and ownership stays unproven. That gap belongs with a separate
decoding fix.
Refs #694
Signed-off-by: Narendran Raghavan <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Signed-off-by: Narendran Raghavan <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent 5485cda commit 025c65e
3 files changed
Lines changed: 715 additions & 23 deletions
File tree
- docs
- src/skillspector/nodes/analyzers
- tests/nodes/analyzers
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
149 | 149 | | |
150 | 150 | | |
151 | 151 | | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
152 | 173 | | |
153 | 174 | | |
154 | 175 | | |
| |||
0 commit comments