Repository navigation
feat(supply_chain): scan tool-specific pyproject.toml dependency tables (Poetry/PDM/Hatch/uv) - #190
Conversation
| table = poetry.get(table_name) | ||
| if isinstance(table, dict): | ||
| specs.extend( | ||
| pkg for pkg in table if isinstance(pkg, str) and pkg != "python" |
There was a problem hiding this comment.
[Automated SkillSpector Review]
Non-blocking: only the Poetry dependency name (the table key) is collected here; the version constraint (the value, e.g. "^2.28" or {version = "..."}) is dropped, so SC-rules that key off versions can't evaluate Poetry-declared deps. The PEP 508 paths (PDM/Hatch/uv) retain versions. Fine as a follow-up.
rng1995
left a comment
There was a problem hiding this comment.
[Automated SkillSpector Review]
Approved.
Closes a real supply-chain false-negative gap: dependencies declared in tool-specific tables (Poetry, PDM, Hatch, uv) were silently skipped and are now extracted. The parsing is defensive throughout (isinstance guards at every level), excludes Poetry's python interpreter key, and the 6 tests include a no-false-positive check for pure [tool.black]/[tool.mypy] config sections.
Non-blocking: for Poetry tables only the package name (dict key) is captured, so the version constraint (the value) is dropped and version-based vuln matching won't apply to Poetry-declared deps. PEP 508 strings (PDM/Hatch/uv) keep their versions. Worth a follow-up if version-aware checks matter for Poetry projects.
|
@Shrotriya-lalit - Please address review comments and resolve merge conflicts. |
|
@Shrotriya-lalit - Please resolve merge conflicts to merge the PR. Thanks for your contribution! |
|
Powered by Codex: PR council review result. This is a triage signal, not a maintainer approval.
|
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Changes are required at 11a0b5a. The tool-table coverage is valuable, but the branch conflicts with a substantially evolved current parser. A direct resolution must preserve current limit enforcement, [build-system].requires coverage, packaging.Requirement parsing, pinned-version semantics, and lockfile reconciliation; the stale implementation shown here drops those contracts and discards Poetry version constraints. Rebase onto current main, integrate the new tables through the bounded parser, add limit/version regressions, and restore hosted CI before approval.
|
@Shrotriya-lalit - Please address review comments and resolve merge conflicts to merge the PR. Thanks for your contribution! |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
@Shrotriya-lalit thank you for following through on the Poetry/PDM/Hatch/uv gap you committed to in #95, and apologies that it sat after the June approval while the parser changed underneath it.
Value and readiness
The gap is still real. Main's _extract_packages_from_pyproject reads only [project], [dependency-groups] and [build-system].requires, so [tool.*] dependency tables are still skipped and #171 is still open. The PR cannot merge, though: main rewrote the parser, and both changed files conflict.
Material findings
Prior findings:
- Blocker: rebase onto the bounded parser. Still open, unchanged since the Sept 12 review. The new tables need to go through
extend_specs, solimitis honoured, and then through_extract_python_requirement(packaging.Requirement) like the other tables. Please add limit and version regressions. - Poetry version constraint dropped (
static_patterns_supply_chain.py:488, open thread): still open. On current main, a bare Poetry version such asrequests = "2.31.0"is an exact pin. It should reach_pinned_versionas==2.31.0, so SC4 can look it up. Ranged constraints like^2.28can stay unpinned. Main's lockfile reconciliation still fills exact versions frompoetry.lockwhen one is present.
There are no new issues.
PIC tradeoffs
There has been no author activity since Jun 24, and three maintainer pings (Aug 14, Aug 20, Sep 26) have gone unanswered. If the author cannot continue, a maintainer could carry the rebase, since the table walk is small. The other option is to close this PR and keep #171 open.
Verification and gaps
- Ran
git merge-treeagainstmainat8b993d32: conflicts instatic_patterns_supply_chain.pyandtests/unit/test_patterns_new.py. - Read main's
_extract_packages_from_pyproject,_extract_python_requirementand_pinned_version. Searched main fortool.poetry,tool.pdm,tool.hatchandtool.uvhandling and found none. - No CI checks are recorded on this head.
- Static review only.
Route [tool.poetry.*], [tool.pdm.dev-dependencies],
[tool.hatch.envs.*.dependencies] and [tool.uv.dev-dependencies] through the
bounded extend_specs parser so `limit` is honoured and PEP 508 entries go
through packaging.Requirement.
Poetry bare versions ("2.31.0") are exact pins and become name==2.31.0 so
SC4 can look them up; ranges (^, ~, *, >=,<) stay unpinned. Poetry's
`python` key is skipped.
Closes NVIDIA#171
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Lalit Shrotriya <[email protected]>
|
Thanks @rng1995, and apologies for the delay. I've rebuilt this on current
|
Head branch was pushed to by a user without write access
11a0b5a to
eac4c42
Compare
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision: Changes Requested (reviewed head eac4c42efef6c1ae7add83ecca9ef951cee2721d).
Value and readiness: The rebuild integrates the tool tables with the current bounded dependency parser and preserves concrete Poetry pins. This addresses the earlier requested architecture and version fixes. One conversion case still silently loses a valid dependency.
Previous findings:
- Resolved — bounded parser integration: all new tables feed
extend_specsand retain its limit; extracted strings pass through_extract_python_requirement. PEP 621, dependency groups, build-system requirements, and the later lockfile reconciliation path remain present. - Resolved — bare Poetry exact versions: string and table
versionvalues such as2.31.0become==2.31.0and reach_pinned_version; ranges and git/path entries retain an unpinned package. Tests cover exact/range behavior, line location, limits, the interpreter key, and non-dependency configuration. - Resolved — stale branch and missing CI: the current diff is rebuilt against the evolved parser, and six hosted checks pass at this head. The author's new follow-up is relevant; the code and checks, rather than that reply, establish the fixes.
Material finding — blocker, P2: src/skillspector/nodes/analyzers/static_patterns_supply_chain.py:1582-1583 assumes any digit-leading version without a comma or * is a concrete version. Poetry supports disjunctions such as requests = "1.2.3 || 1.2.4" (Poetry core's constraint parser explicitly splits ||). This branch creates requests==1.2.3 || 1.2.4, which is not a PEP 508 requirement. _extract_python_requirement returns None on InvalidRequirement at lines 1275-1278, and the loop at lines 1720-1722 skips the dependency entirely. _analyze_dependencies_detailed passes that already-filtered list to lockfile reconciliation at lines 3362-3363, so even a lockfile cannot restore the dropped package. Name-based SC5/SC6 checks also lose it.
Please add == only after proving the complete constraint is a single concrete version. For other Poetry syntax, retain the package name as unpinned. Add regressions for string and {version=...} OR constraints that assert the package survives with version None, including a downstream lockfile or name-based analysis assertion.
PIC tradeoffs: None requiring a separate decision; retaining unpinned identity matches the documented scope.
Verification and gaps: Reviewed the complete two-file diff, pinned parser and scanner call path, all prior own marked reviews and replies, and the supplied official Poetry parser source as data. The added tests have meaningful positive, negative, boundary, and pin assertions, but no digit-leading OR constraint. All six hosted checks passed for this exact head (run). No contributor code, tests, imports, builds, setup, hooks, or regex probes were executed. The dropped-package conclusion is a static trace, not a claimed runtime reproduction.
…e release
A digit-leading Poetry constraint such as "1.2.3 || 1.2.4" was rewritten to
the invalid requirement "name==1.2.3 || 1.2.4", which packaging rejects, so
the dependency was dropped from SC4/SC5/SC6 and lockfile reconciliation.
Gate the == rewrite on a strict single-version pattern; any other Poetry
syntax (||, spaces, commas, wildcards) keeps the package name unpinned.
Add regressions for string and {version = ...} forms.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Lalit Shrotriya <[email protected]>
|
Thanks @rng1995, good catch. Fixed in the latest commit: the |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision: Approved
Value and readiness: This closes missing dependency inventory for widely used pyproject tool tables while retaining the current bounded parser. The latest correction resolves the last reported dropped-package case, and no required code change remains.
Material findings: Prior bounded-parser integration, concrete Poetry pins, current parser compatibility and CI findings remain resolved: new entries flow through extend_specs and Requirement, retain the extraction limit, preserve PEP 621/build-system paths, and reach the unchanged downstream locked-version reconciliation. The last OR-constraint blocker is now resolved at static_patterns_supply_chain.py:1566-1588: only the whole concrete-version pattern gets an == prefix, while disjunctions, spaces and wildcard forms retain the bare package name. Added string/dict OR controls explicitly assert (requests,None,2); the downstream entry-point control also retains the parsed package. Existing tests cover exact pins, line locations, interpreter exclusion, tool sections, limits and unrelated configuration. No new material finding identified. The old unresolved version-preservation thread remains an action for its owner even though the source request is addressed.
PIC tradeoffs: No new tradeoff requiring a separate decision. Poetry-only ranges remain unpinned, retaining name-based checks and later lockfile reconciliation rather than approximating unsupported version syntax.
Verification and gaps: Read the complete two-file diff, exact-head conversion/extraction functions and downstream lockfile call path, prior findings and the author's new OR-fix reply. All six exact-head hosted checks succeeded. Regression assertions were inspected as source; no contributor tests, application code, imports, setup, builds, hooks, scanner calls or regex probes were executed.
Reviewed head: 2ce9c2c6d19e5d59b14dc8ddace00e61c95b8150.
Problem
_extract_packages_from_pyprojectonly read the PEP 621[project]tables and PEP 735[dependency-groups]. Projects using Poetry, PDM, Hatch, or uv declare their dependencies in[tool.*]tables that were silently skipped, producing false negatives in the supply-chain analyzer.Committed to in the PR #95 discussion: "I'll open a follow-up issue to track this... the tool-table coverage can land as an incremental improvement."
Closes #171
Changes
src/skillspector/nodes/analyzers/static_patterns_supply_chain.pyExtended
_extract_packages_from_pyprojectto also read:[tool.poetry.dependencies][tool.poetry.dev-dependencies][tool.poetry.group.<name>.dependencies][tool.pdm.dev-dependencies][tool.hatch.envs.<name>.dependencies][tool.uv.dev-dependencies]Poetry's special
pythonkey (interpreter constraint, not a package) is excluded. All non-dependency tool config sections ([tool.black],[tool.mypy], etc.) are unaffected.tests/unit/test_patterns_new.pySix new tests:
test_pyproject_poetry_deps_extracted—[tool.poetry.dependencies]+[tool.poetry.dev-dependencies], confirmspythonkey excludedtest_pyproject_poetry_groups_extracted—[tool.poetry.group.<name>.dependencies]test_pyproject_pdm_dev_deps_extracted—[tool.pdm.dev-dependencies]groupstest_pyproject_hatch_env_deps_extracted—[tool.hatch.envs.<name>.dependencies]per-env liststest_pyproject_uv_dev_deps_extracted—[tool.uv] dev-dependencieslisttest_pyproject_tool_tables_no_false_positives_on_config_keys— pure config sections produce no findingsTest results
191 tests in
test_patterns_new.pypass. Full suite: 932 passed, 12 skipped, 6 xfailed.