Skip to content

feat(supply_chain): scan tool-specific pyproject.toml dependency tables (Poetry/PDM/Hatch/uv) - #190

Open
Shrotriya-lalit wants to merge 2 commits into
NVIDIA:mainfrom
Shrotriya-lalit:fix/issue-171-pyproject-tool-table-deps
Open

Shrotriya-lalit wants to merge 2 commits into
NVIDIA:mainfrom
Shrotriya-lalit:fix/issue-171-pyproject-tool-table-deps

Conversation

@Shrotriya-lalit

Copy link
Copy Markdown
Contributor

Problem

_extract_packages_from_pyproject only read the PEP 621 [project] tables and PEP 735 [dependency-groups]. Projects using Poetry, PDM, Hatch, or uv declare their dependencies in [tool.*] tables that were silently skipped, producing false negatives in the supply-chain analyzer.

Committed to in the PR #95 discussion: "I'll open a follow-up issue to track this... the tool-table coverage can land as an incremental improvement."

Closes #171

Changes

src/skillspector/nodes/analyzers/static_patterns_supply_chain.py

Extended _extract_packages_from_pyproject to also read:

Table Tool
[tool.poetry.dependencies] Poetry (runtime)
[tool.poetry.dev-dependencies] Poetry (dev, legacy)
[tool.poetry.group.<name>.dependencies] Poetry (new-style groups)
[tool.pdm.dev-dependencies] PDM
[tool.hatch.envs.<name>.dependencies] Hatch (per-environment)
[tool.uv.dev-dependencies] uv

Poetry's special python key (interpreter constraint, not a package) is excluded. All non-dependency tool config sections ([tool.black], [tool.mypy], etc.) are unaffected.

tests/unit/test_patterns_new.py

Six new tests:

  • test_pyproject_poetry_deps_extracted — [tool.poetry.dependencies] + [tool.poetry.dev-dependencies], confirms python key excluded
  • test_pyproject_poetry_groups_extracted — [tool.poetry.group.<name>.dependencies]
  • test_pyproject_pdm_dev_deps_extracted — [tool.pdm.dev-dependencies] groups
  • test_pyproject_hatch_env_deps_extracted — [tool.hatch.envs.<name>.dependencies] per-env lists
  • test_pyproject_uv_dev_deps_extracted — [tool.uv] dev-dependencies list
  • test_pyproject_tool_tables_no_false_positives_on_config_keys — pure config sections produce no findings

Test results

191 tests in test_patterns_new.py pass. Full suite: 932 passed, 12 skipped, 6 xfailed.

table = poetry.get(table_name)
if isinstance(table, dict):
specs.extend(
pkg for pkg in table if isinstance(pkg, str) and pkg != "python"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Non-blocking: only the Poetry dependency name (the table key) is collected here; the version constraint (the value, e.g. "^2.28" or {version = "..."}) is dropped, so SC-rules that key off versions can't evaluate Poetry-declared deps. The PEP 508 paths (PDM/Hatch/uv) retain versions. Fine as a follow-up.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Approved.

Closes a real supply-chain false-negative gap: dependencies declared in tool-specific tables (Poetry, PDM, Hatch, uv) were silently skipped and are now extracted. The parsing is defensive throughout (isinstance guards at every level), excludes Poetry's python interpreter key, and the 6 tests include a no-false-positive check for pure [tool.black]/[tool.mypy] config sections.

Non-blocking: for Poetry tables only the package name (dict key) is captured, so the version constraint (the value) is dropped and version-based vuln matching won't apply to Poetry-declared deps. PEP 508 strings (PDM/Hatch/uv) keep their versions. Worth a follow-up if version-aware checks matter for Poetry projects.

@rng1995

rng1995 commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

@Shrotriya-lalit - Please address review comments and resolve merge conflicts.

@rng1995

rng1995 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

@Shrotriya-lalit - Please resolve merge conflicts to merge the PR. Thanks for your contribution!

@mohgupta-ship-it

mohgupta-ship-it commented Aug 25, 2026 •

Copy link
Copy Markdown
Member

Powered by Codex: PR council review result.

This is a triage signal, not a maintainer approval.

  • Rating: bugfix
  • Confidence: high
  • Status read: Approved, no checks, conflicting
  • Review method: fresh GitHub metadata/body/files/reviews/checks plus selected diffs; council lenses were spec fit, dead-code/reachability, YAGNI/scope, design/coupling, and code standards/tests.
  • Council assessment: Fixes supply-chain false negatives for Poetry/PDM/Hatch/uv pyproject dependency tables. Watch version-preservation during rebase.
  • Recommended action: Rebase onto current parser and preserve version extraction where possible.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Changes are required at 11a0b5a. The tool-table coverage is valuable, but the branch conflicts with a substantially evolved current parser. A direct resolution must preserve current limit enforcement, [build-system].requires coverage, packaging.Requirement parsing, pinned-version semantics, and lockfile reconciliation; the stale implementation shown here drops those contracts and discards Poetry version constraints. Rebase onto current main, integrate the new tables through the bounded parser, add limit/version regressions, and restore hosted CI before approval.

@rng1995

rng1995 commented Sep 26, 2026

Copy link
Copy Markdown
Collaborator

@Shrotriya-lalit - Please address review comments and resolve merge conflicts to merge the PR. Thanks for your contribution!

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

@Shrotriya-lalit thank you for following through on the Poetry/PDM/Hatch/uv gap you committed to in #95, and apologies that it sat after the June approval while the parser changed underneath it.

Value and readiness

The gap is still real. Main's _extract_packages_from_pyproject reads only [project], [dependency-groups] and [build-system].requires, so [tool.*] dependency tables are still skipped and #171 is still open. The PR cannot merge, though: main rewrote the parser, and both changed files conflict.

Material findings

Prior findings:

  1. Blocker: rebase onto the bounded parser. Still open, unchanged since the Sept 12 review. The new tables need to go through extend_specs, so limit is honoured, and then through _extract_python_requirement (packaging.Requirement) like the other tables. Please add limit and version regressions.
  2. Poetry version constraint dropped (static_patterns_supply_chain.py:488, open thread): still open. On current main, a bare Poetry version such as requests = "2.31.0" is an exact pin. It should reach _pinned_version as ==2.31.0, so SC4 can look it up. Ranged constraints like ^2.28 can stay unpinned. Main's lockfile reconciliation still fills exact versions from poetry.lock when one is present.

There are no new issues.

PIC tradeoffs

There has been no author activity since Jun 24, and three maintainer pings (Aug 14, Aug 20, Sep 26) have gone unanswered. If the author cannot continue, a maintainer could carry the rebase, since the table walk is small. The other option is to close this PR and keep #171 open.

Verification and gaps

  • Ran git merge-tree against main at 8b993d32: conflicts in static_patterns_supply_chain.py and tests/unit/test_patterns_new.py.
  • Read main's _extract_packages_from_pyproject, _extract_python_requirement and _pinned_version. Searched main for tool.poetry, tool.pdm, tool.hatch and tool.uv handling and found none.
  • No CI checks are recorded on this head.
  • Static review only.

Route [tool.poetry.*], [tool.pdm.dev-dependencies],
[tool.hatch.envs.*.dependencies] and [tool.uv.dev-dependencies] through the
bounded extend_specs parser so `limit` is honoured and PEP 508 entries go
through packaging.Requirement.

Poetry bare versions ("2.31.0") are exact pins and become name==2.31.0 so
SC4 can look them up; ranges (^, ~, *, >=,<) stay unpinned. Poetry's
`python` key is skipped.

Closes NVIDIA#171

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Lalit Shrotriya <[email protected]>
@Shrotriya-lalit

Copy link
Copy Markdown
Contributor Author

Thanks @rng1995, and apologies for the delay. I've rebuilt this on current main (62c2fe8) instead of rebasing the stale commit, addressing the Oct 14 review:

  1. Bounded parser: Poetry/PDM/Hatch/uv tables now go through extend_specs, so limit is honoured, and PEP 508 entries go through _extract_python_requirement (packaging.Requirement).
  2. Poetry versions: a bare version (requests = "2.31.0", or {version = "2.31.0"}) becomes requests==2.31.0, so it reaches _pinned_version and SC4. Ranges (^, ~, *, >=,<) and git/path deps stay unpinned. Lockfile reconciliation is unchanged.
  3. Regressions added: limit handling, exact-pin vs range, line numbers, python key skipped, and pure config sections yielding nothing.

tests/unit passes locally (4322 passed, 22 skipped) and ruff is clean. The branch is directly on top of main with no conflicts. Could you re-review and approve the CI run? Closes #171.

auto-merge was automatically disabled November 5, 2026 07:14

Head branch was pushed to by a user without write access

@Shrotriya-lalit
Shrotriya-lalit force-pushed the fix/issue-171-pyproject-tool-table-deps branch from 11a0b5a to eac4c42 Compare November 5, 2026 07:14

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Decision: Changes Requested (reviewed head eac4c42efef6c1ae7add83ecca9ef951cee2721d).

Value and readiness: The rebuild integrates the tool tables with the current bounded dependency parser and preserves concrete Poetry pins. This addresses the earlier requested architecture and version fixes. One conversion case still silently loses a valid dependency.

Previous findings:

  • Resolved — bounded parser integration: all new tables feed extend_specs and retain its limit; extracted strings pass through _extract_python_requirement. PEP 621, dependency groups, build-system requirements, and the later lockfile reconciliation path remain present.
  • Resolved — bare Poetry exact versions: string and table version values such as 2.31.0 become ==2.31.0 and reach _pinned_version; ranges and git/path entries retain an unpinned package. Tests cover exact/range behavior, line location, limits, the interpreter key, and non-dependency configuration.
  • Resolved — stale branch and missing CI: the current diff is rebuilt against the evolved parser, and six hosted checks pass at this head. The author's new follow-up is relevant; the code and checks, rather than that reply, establish the fixes.

Material finding — blocker, P2: src/skillspector/nodes/analyzers/static_patterns_supply_chain.py:1582-1583 assumes any digit-leading version without a comma or * is a concrete version. Poetry supports disjunctions such as requests = "1.2.3 || 1.2.4" (Poetry core's constraint parser explicitly splits ||). This branch creates requests==1.2.3 || 1.2.4, which is not a PEP 508 requirement. _extract_python_requirement returns None on InvalidRequirement at lines 1275-1278, and the loop at lines 1720-1722 skips the dependency entirely. _analyze_dependencies_detailed passes that already-filtered list to lockfile reconciliation at lines 3362-3363, so even a lockfile cannot restore the dropped package. Name-based SC5/SC6 checks also lose it.

Please add == only after proving the complete constraint is a single concrete version. For other Poetry syntax, retain the package name as unpinned. Add regressions for string and {version=...} OR constraints that assert the package survives with version None, including a downstream lockfile or name-based analysis assertion.

PIC tradeoffs: None requiring a separate decision; retaining unpinned identity matches the documented scope.

Verification and gaps: Reviewed the complete two-file diff, pinned parser and scanner call path, all prior own marked reviews and replies, and the supplied official Poetry parser source as data. The added tests have meaningful positive, negative, boundary, and pin assertions, but no digit-leading OR constraint. All six hosted checks passed for this exact head (run). No contributor code, tests, imports, builds, setup, hooks, or regex probes were executed. The dropped-package conclusion is a static trace, not a claimed runtime reproduction.

…e release

A digit-leading Poetry constraint such as "1.2.3 || 1.2.4" was rewritten to
the invalid requirement "name==1.2.3 || 1.2.4", which packaging rejects, so
the dependency was dropped from SC4/SC5/SC6 and lockfile reconciliation.

Gate the == rewrite on a strict single-version pattern; any other Poetry
syntax (||, spaces, commas, wildcards) keeps the package name unpinned.
Add regressions for string and {version = ...} forms.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Lalit Shrotriya <[email protected]>
@Shrotriya-lalit

Copy link
Copy Markdown
Contributor Author

Thanks @rng1995, good catch. Fixed in the latest commit: the == rewrite is now gated on a strict single-concrete-version pattern, so || disjunctions, spaces, commas and wildcards (string or {version = ...} form) keep the package unpinned instead of producing an invalid requirement. Added regressions asserting the package survives with version None and that the scanner entry point handles it. test_patterns_new.py passes locally (882 tests) and ruff is clean; waiting on CI. Ready for another look.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Decision: Approved

Value and readiness: This closes missing dependency inventory for widely used pyproject tool tables while retaining the current bounded parser. The latest correction resolves the last reported dropped-package case, and no required code change remains.

Material findings: Prior bounded-parser integration, concrete Poetry pins, current parser compatibility and CI findings remain resolved: new entries flow through extend_specs and Requirement, retain the extraction limit, preserve PEP 621/build-system paths, and reach the unchanged downstream locked-version reconciliation. The last OR-constraint blocker is now resolved at static_patterns_supply_chain.py:1566-1588: only the whole concrete-version pattern gets an == prefix, while disjunctions, spaces and wildcard forms retain the bare package name. Added string/dict OR controls explicitly assert (requests,None,2); the downstream entry-point control also retains the parsed package. Existing tests cover exact pins, line locations, interpreter exclusion, tool sections, limits and unrelated configuration. No new material finding identified. The old unresolved version-preservation thread remains an action for its owner even though the source request is addressed.

PIC tradeoffs: No new tradeoff requiring a separate decision. Poetry-only ranges remain unpinned, retaining name-based checks and later lockfile reconciliation rather than approximating unsupported version syntax.

Verification and gaps: Read the complete two-file diff, exact-head conversion/extraction functions and downstream lockfile call path, prior findings and the author's new OR-fix reply. All six exact-head hosted checks succeeded. Regression assertions were inspected as source; no contributor tests, application code, imports, setup, builds, hooks, scanner calls or regex probes were executed.

Reviewed head: 2ce9c2c6d19e5d59b14dc8ddace00e61c95b8150.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(supply_chain): scan tool-specific dependency tables in pyproject.toml (Poetry, PDM, Hatch, uv)

3 participants