Skip to content

[SCI] add build file tree support for npm/yarn/pnpm/bun workspaces#187

Merged
anderruiz merged 11 commits into
mainfrom
ander/npm-build-file-trees
Jul 8, 2026
Merged

[SCI] add build file tree support for npm/yarn/pnpm/bun workspaces#187
anderruiz merged 11 commits into
mainfrom
ander/npm-build-file-trees

Conversation

@anderruiz

Copy link
Copy Markdown
Contributor

Summary

Test plan

  • Unit tests for each extractor's GetArtifact: workspace monorepo, single package, missing package.json
  • Unit test for SimpleProcessor registration of FileTypePackageJSON
  • Integration test: end-to-end npm workspace fixture producing correct BuildFileRelations
  • Flag gating: ExtractArtifactIds: false produces no artifacts
  • Regression: go test ./... passes (all existing extractors unaffected)
  • Lint clean: golangci-lint passes

🤖 Generated with Claude Code

@anderruiz
anderruiz requested a review from a team as a code owner July 6, 2026 11:50
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Jul 6, 2026

Copy link
Copy Markdown

Tests

🔄 Datadog auto-retried 1 job - 1 passed on retry View in Datadog

🎯 Code Coverage (details)
Patch Coverage: 60.77%
Overall Coverage: 83.95% (-0.16%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 3f83d25 | Docs | Datadog PR Page | Give us feedback!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 138a4f796a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/extractor/javascript/artifact.go Outdated
Comment thread pkg/extractor/javascript/pnpm-lock-artifact.go Outdated
@anderruiz
anderruiz force-pushed the ander/npm-build-file-trees branch from 299da50 to e69e290 Compare July 6, 2026 14:33

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e69e290754

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/extractor/javascript/artifact.go
@anderruiz anderruiz changed the title feat(javascript): add build file tree support for npm/yarn/pnpm/bun workspaces [SCI] add build file tree support for npm/yarn/pnpm/bun workspaces Jul 6, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ffdc82fab4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/extractor/javascript/pnpm-lock-artifact.go

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7b4283374

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/sbomgen/build_files.go
Comment thread pkg/sbomgen/build_files.go Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90a1fb9304

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/sbomgen/sbomgen.go Outdated
Comment on lines +351 to +354
var pkg struct {
Dependencies map[string]string `json:"dependencies"`
DevDependencies map[string]string `json:"devDependencies"`
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include optional workspace deps in package.json edges

When a workspace package declares another workspace under optionalDependencies, this struct never captures that manifest section, so AddNpmWorkspaceEdges cannot add the build-file edge even though optional dependencies are valid package.json dependencies and are already handled by the existing package-json matcher. In that scenario GetBuildFileTrees(..., FileTypePackageJSON) misses the relationship; include optionalDependencies when collecting workspace dependency names.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not going to address this for now.

anderruiz and others added 9 commits July 7, 2026 11:42
Register FileTypePackageJSON with SimpleProcessor so that package.json
build file trees are resolved via BFS transitive closure, enabling
npm/yarn/pnpm/bun workspace dependency edges in the SBOM.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Each JS lockfile extractor now implements ArtifactExtractor.GetArtifact()
to emit ScannedArtifact for the adjacent package.json with workspace
ProjectDeps, enabling build file tree resolution for Node.js monorepos.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Extract shared getArtifactFromAdjacentPackageJSON helper to satisfy
dupl linter while keeping independent GetArtifact per extractor. Add
end-to-end integration test verifying GenerateSBOM -> GetBuildFileTrees
pipeline for npm workspace monorepos, including flag gating.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Add workspaceGlobs custom UnmarshalJSON to handle both array-form
  ("workspaces": [...]) and object-form ("workspaces": {"packages": [...]})
  in package.json, fixing silent decode failure for Yarn nohoist repos
- Rewrite PnpmLockExtractor.GetArtifact to read workspace packages from
  pnpm-lock.yaml importers map instead of package.json workspaces field,
  covering repos that declare workspaces only via pnpm-workspace.yaml
- Add test for pnpm-workspace.yaml-only workspace discovery
- Apply prettier formatting to npm-workspace test fixtures
…ctDeps

Read the `name` field from each workspace child's package.json so that
ArtifactDetail.Name is set for every ProjectDep, giving GetBuildFileTrees
a meaningful artifact ID for each workspace package instead of an empty string.
…ents

Set Ecosystem on ProjectDeps entries so cyclonedx.go can generate a purl
for stub components. Previously workspace children (e.g. packages/x-cli)
appeared in the SBOM as unnamed file nodes; now they carry their npm package
name (e.g. pkg:npm/%40quramy%2Fx-cli) via the datadog:maven-package property.
…ild file trees

When a workspace child (e.g. packages/x-cli) declares a dependency on a
sibling workspace package (e.g. @quramy/x-core), the edge was missing from
the build file tree because only one ScannedArtifact is produced per lockfile.

AddNpmWorkspaceEdges post-processes the BOM in GenerateSBOM (where the scan
root is known) by reading each package.json file component, cross-referencing
its dependencies against the workspace name→path index built from the BOM's
file components, and injecting any missing edges into bom.Dependencies.

Normal scanning and SBOM generation are unaffected — the enrichment only runs
when ExtractArtifactIds is enabled and the result feeds into GetBuildFileTrees.
@anderruiz
anderruiz force-pushed the ander/npm-build-file-trees branch from 90a1fb9 to e093d00 Compare July 7, 2026 09:44
- constrain pnpm importer paths to scan root via filepath.Rel + ctx.RootDir
- restrict AddNpmWorkspaceEdges nameToPath index to npm purls only
- fix range-loop copy bug in AddNpmWorkspaceEdges merge-into-existing path
- support multi-root scans: AddNpmWorkspaceEdges now accepts variadic rootDirs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5cc58b1da0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/extractor/javascript/pnpm-lock-artifact.go Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3f83d25c51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +367 to +370
for depName := range pkg.Dependencies {
if targetPath, ok := nameToPath[depName]; ok && targetPath != comp.BOMRef {
if _, already := existingSet[targetPath]; !already {
edges = append(edges, targetPath)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Sort workspace edges before serializing

When a package.json has two or more workspace dependencies, iterating these dependency maps appends edges in Go's randomized map order, and the augmented CycloneDX dependency list is serialized without being sorted afterward. This makes GenerateSBOM output nondeterministic for the same workspace across runs, which can break snapshots and cache/diff-based consumers; collect/sort the target paths before writing them back to bom.Dependencies.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not going to address this for now — consumers of GetBuildFileTrees sort their own output, and the SBOM dependency ordering in this section is not guaranteed anyway.

@anderruiz
anderruiz merged commit 9e43bc2 into main Jul 8, 2026
11 of 12 checks passed
@anderruiz
anderruiz deleted the ander/npm-build-file-trees branch July 8, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants