[SCI] Add Ruby build file tree support via Gemfile path directives#186
Conversation
|
🎯 Code Coverage (details) 🔗 Commit SHA: 86ce5e2 | Docs | Datadog PR Page | Give us feedback! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 949e069d07
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| gemfile, err := extractor.OpenLocalDepFile(gemfilePath) | ||
| if err != nil { | ||
| // No adjacent Gemfile — return bare artifact | ||
| return artifact, nil |
There was a problem hiding this comment.
Avoid emitting Gemfile artifacts when the Gemfile is absent
When Gemfile.lock is scanned with artifact extraction enabled but the adjacent Gemfile is missing, this returns an artifact whose filename is the non-existent Gemfile. The SBOM code turns file-type artifacts into real file components, and GetBuildFileTrees treats those components as actual build files, so lockfile-only scans now report a bogus Gemfile node instead of omitting Ruby build-file relationships or falling back to the lockfile. Return nil or a lockfile-keyed bare artifact in the missing-Gemfile path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed. When no adjacent Gemfile exists, GetArtifact now returns nil instead of an artifact with a non-existent filename. Additionally, the artifact Name is now populated from the adjacent .gemspec (spec.name = "...") to provide a meaningful semantic identifier rather than leaving it empty.
Implement ArtifactExtractor on GemfileLockExtractor to parse `path:` directives from Gemfile `gem()` calls using tree-sitter, enabling build file tree analysis for Ruby monorepos. - Add findGemsWithPath/findPathInPairs tree-sitter helpers in match-gemfile.go - Add GetArtifact method on GemfileLockExtractor in parse-gemfile-lock.go - Register FileTypeGemfile with SimpleProcessor for BFS traversal - Add comprehensive tests covering path extraction, deduplication, missing targets, scan root boundaries, and group blocks Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…emfile absent - Extract gem name from adjacent .gemspec (spec.name = "...") to populate ArtifactDetail.Name with a semantic identifier, not just a file path - Return nil artifact when no adjacent Gemfile exists, avoiding emission of artifacts with non-existent filenames - Add tests for gemspec name extraction and no-gemspec fallback
7bd5a50 to
55fb242
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55fb2420d4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| seen := make(map[string]struct{}) | ||
|
|
||
| for _, pathVal := range pathValues { |
There was a problem hiding this comment.
Filter Gemfile path edges through the lockfile
When a Gemfile is stale or has conditionally evaluated gem ..., path: entries that are not selected in Gemfile.lock, this loop still emits every parsed path as a ProjectDep; the SBOM writer then creates file components and dependency edges for those paths, so GetBuildFileTrees can report Ruby project dependencies that are absent from the locked package graph. The existing Gemfile matcher treats the lockfile as the source of truth by skipping gems missing from Gemfile.lock, so artifact extraction should similarly only emit path edges for gems/specs present in the lockfile.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Not applying. The os.Stat check already filters out non-existent path targets, which covers the meaningful case. Cross-referencing the lockfile's PATH section would add complexity to guard against a stale/misconfigured Gemfile — a project state we treat as out of scope here.
| (pair | ||
| key: [(hash_key_symbol) (simple_symbol)] @pair_key | ||
| (#match? @pair_key "path") | ||
| value: (string) @pair_value | ||
| ) | ||
| )` | ||
|
|
||
| var pathValue string | ||
| err := node.Query(pairQuery, func(match *extractor.MatchResult) error { | ||
| pairValueNode := match.FindFirstByName("pair_value") | ||
| val, err := node.Ctx.ExtractTextValue(pairValueNode.TSNode) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| pathValue = val | ||
|
|
||
| return nil | ||
| }) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
|
|
||
| return pathValue, nil | ||
| } | ||
|
|
||
| // findGemsWithPath returns the path: values from gem() calls in the given node. | ||
| // It searches both top-level gem calls and gem calls inside group blocks. | ||
| func findGemsWithPath(node *extractor.Node) ([]string, error) { | ||
| var paths []string | ||
|
|
||
| // Find top-level gem calls and extract path: pairs from each | ||
| rootGems, err := collectPathsFromGemCalls(node) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| paths = append(paths, rootGems...) | ||
|
|
||
| // Find grouped gem calls and extract path: pairs from each | ||
| groupedGems, err := collectPathsFromGroupedGemCalls(node) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| paths = append(paths, groupedGems...) | ||
|
|
||
| return paths, nil | ||
| } | ||
|
|
||
| // collectPathsFromGemCalls finds top-level gem() calls and returns any path: values. | ||
| func collectPathsFromGemCalls(node *extractor.Node) ([]string, error) { | ||
| gemQueryString := `( | ||
| (call | ||
| method: (identifier) @method_name |
There was a problem hiding this comment.
this function calls collectPathsFromGemCalls which we are already calling few lines above, so I'm guessing we are possibly returning some duplicated entries. Can't we simply rely on the first collectPathsFromGemCalls ?
There was a problem hiding this comment.
Fixed. The tree-sitter query in collectPathsFromGemCalls already matches gem() calls at any depth, so gems inside group blocks were being collected twice. Removed collectPathsFromGroupedGemCalls and simplified findGemsWithPath to a single call.
collectPathsFromGemCalls already searches the full node subtree via tree-sitter, so gems inside group blocks are already captured. Remove collectPathsFromGroupedGemCalls and simplify findGemsWithPath to a single call, eliminating potential duplicates.
…nto one The wrapper was a one-liner with no added value; inline the implementation directly into findGemsWithPath.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e8b240b4a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| artifact.ProjectDeps = append(artifact.ProjectDeps, models.ArtifactDetail{ | ||
| Filename: targetGemfile, | ||
| }) |
There was a problem hiding this comment.
Skip self-resolving Gemfile path refs
When a Gemfile contains a valid local self-reference such as gem "my_gem", path: ".", targetGemfile cleans to the same path as gemfilePath, but this still emits it as a ProjectDep. The CycloneDX writer turns that into a build-file dependency from the Gemfile to itself, which is a false internal project edge and can confuse downstream consumers of the dependency graph; skip targets equal to the current Gemfile before appending.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed. Added a guard before the dedup check to skip targetGemfile when it equals gemfilePath, preventing a self-loop in the build file tree.
- skip targets equal to gemfilePath before dedup to avoid a self-loop in the build file tree
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f2bc275e74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ArtifactDetail: models.ArtifactDetail{ | ||
| Filename: gemfilePath, | ||
| Ecosystem: models.EcosystemRubyGems, | ||
| Name: findAdjacentGemspecName(gemfileDir), |
There was a problem hiding this comment.
Avoid matching gemspec self packages to Gemfile artifacts
When a gem project uses the standard gemspec form in its Gemfile, the lockfile contains the project gem itself in a PATH section while that package's BlockLocation remains Gemfile.lock because the Gemfile/gemspec matchers only enrich declared dependencies. Populating artifact.Name here makes findArtifact match that self package, so createFileComponents adds a dependency with Ref = Gemfile.lock pointing to this Gemfile, but addFileDependencies only emits the Gemfile file component. The generated CycloneDX SBOM therefore has a dangling lockfile→Gemfile dependency for ordinary gem projects; either avoid matching the current artifact to the self package or emit the self package location/component consistently.
Useful? React with 👍 / 👎.
…fact matching A standard gem project (gemspec form in Gemfile) caused a dangling Gemfile.lock→Gemfile dependency in the CycloneDX SBOM. The self-gem appears in the PATH section of Gemfile.lock with BlockLocation=Gemfile.lock; because artifact.Name was set to the gemspec name, findArtifact matched it and createFileComponents emitted a Ref=Gemfile.lock dependency that had no corresponding file component. Fix: add ArtifactDetail.PackageName, used exclusively for the datadog:maven-package SBOM property (and thus ctx.ArtifactIDs / BuildFileRelations.ID). Ruby's GetArtifact now sets PackageName instead of Name, so findArtifact never matches the self-gem. Gradle and other ecosystems that need cross-project findArtifact matching continue to use Name unchanged.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 86ce5e29ab
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| gemfile, err := extractor.OpenLocalDepFile(gemfilePath) | ||
| if err != nil { | ||
| // No adjacent Gemfile — no build file tree to emit | ||
| return nil, nil |
There was a problem hiding this comment.
Suppress fallback artifacts when Gemfile is missing
When artifact extraction scans a Ruby directory that has Gemfile.lock but no adjacent Gemfile, this nil return does not actually omit the artifact: ExtractDeps synthesizes a bare ScannedArtifact{Filename: f.Path()} whenever an ArtifactExtractor returns nil. Fresh evidence after the prior fix is that fallback path, so lockfile-only Ruby scans still emit a Gemfile.lock file component, and unfiltered GetBuildFileTrees can report the lockfile as a manifest build file instead of omitting Ruby build-file relationships for that directory.
Useful? React with 👍 / 👎.
Summary
ArtifactExtractoronGemfileLockExtractorto detect internal Ruby gem dependencies viapath:directives in Gemfilegem()callsSimpleProcessorregistration forFileTypeGemfilesoGetBuildFileTreescan resolve parent/child relationships for Gemfile build filesparse-go-lock.go) and .NET (csproj-project-refs.go) forProjectDepsemissionHow it works
In Ruby monorepos, Gemfiles reference local gems via
path:directives:GetArtifactopens the adjacent Gemfile (from the Gemfile.lock directory), parses it with tree-sitter to findpath:keyword arguments ingem()calls, resolves each path to an absolute Gemfile path, verifies the target exists on disk, and emits it as aProjectDep. This enables the SBOM pipeline to construct internal dependency trees for Ruby monorepos.Path directives inside
groupblocks are also captured.Changes
pkg/extractor/ruby/match-gemfile.gofindGemsWithPath,findPathInPairs, and helper functions for tree-sitter path extractionpkg/extractor/ruby/parse-gemfile-lock.goGetArtifactwith path resolution, deduplication, scan root boundary checkspkg/sbomgen/simple_processor.goSimpleProcessorforFileTypeGemfilepkg/extractor/ruby/parse-gemfile-lock-artifact_test.goTest plan
./scripts/run_tests.sh)./scripts/run_lints.sh)var _ extractor.ArtifactExtractor = GemfileLockExtractor{}🤖 Generated with Claude Code