Skip to content

[SCI] Add Ruby build file tree support via Gemfile path directives#186

Merged
anderruiz merged 7 commits into
mainfrom
ander/rubygems-build-file-trees
Jul 7, 2026
Merged

[SCI] Add Ruby build file tree support via Gemfile path directives#186
anderruiz merged 7 commits into
mainfrom
ander/rubygems-build-file-trees

Conversation

@anderruiz

Copy link
Copy Markdown
Contributor

Summary

  • Implements ArtifactExtractor on GemfileLockExtractor to detect internal Ruby gem dependencies via path: directives in Gemfile gem() calls
  • Adds SimpleProcessor registration for FileTypeGemfile so GetBuildFileTrees can resolve parent/child relationships for Gemfile build files
  • Follows the same pattern established by Go (parse-go-lock.go) and .NET (csproj-project-refs.go) for ProjectDeps emission

How it works

In Ruby monorepos, Gemfiles reference local gems via path: directives:

gem 'my_lib', path: '../libs/my_lib'

GetArtifact opens the adjacent Gemfile (from the Gemfile.lock directory), parses it with tree-sitter to find path: keyword arguments in gem() calls, resolves each path to an absolute Gemfile path, verifies the target exists on disk, and emits it as a ProjectDep. This enables the SBOM pipeline to construct internal dependency trees for Ruby monorepos.

Path directives inside group blocks are also captured.

Changes

File Change
pkg/extractor/ruby/match-gemfile.go Add findGemsWithPath, findPathInPairs, and helper functions for tree-sitter path extraction
pkg/extractor/ruby/parse-gemfile-lock.go Implement GetArtifact with path resolution, deduplication, scan root boundary checks
pkg/sbomgen/simple_processor.go Register SimpleProcessor for FileTypeGemfile
pkg/extractor/ruby/parse-gemfile-lock-artifact_test.go 6 test cases covering all acceptance criteria

Test plan

  • All 6 new test cases pass (path directives, missing target skip, dedup, outside scan root, no Gemfile, path in group)
  • Full test suite passes (./scripts/run_tests.sh)
  • Linter passes (./scripts/run_lints.sh)
  • Compile-time interface check: var _ extractor.ArtifactExtractor = GemfileLockExtractor{}

🤖 Generated with Claude Code

@anderruiz
anderruiz requested a review from a team as a code owner July 6, 2026 09:03
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jul 6, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 78.68%
Overall Coverage: 84.17% (+0.03%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 86ce5e2 | Docs | Datadog PR Page | Give us feedback!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 949e069d07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +230 to +233
gemfile, err := extractor.OpenLocalDepFile(gemfilePath)
if err != nil {
// No adjacent Gemfile — return bare artifact
return artifact, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid emitting Gemfile artifacts when the Gemfile is absent

When Gemfile.lock is scanned with artifact extraction enabled but the adjacent Gemfile is missing, this returns an artifact whose filename is the non-existent Gemfile. The SBOM code turns file-type artifacts into real file components, and GetBuildFileTrees treats those components as actual build files, so lockfile-only scans now report a bogus Gemfile node instead of omitting Ruby build-file relationships or falling back to the lockfile. Return nil or a lockfile-keyed bare artifact in the missing-Gemfile path.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. When no adjacent Gemfile exists, GetArtifact now returns nil instead of an artifact with a non-existent filename. Additionally, the artifact Name is now populated from the adjacent .gemspec (spec.name = "...") to provide a meaningful semantic identifier rather than leaving it empty.

anderruiz and others added 2 commits July 6, 2026 14:58
Implement ArtifactExtractor on GemfileLockExtractor to parse `path:`
directives from Gemfile `gem()` calls using tree-sitter, enabling build
file tree analysis for Ruby monorepos.

- Add findGemsWithPath/findPathInPairs tree-sitter helpers in match-gemfile.go
- Add GetArtifact method on GemfileLockExtractor in parse-gemfile-lock.go
- Register FileTypeGemfile with SimpleProcessor for BFS traversal
- Add comprehensive tests covering path extraction, deduplication,
  missing targets, scan root boundaries, and group blocks

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…emfile absent

- Extract gem name from adjacent .gemspec (spec.name = "...") to populate
  ArtifactDetail.Name with a semantic identifier, not just a file path
- Return nil artifact when no adjacent Gemfile exists, avoiding emission
  of artifacts with non-existent filenames
- Add tests for gemspec name extraction and no-gemspec fallback
@anderruiz
anderruiz force-pushed the ander/rubygems-build-file-trees branch from 7bd5a50 to 55fb242 Compare July 6, 2026 13:00

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55fb2420d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


seen := make(map[string]struct{})

for _, pathVal := range pathValues {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Filter Gemfile path edges through the lockfile

When a Gemfile is stale or has conditionally evaluated gem ..., path: entries that are not selected in Gemfile.lock, this loop still emits every parsed path as a ProjectDep; the SBOM writer then creates file components and dependency edges for those paths, so GetBuildFileTrees can report Ruby project dependencies that are absent from the locked package graph. The existing Gemfile matcher treats the lockfile as the source of truth by skipping gems missing from Gemfile.lock, so artifact extraction should similarly only emit path edges for gems/specs present in the lockfile.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not applying. The os.Stat check already filters out non-existent path targets, which covers the meaningful case. Cross-referencing the lockfile's PATH section would add complexity to guard against a stale/misconfigured Gemfile — a project state we treat as out of scope here.

Comment on lines +192 to +244
(pair
key: [(hash_key_symbol) (simple_symbol)] @pair_key
(#match? @pair_key "path")
value: (string) @pair_value
)
)`

var pathValue string
err := node.Query(pairQuery, func(match *extractor.MatchResult) error {
pairValueNode := match.FindFirstByName("pair_value")
val, err := node.Ctx.ExtractTextValue(pairValueNode.TSNode)
if err != nil {
return err
}

pathValue = val

return nil
})
if err != nil {
return "", err
}

return pathValue, nil
}

// findGemsWithPath returns the path: values from gem() calls in the given node.
// It searches both top-level gem calls and gem calls inside group blocks.
func findGemsWithPath(node *extractor.Node) ([]string, error) {
var paths []string

// Find top-level gem calls and extract path: pairs from each
rootGems, err := collectPathsFromGemCalls(node)
if err != nil {
return nil, err
}
paths = append(paths, rootGems...)

// Find grouped gem calls and extract path: pairs from each
groupedGems, err := collectPathsFromGroupedGemCalls(node)
if err != nil {
return nil, err
}
paths = append(paths, groupedGems...)

return paths, nil
}

// collectPathsFromGemCalls finds top-level gem() calls and returns any path: values.
func collectPathsFromGemCalls(node *extractor.Node) ([]string, error) {
gemQueryString := `(
(call
method: (identifier) @method_name

@cespio cespio Jul 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function calls collectPathsFromGemCalls which we are already calling few lines above, so I'm guessing we are possibly returning some duplicated entries. Can't we simply rely on the first collectPathsFromGemCalls ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The tree-sitter query in collectPathsFromGemCalls already matches gem() calls at any depth, so gems inside group blocks were being collected twice. Removed collectPathsFromGroupedGemCalls and simplified findGemsWithPath to a single call.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

collectPathsFromGemCalls already searches the full node subtree via
tree-sitter, so gems inside group blocks are already captured. Remove
collectPathsFromGroupedGemCalls and simplify findGemsWithPath to a
single call, eliminating potential duplicates.
@anderruiz
anderruiz requested a review from cespio July 6, 2026 17:54
…nto one

The wrapper was a one-liner with no added value; inline the implementation
directly into findGemsWithPath.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e8b240b4a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +280 to +282
artifact.ProjectDeps = append(artifact.ProjectDeps, models.ArtifactDetail{
Filename: targetGemfile,
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip self-resolving Gemfile path refs

When a Gemfile contains a valid local self-reference such as gem "my_gem", path: ".", targetGemfile cleans to the same path as gemfilePath, but this still emits it as a ProjectDep. The CycloneDX writer turns that into a build-file dependency from the Gemfile to itself, which is a false internal project edge and can confuse downstream consumers of the dependency graph; skip targets equal to the current Gemfile before appending.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Added a guard before the dedup check to skip targetGemfile when it equals gemfilePath, preventing a self-loop in the build file tree.

- skip targets equal to gemfilePath before dedup to avoid a self-loop in the build file tree

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f2bc275e74

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ArtifactDetail: models.ArtifactDetail{
Filename: gemfilePath,
Ecosystem: models.EcosystemRubyGems,
Name: findAdjacentGemspecName(gemfileDir),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid matching gemspec self packages to Gemfile artifacts

When a gem project uses the standard gemspec form in its Gemfile, the lockfile contains the project gem itself in a PATH section while that package's BlockLocation remains Gemfile.lock because the Gemfile/gemspec matchers only enrich declared dependencies. Populating artifact.Name here makes findArtifact match that self package, so createFileComponents adds a dependency with Ref = Gemfile.lock pointing to this Gemfile, but addFileDependencies only emits the Gemfile file component. The generated CycloneDX SBOM therefore has a dangling lockfile→Gemfile dependency for ordinary gem projects; either avoid matching the current artifact to the self package or emit the self package location/component consistently.

Useful? React with 👍 / 👎.

anderruiz added 2 commits July 7, 2026 11:11
…fact matching

A standard gem project (gemspec form in Gemfile) caused a dangling
Gemfile.lock→Gemfile dependency in the CycloneDX SBOM. The self-gem
appears in the PATH section of Gemfile.lock with BlockLocation=Gemfile.lock;
because artifact.Name was set to the gemspec name, findArtifact matched it
and createFileComponents emitted a Ref=Gemfile.lock dependency that had no
corresponding file component.

Fix: add ArtifactDetail.PackageName, used exclusively for the
datadog:maven-package SBOM property (and thus ctx.ArtifactIDs /
BuildFileRelations.ID). Ruby's GetArtifact now sets PackageName instead of
Name, so findArtifact never matches the self-gem. Gradle and other
ecosystems that need cross-project findArtifact matching continue to use
Name unchanged.
@anderruiz
anderruiz merged commit d836eb6 into main Jul 7, 2026
11 checks passed
@anderruiz
anderruiz deleted the ander/rubygems-build-file-trees branch July 7, 2026 09:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 86ce5e29ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

gemfile, err := extractor.OpenLocalDepFile(gemfilePath)
if err != nil {
// No adjacent Gemfile — no build file tree to emit
return nil, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Suppress fallback artifacts when Gemfile is missing

When artifact extraction scans a Ruby directory that has Gemfile.lock but no adjacent Gemfile, this nil return does not actually omit the artifact: ExtractDeps synthesizes a bare ScannedArtifact{Filename: f.Path()} whenever an ArtifactExtractor returns nil. Fresh evidence after the prior fix is that fallback path, so lockfile-only Ruby scans still emit a Gemfile.lock file component, and unfiltered GetBuildFileTrees can report the lockfile as a manifest build file instead of omitting Ruby build-file relationships for that directory.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants