[K9VULN-9602] Remove hard requirement for packages.lock.json when scanning .csproj files by piloulacdog · Pull Request #84 · DataDog/datadog-sbom-generator

piloulacdog · 2025-11-24T16:36:58Z

🚀 Motivation

Enable the SBOM generator to scan .NET repositories that only contain *.csproj files without requiring packages.lock.json, allowing us to start reporting libraries in repositories with incomplete dependency information.

📝 Summary

This PR implements Phase 1 of the RFC to remove the hard requirement for packages.lock.json when scanning .NET projects:

Added parser-nuget-csproj: Directly extracts NuGet dependencies from PackageReference elements in .csproj files
Updated matcher-nuget-csproj: reuse logic from parser-nuget-csproj

The implementation prioritizes maintaining full backward compatibility with existing .NET repositories that use packages.lock.json while enabling basic scanning for .csproj-only repositories.

🧪 Testing

New tests were added for new logic.
Existing tests were updated for new logic, and not only so that they pass!
Benchmark results prove that performance is the same or better.

🚧 Staging validation

Deployed and monitored using Datadog dashboards.
Proof that it works as expected, including profiling or UX screenshots.

🆘 Recovery

Notes for on-call - select only one:

The change can be rolled back.
Do not roll back. Why?:

📚 Documentation

Document	Link or Detail
RFC	RFC
Incident	N/A
Jira Ticket	K9VULN-9602

… which is followed by other attributes

…ger used

github-actions · 2025-11-24T17:21:44Z

Go test coverage report

Total test coverage: 91.0% (4078/4480)

Test coverage has changed in the current files, with 6 lines missing coverage.
file: uncovered: current coverage: base coverage:
pkg/lockfile/match-nuget-csproj.go 2 91.3% (21/23) 91.2% (62/68)
pkg/lockfile/parse-nuget-csproj.go 4 95.3% (82/86) /

piloulacdog · 2025-11-24T18:19:00Z

internal/semantic/parse_test.go

 func TestParse(t *testing.T) {
 	t.Parallel()

-	ecosystems := lockfile.KnownEcosystems()


I deleted this method as it was only existing in tests, and moved it here

piloulacdog · 2025-11-24T18:19:09Z

pkg/lockfile/ecosystems.go

@@ -1,25 +0,0 @@
-package lockfile


this was only used in tests

piloulacdog · 2025-11-24T18:20:55Z

pkg/lockfile/fixtures/nuget/project.csproj


 	<ItemGroup>
 		<PackageReference Include="Downloader" Version="3.*" />
-		<PackageReference Include="MaterialDesignThemes" Version="5.1.0" />


I modified this fixture because I realized that we were previously wrongly reporting the location for versions in csproj.
For this given example, we would have been reporting the location of: version == "5.1.0" PrivateAssets="All"

and not version == "5.1.0"

piloulacdog · 2025-11-24T18:25:41Z

pkg/lockfile/match-nuget-csproj.go

-	models.FilePosition
-}
-
-func (itemGroup *ItemGroup) UnmarshalXML(decoder *xml.Decoder, start xml.StartElement) error {


those methods and the shared logic were moved to the parse-nuget-csproj.go

piloulacdog · 2025-11-24T18:25:54Z

pkg/lockfile/match-nuget-csproj.go

-	content, err := io.ReadAll(sourcefile)
+func (m NugetCsprojMatcher) Match(sourceFile DepFile, packages []PackageDetails) error {
+	// Extract all package details from the csproj file using the extractor
+	extractedPackages, err := NuGetCsprojExtractor{}.Extract(sourceFile)


we are reusing the logic from parse-nuget-csproj.go

piloulacdog · 2025-11-24T18:27:40Z

pkg/lockfile/parse-nuget-csproj.go

+
+// extractPackageVersionLocation extracts the file position of a package version from a block of lines
+func extractPackageVersionLocation(block []string, lineStart int, filename string) *models.FilePosition {
+	versionLocation := fileposition.ExtractDelimitedRegexpPositionInBlock(block, "[^\"]*", lineStart, "Version=\"", "\"")


the fix of the version location reported is here. It used to be:

versionLocation := fileposition.ExtractDelimitedRegexpPositionInBlock(block, ".*", lineStart, "Version=\"", "\"")

and now it is:

versionLocation := fileposition.ExtractDelimitedRegexpPositionInBlock(block, "[^\"]*", lineStart, "Version=\"", "\"")

but, because .* is greedy, it would match too much

piloulacdog · 2025-11-24T18:28:27Z

pkg/lockfile/parse.go

@@ -1,151 +0,0 @@
-package lockfile


this file declared methods which weren't used. So dropped it and I had to update the references in tests for it.

piloulacdog · 2025-11-25T14:23:43Z

pkg/lockfile/extract_test.go

 	t.Parallel()

 	lockfiles := []string{
+		"*.csproj",


rename csproj

piloulacdog · 2025-11-25T14:36:02Z

pkg/models/lockfile.go

 	GradleLockfile            = "gradle.lockfile"
 	GradleVerificationXML     = "gradle/verification-metadata.xml"
 	MixLock                   = "mix.lock"
+	NugetCsProjFile           = "*.csproj"


marcwieserdev

Thanks for your live review and for the local compare tool ! 👏

piloulacdog added 6 commits November 24, 2025 12:00

tests: create fixtures for csproj without lockfile

73784e9

tests: make sure we have test in match-nuget-csproj.go for a versions…

a5967ef

… which is followed by other attributes

tests: add tests for cmd with csproj support

ab200db

feat: remove requirement for package.lock.json for csproj

5c29b89

tests: fix extract_test.go with the addition of *.csproj parser

27fc3fe

tests: fix tests by dropping unecessary parser.go files - were no lon…

99f9035

…ger used

piloulacdog commented Nov 24, 2025

View reviewed changes

pkg/lockfile/ecosystems.go

@@ -1,25 +0,0 @@

package lockfile

Copy link
Copy Markdown

Contributor Author

piloulacdog Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was only used in tests

piloulacdog commented Nov 24, 2025

View reviewed changes

piloulacdog marked this pull request as ready for review November 24, 2025 18:29

piloulacdog requested a review from a team as a code owner November 24, 2025 18:29

merge main

2fc29b0

piloulacdog mentioned this pull request Nov 25, 2025

[K9VULN-9603] Add property substitution support for NuGet .csproj files #85

Merged

7 tasks

piloulacdog commented Nov 25, 2025

View reviewed changes

pkg/lockfile/extract_test.go Outdated

t.Parallel()

lockfiles := []string{

"*.csproj",

Copy link
Copy Markdown

Contributor Author

piloulacdog Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename csproj

piloulacdog commented Nov 25, 2025

View reviewed changes

marcwieserdev previously approved these changes Nov 25, 2025

View reviewed changes

refactor: rename *.csproj parser to csproj for simplicity

9b3e546

piloulacdog dismissed marcwieserdev’s stale review via 9b3e546 November 25, 2025 15:50

merge: merge main and address git conflict

34c6a57

piloulacdog force-pushed the pierrelouis.lacorte/K9VULN-9602-csproj-do-not-require-lockfile branch from 68137fc to 34c6a57 Compare November 25, 2025 16:09

marcwieserdev approved these changes Nov 25, 2025

View reviewed changes

piloulacdog merged commit 6eeddc6 into main Nov 25, 2025
10 checks passed

piloulacdog deleted the pierrelouis.lacorte/K9VULN-9602-csproj-do-not-require-lockfile branch November 25, 2025 16:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[K9VULN-9602] Remove hard requirement for packages.lock.json when scanning .csproj files#84

[K9VULN-9602] Remove hard requirement for packages.lock.json when scanning .csproj files#84
piloulacdog merged 9 commits intomainfrom
pierrelouis.lacorte/K9VULN-9602-csproj-do-not-require-lockfile

piloulacdog commented Nov 24, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 24, 2025 •

edited

Loading

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 24, 2025

Uh oh!

piloulacdog Nov 25, 2025

Uh oh!

piloulacdog Nov 25, 2025

Uh oh!

marcwieserdev left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

piloulacdog commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Motivation

📝 Summary

🧪 Testing

🚧 Staging validation

🆘 Recovery

📚 Documentation

Uh oh!

github-actions bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Go test coverage report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcwieserdev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

piloulacdog commented Nov 24, 2025 •

edited

Loading

github-actions bot commented Nov 24, 2025 •

edited

Loading