Skip to content

refactor: Migrate to new OSV bindings#2328

Merged
cuixq merged 20 commits intogoogle:mainfrom
cuixq:test
Nov 13, 2025
Merged

refactor: Migrate to new OSV bindings#2328
cuixq merged 20 commits intogoogle:mainfrom
cuixq:test

Conversation

@cuixq
Copy link
Contributor

@cuixq cuixq commented Nov 6, 2025

Fixes #2314 and unblocks #2286

This PR migrates to the new OSV bindings osv.dev/bindings/go and github.com/ossf/osv-schema/bindings/go:

  • Ecosystem Constants: Updated references to the new osvconstants.Ecosystem to improve separation of concerns.
  • API Updates: Adapted the code to breaking changes in the osv.dev/bindings/go/api for batch queries and version determination.
    • Protobuf Serialization: Implemented custom MarshalJSON and UnmarshalJSON methods with protojson to correctly handle the serialization of osvschema.Vulnerability.
  • Binding Migration: Replaced direct field access (e.g., v.OSV.ID) with getter methods (e.g., v.OSV.GetId()).
  • Testing: Modified test fixtures, snapshots, and helper functions to accommodate the new binding structure, including using protocmp.Transform for comparisons.

@cuixq
Copy link
Contributor Author

cuixq commented Nov 7, 2025

/gemini summary
/gemini review

@gemini-code-assist
Copy link
Contributor

Thank you for the request! I've reviewed the pull request and generated a summary below.

Pull Request Summary

This pull request primarily focuses on migrating the codebase to use new bindings for the osv-schema library. The changes involve updating how vulnerability data is accessed and serialized, aligning with a protobuf-based approach. Key updates include:

  • Binding Migration: Replaced direct field access (e.g., v.OSV.ID) with getter methods (e.g., v.OSV.GetId()) across various files (cmd/osv-scanner/fix/noninteractive.go, cmd/osv-scanner/fix/state-choose-strategy.go, internal/clients/clientimpl/localmatcher/zip.go, etc.).
  • Ecosystem Constants: Updated references from osvschema.Ecosystem to osvconstants.Ecosystem for better separation of concerns (cmd/osv-scanner/fix/output.go, internal/depsdev/depsdev.go).
  • Protobuf Serialization: Introduced google.golang.org/protobuf/encoding/protojson for marshaling and unmarshaling osvschema.Vulnerability objects, particularly in internal/output/result.go and pkg/models/results.go, to correctly handle the new protobuf structure.
  • API Updates: Adapted to changes in the osv.dev/bindings/go/api for batch queries and version determination (internal/clients/clientimpl/osvmatcher/cachedosvmatcher.go, internal/scalibrextract/filesystem/vendored/vendored.go).
  • Dependency Updates: Updated go.mod and go.sum to reflect newer versions of osv-scalibr, osv-schema/bindings/go, and osv.dev/bindings/go, along with other indirect dependencies.
  • Test Infrastructure: Modified test fixtures and helper functions to accommodate the new binding structure, including the introduction of separate JSON files for vulnerability data in remediation tests and using protocmp.Transform for comparisons.

These changes are essential for keeping the project aligned with the latest osv-schema bindings, improving code consistency, and leveraging the benefits of protobufs for data handling.

Review

Overall, the changes in this pull request are a necessary and well-executed migration to the new osv-schema bindings. The adoption of getter methods for accessing protobuf fields, the use of osvconstants for ecosystem definitions, and the integration of protojson for serialization are all standard and beneficial practices when working with protobufs in Go. The corresponding updates to test files and snapshots demonstrate a thorough approach to ensuring correctness during this refactoring.

Maintainability

  • The consistent use of getter methods (GetId(), GetSummary(), etc.) improves code readability and makes the code more robust against future changes in the underlying osvschema struct definitions. This is a significant improvement for maintainability.
  • The separation of ecosystem constants into osvconstants is a good architectural decision, preventing potential circular dependencies and making the code easier to reason about.

Correctness

  • The implementation of MarshalJSON and UnmarshalJSON for groupedSARIFFinding, VulnerabilityFlattened, and PackageVulns using protojson is crucial for correctly handling the protobuf-generated osvschema.Vulnerability objects. This ensures that the JSON output and input remain valid and consistent with the new bindings.
  • The addition of defer file.Close() in internal/ci/utility.go is a good practice for resource management, preventing potential file handle leaks.
  • The various if v == nil and if affected.GetPackage() == nil checks added in internal/output/sarif.go and internal/utility/vulns/vulnerability.go enhance the robustness of the code by preventing panics when dealing with potentially nil protobuf fields.

Efficiency

  • The changes do not appear to introduce any significant performance bottlenecks. The overhead of getter methods is generally negligible, and the serialization/deserialization changes are necessary for correctness rather than performance optimization.

Testing

  • The comprehensive updates to test files, including the use of protocmp.Transform() for cmp.Diff and the introduction of separate vulns.json files for test data, indicate a strong commitment to maintaining test coverage and accuracy with the new bindings.

No critical or high-severity issues were found. The changes are primarily refactoring and dependency updates, which are well-handled.

@cuixq cuixq changed the title refactor: Migrate to the new bindings refactor: Migrate to new OS bindings Nov 7, 2025
@cuixq cuixq changed the title refactor: Migrate to new OS bindings refactor: Migrate to new OSV bindings Nov 7, 2025
@cuixq
Copy link
Contributor Author

cuixq commented Nov 7, 2025

protojson output being unstable is so annoying :(

@codecov-commenter
Copy link

codecov-commenter commented Nov 12, 2025

Codecov Report

❌ Patch coverage is 69.00685% with 181 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.74%. Comparing base (48b379f) to head (cdc0dd0).

Files with missing lines Patch % Lines
scripts/generate_mock_resolution_universe/main.go 0.00% 53 Missing ⚠️
pkg/models/results.go 53.94% 28 Missing and 7 partials ⚠️
...al/resolution/clienttest/mock_resolution_client.go 56.00% 6 Missing and 5 partials ⚠️
internal/output/result.go 79.16% 5 Missing and 5 partials ⚠️
internal/utility/vulns/vulnerability.go 85.71% 5 Missing and 4 partials ⚠️
internal/output/sarif.go 66.66% 4 Missing and 4 partials ⚠️
internal/sourceanalysis/rust.go 0.00% 8 Missing ⚠️
internal/tui/vuln-list.go 0.00% 8 Missing ⚠️
internal/tui/vuln-info.go 0.00% 6 Missing ⚠️
internal/output/sbom/cyclonedx_common.go 89.47% 2 Missing and 2 partials ⚠️
... and 16 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2328      +/-   ##
==========================================
- Coverage   68.03%   67.74%   -0.29%     
==========================================
  Files         172      172              
  Lines       12835    13057     +222     
==========================================
+ Hits         8732     8846     +114     
- Misses       3434     3517      +83     
- Partials      669      694      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cuixq cuixq marked this pull request as ready for review November 12, 2025 04:30
Copy link
Collaborator

@another-rex another-rex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't go through every file, but spot checking looks good to me.

@another-rex
Copy link
Collaborator

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the codebase to the new OSV bindings, which is a significant and well-executed refactoring. The changes are consistent across the project, and the use of protojson for custom marshalling of osvschema.Vulnerability is a good approach. The updates to test data and helpers, including the use of protocmp, are also well done.

I've found a few potential issues, mostly related to safe access of nested fields in the new protobuf structures, which could lead to panics. I've also identified a pre-existing bug that could be fixed as part of this refactoring. My comments provide specific suggestions for these points.

@cuixq
Copy link
Contributor Author

cuixq commented Nov 13, 2025

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a large refactoring to migrate to the new OSV bindings. The changes are extensive, touching many parts of the codebase to adapt to the new protobuf-based structures. The migration appears to be handled well, with updates to use getter methods, new constant types, and custom JSON marshaling for protobuf messages. I've found one potential bug in the logic for handling fixed versions that I've commented on. Overall, this is a solid effort to modernize the dependency bindings.

@cuixq
Copy link
Contributor Author

cuixq commented Nov 13, 2025

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a significant refactoring to migrate the codebase to new OSV bindings. The changes are extensive but well-executed, consistently applying the new API across many files. Key changes include using getter methods for field access, adopting new enum types, handling pointer types for vulnerability data, and implementing custom JSON marshalling for protobuf messages. The test suite has been updated accordingly, including the use of protocmp for comparisons and refactoring test data. I've identified a minor improvement opportunity in a test data generation script to enhance readability, but overall, this is a solid and necessary migration.

@cuixq cuixq merged commit 82836fa into google:main Nov 13, 2025
16 checks passed
@cuixq cuixq deleted the test branch November 13, 2025 02:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants