Skip to content

feat(internal/librarian): allow language-specific parallelism#4005

Merged
jskeet merged 2 commits intogoogleapis:mainfrom
jskeet:parallel-generate
Feb 19, 2026
Merged

feat(internal/librarian): allow language-specific parallelism#4005
jskeet merged 2 commits intogoogleapis:mainfrom
jskeet:parallel-generate

Conversation

@jskeet
Copy link
Copy Markdown
Contributor

@jskeet jskeet commented Feb 11, 2026

Refactors generation such that each phase of clean/generate/format is treated as a whole across all libraries, moving towards language-specific determination of when to perform operations in parallel within each phase.

This commit only delegates that choice for generation, but the same pattern can be followed for cleaning and formatting when there's a desire to do so.

Fixes #3933

@jskeet jskeet requested a review from julieqiu February 11, 2026 12:18
@jskeet
Copy link
Copy Markdown
Contributor Author

jskeet commented Feb 11, 2026

@julieqiu This is a draft PR as I haven't added tests yet, but I wanted to get feedback before going further. This is basically a replacement for #3934.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the library generation process, separating the clean, generate, and format phases, and delegating parallel execution to language-specific implementations, which improves modularity and flexibility. However, a critical security vulnerability exists due to the lack of validation for the Output path in the library configuration. This unvalidated path is used in dangerous filesystem operations, such as arbitrary file deletion and creation, which could lead to the compromise of sensitive system files if a malicious configuration is used. Furthermore, critical issues related to goroutine safety in the new parallel generation logic for Dart and Rust have been identified, potentially causing race conditions.

@codecov
Copy link
Copy Markdown

codecov bot commented Feb 11, 2026

Codecov Report

❌ Patch coverage is 28.76712% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.19%. Comparing base (fa4618f) to head (4884631).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/librarian/generate.go 26.31% 38 Missing and 4 partials ⚠️
internal/librarian/golang/generate.go 0.00% 6 Missing ⚠️
internal/librarian/fake.go 60.00% 1 Missing and 1 partial ⚠️
internal/librarian/golang/clean.go 60.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4005      +/-   ##
==========================================
- Coverage   82.30%   82.19%   -0.11%     
==========================================
  Files          81       81              
  Lines        6629     6633       +4     
==========================================
- Hits         5456     5452       -4     
- Misses        818      825       +7     
- Partials      355      356       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jskeet jskeet force-pushed the parallel-generate branch 7 times, most recently from 31f0c08 to e68c5c9 Compare February 18, 2026 17:47
@jskeet jskeet marked this pull request as ready for review February 18, 2026 18:00
@jskeet jskeet requested a review from a team as a code owner February 18, 2026 18:00
@jskeet
Copy link
Copy Markdown
Contributor Author

jskeet commented Feb 18, 2026

@JoeWang1127 This is now ready for review - it's only huge because I've added a new testdata API so that there are two APIs to generate for the tests :) (There's much more test code than production code.)

@jskeet jskeet force-pushed the parallel-generate branch 2 times, most recently from df346bf to 6c4cc15 Compare February 19, 2026 08:12
Refactors generation such that each phase of clean/generate/format is
treated as a whole across all libraries, moving towards
language-specific determination of when to perform operations in
parallel within each phase.

This commit only delegates that choice for generation, but the same
pattern can be followed for cleaning and formatting when there's a
desire to do so.

Fixes googleapis#3933
@jskeet jskeet requested review from JoeWang1127 and removed request for julieqiu February 19, 2026 08:25
JoeWang1127
JoeWang1127 previously approved these changes Feb 19, 2026
@jskeet jskeet merged commit 4df7577 into googleapis:main Feb 19, 2026
14 checks passed
@jskeet jskeet deleted the parallel-generate branch February 19, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

librarian: allow degrees of parallelization to be configured

2 participants