feat(internal/librarian): only generate libraries with changed APIs by jskeet · Pull Request #2618 · googleapis/librarian

jskeet · 2025-10-20T09:52:18Z

The new -generate-unchanged flag can be used to generate all (unblocked) libraries. Libraries which are blocked by configuration are still not generated when -generate-unchanged is specified.

Specifying -library ignores this check (and blocking) and will always generate the specified library.

Fixes #818

The new -generate-unchanged flag can be used to generate all (unblocked) libraries. Libraries which are blocked by configuration are still not generated when -generate-unchanged is specified. Specifying -library ignores this check (and blocking) and will always generate the specified library. Fixes googleapis#818

gemini-code-assist · 2025-10-20T09:52:39Z

Summary of Changes

Hello @jskeet, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the librarian tool by introducing a feature that allows for more intelligent and efficient library generation. Instead of regenerating all libraries every time, librarian can now detect if a library's underlying APIs have changed and only generate those that require updates. This change includes a new command-line flag to override this behavior and force full generation when needed, along with robust internal mechanisms for tracking API changes via Git path hashing.

Highlights

Conditional Library Generation: Introduced a new mechanism to generate libraries only if their associated APIs have changed since the last generation. This significantly improves efficiency by avoiding unnecessary regeneration.
New generate-unchanged Flag: Added a -generate-unchanged flag to the librarian command. When set to true, this flag overrides the API change detection and forces the generation of all unblocked libraries, providing flexibility for specific use cases.
Git Repository Path Hashing: Implemented a new GetHashForPathOrEmpty method in the gitrepo.Repository interface and LocalRepository implementation. This function efficiently retrieves the tree hash for a given path at a specific commit, enabling precise detection of API changes.
Enhanced Generation Logic: The generateRunner now incorporates a shouldGenerate function that encapsulates the logic for determining whether a library needs to be generated. This logic considers GenerateBlocked configuration, API presence, the generate-unchanged flag, and API path changes.
Comprehensive Testing: Added extensive unit tests for the new GetHashForPathOrEmpty and shouldGenerate functions, covering various scenarios including existing/non-existent paths, different commit states, and error handling.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a significant optimization by only generating libraries when their underlying APIs have changed. This is controlled by a new -generate-unchanged flag. The implementation adds a new GetHashForPathOrEmpty function to the git repository interface to detect changes between commits, which is a robust approach. The changes are well-tested, including new unit tests for the git function and the generation logic.

My review has identified one critical issue regarding an unchecked error that could lead to incorrect behavior, and a couple of medium-severity suggestions to improve test code clarity and diagnostics. Overall, this is a great feature addition.

internal/librarian/generate_command.go

internal/gitrepo/gitrepo_test.go

internal/librarian/mocks_test.go

codecov · 2025-10-20T10:15:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.66%. Comparing base (739ce0d) to head (46651e7).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2618      +/-   ##
==========================================
+ Coverage   85.65%   85.66%   +0.01%     
==========================================
  Files         108      108              
  Lines       10985    11046      +61     
==========================================
+ Hits         9409     9463      +54     
- Misses       1244     1250       +6     
- Partials      332      333       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

internal/librarian/generate_command.go

jskeet

(Leaving comments from the group review.)

internal/librarian/generate_command.go

internal/gitrepo/gitrepo.go

internal/librarian/generate_command.go

internal/librarian/generate_command_test.go

internal/librarian/generate_command.go

jskeet · 2025-10-21T07:18:29Z

@codyoss I think I've addressed your comments, but I may have misinterpreted them. PTAL.

Librarian Version: v0.0.0-20251022154542-dd249212325d Language Image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/librarian-release-container:latest <details><summary>librarian: 0.5.0</summary> ## [0.5.0](v0.4.0...v0.5.0) (2025-10-22) ### Features * speed up finding changed files in commits (#2645) ([1060946](10609465)) * Can config dart `export` (#2641) ([97eacd2](97eacd29)) * minor whitespace and doc changes to the Dart templates (#2636) ([b3ac7b4](b3ac7b42)) * only generate libraries with changed APIs (#2618) ([82171be](82171bed)) * make extra modules public (#2622) ([2c94a53](2c94a53f)) * skip a GitHub release for a library (#2612) ([6258f4d](6258f4d1)) * allow skipping semver checks for rust-publish (#2584) ([739ce0d](739ce0d5)) * add ability to open pull request as a draft (#2604) ([c1f0285](c1f02859)) * capture discovery revision (#2605) ([14a1483](14a14830)) * Add conditional instrumentation to gRPC clients (#2594) ([3cc63b2](3cc63b22)) * add update-image CLI command (#2580) ([90e0f6e](90e0f6e5)) * Generate more samples for oneof main setters. (#2592) ([c55f3ce](c55f3ceb)) * Add documentation for generated service constructors (#2575) ([6a4aead](6a4aeade)) * disable some clippy warnings (#2567) ([9f51084](9f510842)) * Generate setter samples for oneof fields. (#2573) ([8c2416a](8c2416ab)) * add default Rust features option (#2562) ([892f42b](892f42b7)) * Include a correct URL for issues (#2570) ([10493ed](10493ed9)) * add ability to find the latest docker image SHA (#2539) ([62e80f1](62e80f19)) * add ability to checkout a repo at a certain commit (#2555) ([23b8ffe](23b8ffea)) * add onboarding PR body (#2552) ([e32719c](e32719cd)) ### Bug Fixes * associate bulk change to individual libraries (#2626) ([dd24921](dd249212)) * Allow `unnecessary_import`s (#2642) ([88c4379](88c43794)) * resolve issue where onboarded library can't be released (#2632) ([b300a4e](b300a4ea)) * resolve issue where commits cannot be fetched for new library (#2631) ([45652c0](45652c03)) * address a typo in the Message.ServicePlaceholder docs (#2616) ([82fda96](82fda96b)) * bad version bump edits (#2613) ([9902b1d](9902b1d5)) * use templates for update-image PR body (#2602) ([7309cad](7309cadd)) * populate configure command pr content (#2591) ([811eb8e](811eb8e2)) * shrink release PR size when there are bulk changes (#2585) ([bcb914a](bcb914ac)) * Fixes several issues with oneof main setter samples (#2589) ([e4958d0](e4958d00)) * resolve broken link in PR body (#2579) ([098c1d2](098c1d24)) * Remove double sample code blocks (#2582) ([6b10456](6b104567)) * change commit package (#2571) ([45ee48f](45ee48f0)) * show chores in release notes (#2544) ([88b62cc](88b62ccb)) * avoid work duplication when finding changes (#2558) ([0adeeac](0adeeac6)) * mangled method names and doc links (#2565) ([895dac9](895dac94)) ### Reverts * show chores in release notes (#2601) ([7e6740f](7e6740ff)) </details>

jskeet requested a review from a team as a code owner October 20, 2025 09:52

gemini-code-assist bot reviewed Oct 20, 2025

View reviewed changes

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

internal/gitrepo/gitrepo_test.go Outdated Show resolved Hide resolved

internal/librarian/mocks_test.go Outdated Show resolved Hide resolved

jskeet added 2 commits October 20, 2025 10:06

chore: address review feedback

d5619fe

chore: fix reported hash on failure

a6dcb5a

chore: add test for run() observing shouldGenerate failure

70bb500

jskeet commented Oct 20, 2025

View reviewed changes

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

jskeet commented Oct 20, 2025

View reviewed changes

codyoss reviewed Oct 20, 2025

View reviewed changes

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

codyoss reviewed Oct 20, 2025

View reviewed changes

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

codyoss reviewed Oct 20, 2025

View reviewed changes

internal/gitrepo/gitrepo.go Outdated Show resolved Hide resolved

miguelvelezsa reviewed Oct 21, 2025

View reviewed changes

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

internal/librarian/generate_command.go Show resolved Hide resolved

internal/librarian/generate_command_test.go Show resolved Hide resolved

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

jskeet commented Oct 21, 2025

View reviewed changes

internal/librarian/generate_command.go Outdated Show resolved Hide resolved

chore: address review comments

46651e7

codyoss approved these changes Oct 21, 2025

View reviewed changes

jskeet merged commit 82171be into googleapis:main Oct 21, 2025
6 checks passed

jskeet deleted the only-generate-changed branch October 21, 2025 17:06

cloud-sdk-librarian-robot mentioned this pull request Oct 22, 2025

chore: librarian release pull request: 20251022T180958Z #2653

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(internal/librarian): only generate libraries with changed APIs#2618

feat(internal/librarian): only generate libraries with changed APIs#2618
jskeet merged 5 commits intogoogleapis:mainfrom
jskeet:only-generate-changed

jskeet commented Oct 20, 2025

Uh oh!

gemini-code-assist bot commented Oct 20, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Oct 20, 2025 •

edited

Loading

Uh oh!

Uh oh!

jskeet left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jskeet commented Oct 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jskeet commented Oct 20, 2025

Uh oh!

gemini-code-assist bot commented Oct 20, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

jskeet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jskeet commented Oct 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Oct 20, 2025 •

edited

Loading