Skip to content

chore(internal/librariangen): add .repo-metadata.json generation#13290

Merged
quartzmo merged 12 commits intogoogleapis:mainfrom
quartzmo:librariangen-repo-metadata-json
Nov 7, 2025
Merged

chore(internal/librariangen): add .repo-metadata.json generation#13290
quartzmo merged 12 commits intogoogleapis:mainfrom
quartzmo:librariangen-repo-metadata-json

Conversation

@quartzmo
Copy link
Copy Markdown
Member

@quartzmo quartzmo commented Nov 3, 2025

@quartzmo quartzmo requested a review from a team November 3, 2025 17:31
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @quartzmo, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the librariangen tool by adding the capability to generate .repo-metadata.json files for client libraries. This automation streamlines the process of providing essential metadata for each API, which is vital for documentation, discovery, and integration within the broader Google Cloud ecosystem.

Highlights

  • Repo Metadata Generation: The librariangen tool now automatically generates .repo-metadata.json files for each client library API, streamlining metadata provision.
  • New Metadata Logic: A new manifest.go file has been added, containing the core logic for collecting and structuring metadata, including API shortname, documentation URL, and release level determination.
  • Integration with Protoc: The metadata generation process is integrated into the invokeProtoc function, ensuring that .repo-metadata.json files are created immediately after client library generation.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@quartzmo quartzmo changed the title Librariangen repo metadata json chore(internal/librariangen): add .repo-metadata.json generation Nov 3, 2025
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the functionality to generate .repo-metadata.json files for each generated API client. The implementation is well-structured within a new manifest.go file and includes a comprehensive set of tests. My review includes several suggestions to enhance the correctness, robustness, and maintainability of the new code, primarily focusing on path calculations, function signatures, and removing dead or brittle code.

@quartzmo
Copy link
Copy Markdown
Member Author

quartzmo commented Nov 3, 2025

Local integration testing produces NO DIFF versus latest version of .repo-metadata.json.

➜  librariangen git:(librariangen-repo-metadata-json) ✗ ./run-binary-generate-test.sh
Cleaning up from last time: rm -f librariangen.log
Cleaning up from last time...
Using temporary directory: /var/folders/zk/_5rh6nq91db_fkzrcrh5v0w400xrml/T/tmp.XXXXXXXXXX.9N2DwvyrDf
Using googleapis source from /Users/chrisdsmith/oss/googleapis
Cleaning up from last time: rm -f ./librariangen
Compiling librariangen...
Running librariangen...
Verifying output...
Librariangen logs are available in: librariangen.log
Using cached google-cloud-go from /Users/chrisdsmith/oss/google-cloud-go-2
Verifying output by comparing with the goldens repository...
The script will modify files in your local goldens clone.

Verification complete. The status in librariangen.log shows the difference between the
expected generated output (goldens) and the current modified state of your goldens repository (librariangen).

To reset your goldens repository:
  cd /Users/chrisdsmith/oss/google-cloud-go-2
  git reset --hard HEAD && git clean -fd
Binary integration test passed successfully.
Generated files are available for inspection in: /var/folders/zk/_5rh6nq91db_fkzrcrh5v0w400xrml/T/tmp.XXXXXXXXXX.9N2DwvyrDf/output
➜  librariangen git:(librariangen-repo-metadata-json) ✗ cat librariangen.log
--- Tool Versions ---
Go: go version go1.25.1 darwin/arm64
protoc: libprotoc 25.7
protoc-gen-go: protoc-gen-go v1.35.2
protoc-gen-go-grpc: protoc-gen-go-grpc 1.3.0
protoc-gen-go_gapic: v0.53.1
---------------------
time=2025-11-03T10:29:36.286-07:00 level=INFO msg="processing api" service_dir=/Users/chrisdsmith/oss/googleapis/google/cloud/chronicle/v1
time=2025-11-03T10:29:36.454-07:00 level=INFO msg="librariangen: updating snippet metadata file" path=internal/generated/snippets/chronicle/apiv1/snippet_metadata.google.cloud.chronicle.v1.json
time=2025-11-03T10:29:36.454-07:00 level=INFO msg="librariangen: updating version in snippets metadata file" destPath=internal/generated/snippets/chronicle/apiv1/snippet_metadata.google.cloud.chronicle.v1.json old=$VERSION new=0.1.1
time=2025-11-03T10:29:36.454-07:00 level=INFO msg="librariangen: running goimports" directory=/var/folders/zk/_5rh6nq91db_fkzrcrh5v0w400xrml/T/tmp.XXXXXXXXXX.9N2DwvyrDf/output
--- Git Reset Summary ---
HEAD is now at d19f797b33 feat(spansql): support subquery in View Join (#13266)
--- Git Status Summary ---
On branch main
Your branch is up to date with 'upstream/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	deleted:    chronicle/CHANGES.md
	deleted:    chronicle/README.md
	deleted:    chronicle/apiv1/version.go
	deleted:    chronicle/go.mod
	deleted:    chronicle/go.sum
	deleted:    chronicle/internal/version.go

Copy link
Copy Markdown
Contributor

@jskeet jskeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested this by creating a local Docker image and running librarian update-image? If you've done that and there are no diffs, that sounds good to me.

I think we'll need to update the state file though to say that the .repo-metadata.json file should be deleted (in the remove regex) and change the configure command to add the appropriate removal as well. Otherwise, now that googleapis/librarian#2593 is fixed, I'd expect generation to fail with this.

@quartzmo
Copy link
Copy Markdown
Member Author

quartzmo commented Nov 4, 2025

I think we'll need to update the state file though to say that the .repo-metadata.json file should be deleted (in the remove regex) and change the configure command to add the appropriate removal as well.

Of course you're right that now that we re-generate .repo-metadata.json, we need to add it to remove_regex. I had this in my mental list originally (should have been an actual list), but forgot. Thanks for the catch.

jskeet
jskeet previously approved these changes Nov 6, 2025
Copy link
Copy Markdown
Contributor

@jskeet jskeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, but we'll need to coordinate on submitting this and updating the image, to avoid issues with generation.

bhshkh
bhshkh previously approved these changes Nov 6, 2025
@quartzmo quartzmo dismissed stale reviews from bhshkh and jskeet via 37b6c4d November 6, 2025 17:38
@quartzmo quartzmo requested review from a team and daniel-sanche November 6, 2025 23:23
@quartzmo
Copy link
Copy Markdown
Member Author

quartzmo commented Nov 6, 2025

Because of the commits in this PR that fix issues in existing .repo-metadata.json files, running librarian update-image against this branch (with commits up to
d89960e) produces zero diff for existing .repo-metadata.json files, and adds three .repo-metadata.json files that were known to be missing from recently added new clients.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   .librarian/state.yaml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	capacityplanner/apiv1beta/.repo-metadata.json
	gkerecommender/apiv1/.repo-metadata.json
	saasplatform/saasservicemgmt/apiv1beta1/.repo-metadata.json

This PR is ready to merge, and as soon as the corresponding new image is ready (roughly 10 minutes), merging it must be followed with librarian update-image to update state.yaml in main branch.

}

importPath := "cloud.google.com/go/foo/apiv1"
if tt.name == "import_path_alpha" {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just make this another field in the test struct?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest for reasons of pragmatism, I do this in a follow-up PR.

Copy link
Copy Markdown
Contributor

@jskeet jskeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's merge and then update once the new image is built.

@quartzmo quartzmo merged commit 86f6198 into googleapis:main Nov 7, 2025
13 checks passed
@quartzmo quartzmo deleted the librariangen-repo-metadata-json branch November 7, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

go: support .repo-metadata-full.json or per-library .repo-metadata.json

3 participants