build(internal/librarianops/upgrade): add version config file updater#3926
build(internal/librarianops/upgrade): add version config file updater#3926miguelvelezsa wants to merge 12 commits intoupgrade-version-fetcherfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces new functionality to update the version in a librarian.yaml file, along with corresponding tests. The implementation is straightforward and the tests are well-written, covering the main success and failure scenarios. I have provided a suggestion to improve the code's clarity by correcting a function comment to accurately reflect its behavior.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: miguel <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## upgrade-version-fetcher #3926 +/- ##
===========================================================
+ Coverage 83.32% 83.35% +0.03%
===========================================================
Files 75 76 +1
Lines 6405 6417 +12
===========================================================
+ Hits 5337 5349 +12
Misses 697 697
Partials 371 371 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
|
||
| // UpdateLibrarianVersion updates the version field in the librarian.yaml config file with the provided version. | ||
| // If the file does not exist, returns an error. | ||
| func UpdateLibrarianVersion(version, configPath string) error { |
There was a problem hiding this comment.
I think you can just use repoPath as an input and remove GenerateLibrarianConfigPath and getConfigFile, since they seem very straightforward.
There was a problem hiding this comment.
librarian/internal/librarianops/generate.go
Line 222 in e20ee88
There was a problem hiding this comment.
librarian/internal/librarianops/generate.go
Line 222 in e20ee88
func updateLibrarianVersion(version, repoDir string) error {
already exists, so you can reuse that
If I use that function the upgrade.go will depend on generate.go. I prefer to have a separated file with the update logic so we can easily test it and modify it. Also that logic from generate.go will be deleted in a follow up PR.
There was a problem hiding this comment.
I think you can just use
repoPathas an input and removeGenerateLibrarianConfigPathandgetConfigFile, since they seem very straightforward.
I think is better to keep them. Even though the logic seems straightforward, now we have a few functions doing one thing, we can fully and easily test them and if something change we can directly go the function and change it.
|
|
||
| // UpdateLibrarianVersionInConfigFile updates the version field in the librarian.yaml config file with the provided | ||
| // version. If the file does not exist, returns an error. | ||
| func UpdateLibrarianVersionInConfigFile(version, configPath string) error { |
There was a problem hiding this comment.
This looks like a duplicate of
librarian/internal/librarianops/generate.go
Line 222 in 43b900a
|
@miguelvelezsa - I left a comment on https://github.com/googleapis/librarian/pull/3926/changes#r2790308453 and https://github.com/googleapis/librarian/pull/3928/changes#r2790347773. I don't think we need this PR right now. My recommendation would be to move forward without this change, and we can always reopen this later if it becomes necessary. |
…#2670) This PR moves Java Librarian container implementation WIP from `github.com/googleapis/sdk-platform-java/internal/librariangen` to `github.com/googleapis/librarian/internal/container/java` while preserving the git history. Here are the steps that were followed: ``` # 1. Clone the source repository git clone [email protected]:googleapis/sdk-platform-java.git cd sdk-platform-java # 2. Filter the repository to keep only the 'internal/librariangen' subdirectory git filter-repo --subdirectory-filter internal/librariangen/ --force # 3. Go back to the parent directory cd .. # 4. Clone the destination repository git clone [email protected]:googleapis/librarian.git cd librarian # 5. Create a new branch for the migration git checkout -b feat/migrate-librariangen-java main # 6. Add the filtered repository as a remote git remote add sdk-platform-java-librariangen ../sdk-platform-java git fetch sdk-platform-java-librariangen # 7. Merge the histories. This command creates a merge commit but keeps the content of the 'librarian' repository. git merge --allow-unrelated-histories --no-ff sdk-platform-java-librariangen/main -s ours --no-commit # 8. Read the tree from the filtered repository into the desired subdirectory git read-tree --prefix=internal/container/java/ -u sdk-platform-java-librariangen/main # 9. Commit the changes git commit -m "feat(internal/container/java): migrate librariangen from sdk-platform-java" ``` Since we are still in early stages of implementing the Java container, the plan is to squash-and-merge this PR. original commits: ``` commit 0a1bbea44d6a1b024c569db89fef8d08ad7bd2d2 Author: Tomo Suzuki <[email protected]> Date: Thu Oct 23 23:58:54 2025 -0400 chore(librariangen): Generate to use languagecontainer.Run (googleapis#3968) commit 452d703b703ab3222fd1a7060ed5e1ac6363322b Author: Mike Eltsufin <[email protected]> Date: Thu Oct 23 17:08:42 2025 -0400 feat(librariangen): generate grpc stubs and resource helpers (googleapis#3967) * Introduces the generation of gRPC stubs and resource helpers. * Modifies the `Generate` function to create output directories for GAPIC, gRPC, and proto files. * Updates the `invokeProtoc` function to pass an `OutputConfig` struct. * Updates the `restructureOutput` function to handle gRPC stubs and resource names. * Adds a `copyAndMerge` function to merge resource name files into the proto destination. * Updates the `cleanupIntermediateFiles` function to remove the GAPIC, gRPC, and proto directories. * Updates tests to reflect these changes. * Improvements in error handling. commit a26a6d937e0523b9a6c193504645f96649e48573 Author: Tomo Suzuki <[email protected]> Date: Thu Oct 23 10:15:24 2025 -0400 chore(librariangen): languagecontainer package to parse release-init request (googleapis#3965) Introducing languagecontainer package. * **New languagecontainer package**: A new `languagecontainer` package has been introduced to encapsulate language-specific container operations and command execution logic, promoting modularity. This package parses the request JSON file and calls the corresponding implementation method in each language container. The languagecontainer package itself should not have language-specific implementation. * **release-init command parsing**: The `languagecontainer.Run` function now includes robust logic to parse `release-init` requests from JSON files, handle command-line flags, and invoke the `ReleaseInit` function. * Why isn't this providing an interface? It's because if `LanguageContainer` is an interface, then there would be package name conflict of `languagecontainer/release` package (language agostic) and `release` package (language-specific. In this case Java-specific). [Here is a piece of code](googleapis#2516 (comment)). * **Main command dispatch refactor**: The `librariangen` `main.go` has been refactored to delegate non-`generate` commands to the new `languagecontainer.Run` function, centralizing command execution and wiring up `release.Init` for the `release-init` command. * **message.Change struct update**: The `SourceCommitHash` field in the `message.Change` struct has been renamed to `CommitHash` for improved clarity and consistency. This is due to the recent renaming of the field and [google-cloud-go/internal/librariangen/request.Change](https://github.com/googleapis/google-cloud-go/blob/7a85df39319e3a4870d4ad413f714ae5edd78ac8/internal/librariangen/request/request.go#L60) already has the field. The user (Java's language container in this case) doesn't have to implement the parsing logic and the tests. I moved the argument parsing tests to languagecontainer/languagecontainer_test.go from main_test.go. commit f22935d55d811acfa6903f9778d02b94aed34d13 Author: Tomo Suzuki <[email protected]> Date: Mon Oct 20 21:57:59 2025 -0400 chore(librariangen): introduce message package and add ReleaseInitRequest (googleapis#3958) commit 2f6c75da3021d030c7a192f1fbb4b30908ef9dad Author: Mike Eltsufin <[email protected]> Date: Fri Oct 17 11:29:31 2025 -0400 feat(librariangen): add generate package (googleapis#3952) Based on https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/generate with adaptation for Java. Currently it's just the scaffolding and more work is needed to generate a usable GAPIC library. The `generate` package contains the core logic for the generation process, including: - Reading and parsing the `generate-request.json` from librarian. - Parsing `BUILD.bazel` files in the googleapis repository to extract GAPIC configuration. - Building and executing `protoc` with the `gapic-generator-java` plugin. - Unzipping and restructuring the generated files into the final library layout. A `run-generate-library.sh` script is included for local development and end-to-end testing of the generation process. Additionally, a `go.work` file has been added to the root of the repository to support the multi-module workspace structure. commit feabef32c4c45be0fb1db3615568365b902ece24 Author: Mike Eltsufin <[email protected]> Date: Thu Oct 16 10:25:53 2025 -0400 feat(librariangen): add bazel package (googleapis#3940) Based on https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/bazel with adaptation for Java. commit 598de0697957ea3b05b1347c0898108648c3d7d1 Author: Mike Eltsufin <[email protected]> Date: Mon Oct 13 13:37:41 2025 -0400 feat(librariangen): add protoc package (googleapis#3935) This will be used for constructing the protoc command for generating Java GAPICs. Based on https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/protoc with the following important changes: * The `Build` function in `protoc.go` is updated to construct the `protoc` command with the correct arguments for Java GAPIC generation. * The tests in `protoc_test.go` are updated to reflect the changes in `protoc.go`. * The `gapicImportPath` is removed from the test configuration, as it is not relevant for Java.' * The testdata is included with modifications for Java. commit f32325e1d04a804e881b08091704a81c29461eb7 Author: Mike Eltsufin <[email protected]> Date: Mon Oct 13 10:46:41 2025 -0400 test(librariangen): add codecov and improve main.go coverage (googleapis#3936) Refactors the main function to be more testable by extracting the logic into a `runCLI` function. The logging setup is also extracted into a `parseLogLevel` function to allow for easier testing. This change increases the test coverage of main.go from 65.5% to 96.9%. commit fe44aede2aba9627db4aa0397d2ac4de353cea03 Author: Mike Eltsufin <[email protected]> Date: Fri Oct 10 20:01:26 2025 -0400 feat(librariangen): add request package (googleapis#3933) This will be used for parsing Librarian CLI requests. Copied from https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/request. commit f6b0b47a14eddd9c76540362b5922a5acc56b5d4 Author: Mike Eltsufin <[email protected]> Date: Fri Oct 10 20:01:07 2025 -0400 feat(librariangen): add execv package (googleapis#3932) This will be used for executing commands like protoc. Copied from https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/execv. commit 4c5a1c417a7033deb82039239728309b35e3e70b Author: Tomo Suzuki <[email protected]> Date: Thu Oct 9 16:05:42 2025 -0400 build: AR exit gate YAML to declare dependencies.gitSource (googleapis#3929) This should suppress the warning in b/450542318. commit 8f7ef85481fd9a926abcabb6de6451cd5f5faeee Author: Tomo Suzuki <[email protected]> Date: Wed Oct 8 17:18:00 2025 -0400 build: prepare Cloud Build YAML file for librarian-java (googleapis#3928) commit 29d188d20e530570f46774b37d88703822774a33 Author: Mike Eltsufin <[email protected]> Date: Wed Oct 8 13:32:54 2025 -0400 feat(librariangen): scaffold Java language container for Librarian (googleapis#3926) This change introduces the initial scaffolding for `librariangen`, a containerized Go application that will serve as the Java-specific code generator within the Librarian pipeline. The primary goal of this change is to establish the foundational components for the Java language container, including: * A new `librariangen` CLI application: Written in Go, this will be the entry point for Java code generation tasks invoked by Librarian. It currently includes basic command dispatching and support for `--version`. * A multi-stage `Dockerfile`: This creates a minimal container image for librariangen. * Unit tests and CI: A new GitHub Actions workflow has been added to build and test the librariangen executable. This scaffolding is the first step in implementing the full code generation logic within the Librarian system, eventually replacing the Java hermetic code generation system. The code is largely adapted from the Go language container at https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen. Fixes: googleapis#2500 ``` --------- Co-authored-by: Tomo Suzuki <[email protected]>
[2 of 4]
Create librarian_config_updater file. File consist in the function to update a librarian.yaml config file.
Second step in #3911.