refactor(aqua): bake aqua registry from merged yaml by risu729 · Pull Request #9043 · jdx/mise

risu729 · 2026-04-11T07:12:49Z

Summary

Replace the baked aqua-registry source with the upstream merged registry.yaml file, pinned by crates/aqua-registry/aqua-registry/metadata.json.
Parse the merged registry in crates/aqua-registry/build.rs and generate a canonical package-id to YAML map plus a separate alias lookup map, so runtime baked lookup only parses the selected package YAML.
Expose the pinned baked registry metadata from the aqua-registry crate and show it in mise doctor.
Update release tooling to pin aqua-registry by tag and list included upstream aqua-registry release tags in the changelog.
Update docs, vendored-file metadata, and the vendored-file warning workflow for the vendored crates/aqua-registry/aqua-registry/** tree.

Notes

The pinned upstream aqua-registry tag is v4.492.0; the vendored registry.yaml and LICENSE match that tag.
The baked registry build script uses eyre::Result/WrapErr for contextual build errors.
Baked registry keys intentionally follow the merged registry's canonical package IDs: name, then repo_owner/repo_name, then path for path-only packages. Legacy pkgs/** file paths such as _go/sigsum.org/sigsum-go/cmd/sigsum-key and crates.io/dskkato/rjo are aqua-registry implementation details and are not preserved as baked aliases.
Release changelog generation lists the included upstream aqua-registry release tags between the previous and new pinned tags. If the previous tag is not found, it skips the aqua-registry changelog section.
External/local aqua registry handling intentionally stays on the existing pkgs/<id>/registry.yaml path for a follow-up PR.
mise registry aliases are intentionally unchanged.
crates/aqua-registry/aqua-registry/pkgs/** is restored in a separate commit so it can be reverted later.

Future follow-ups

Remove crates/aqua-registry/aqua-registry/pkgs/** in a separate PR to keep this diff reviewable.
Compress the generated per-package YAML blobs with zstd to reduce the final binary size while keeping lazy per-package parsing.

Tests

cargo fmt
cargo test -p aqua-registry
cargo check --all-features
cargo check -p mise --all-features
cargo build -p mise --all-features
git diff --check
bash -n scripts/gen-aqua-changelog.sh xtasks/release-plz
cargo package -p aqua-registry --allow-dirty --list
./scripts/gen-aqua-changelog.sh v4.491.0 v4.492.0 ###
./scripts/gen-aqua-changelog.sh v4.492.0 v4.492.0 ###
./scripts/gen-aqua-changelog.sh v0.0.0 v4.492.0 ###
target/debug/mise doctor -J | rg 'baked_in_registry'
target/debug/mise doctor | rg -A2 '^aqua'

This PR body was generated by Codex.

gemini-code-assist

Code Review

This pull request updates the Cargo.toml file to explicitly include LICENSE and registry.yaml from the aqua-registry/ directory in the packaged crate. I have suggested that it might be more robust to include the entire aqua-registry/ directory to ensure that any future data files or metadata are automatically included without needing further updates to the include list.

greptile-apps · 2026-04-11T08:14:30Z

Greptile Summary

This PR replaces the per-package pkgs/**/registry.yaml vendoring strategy with a single merged registry.yaml snapshot pinned by metadata.json, and updates the build script to parse it into three generated maps (canonical files, aliases, and metadata). The release tooling is updated to download the merged file and generate a tag-range changelog via the GitHub API, and mise doctor now surfaces the pinned repository and tag.

Confidence Score: 5/5

Safe to merge; all remaining findings are P2 style/cleanup suggestions that do not affect correctness.

The refactor is well-scoped, tests cover canonical lookup, path-only packages, alias resolution, and metadata structure. The two flagged items are a dead code branch in the shell script (harmless) and a minor semantic mismatch using serde_yaml to parse JSON (works in practice). No P0/P1 issues found.

No files require special attention.

Important Files Changed

Filename	Overview
crates/aqua-registry/build.rs	Rewrites the build script to parse the merged registry.yaml and emit three generated files (files, aliases, metadata). Logic is sound; minor issue is using serde_yaml to parse the JSON metadata.json.
crates/aqua-registry/src/registry.rs	Adds AQUA_STANDARD_REGISTRY_METADATA static and alias lookup map; baked_registry_file helper correctly falls through canonical to alias to files; tests are thorough and properly structural.
scripts/gen-aqua-changelog.sh	Switches from per-file diff to tag-range listing via gh CLI. Main logic is correct; the empty-tags guard on line 49 is unreachable dead code.
xtasks/release-plz	Replaces git-clone + find with gh release view + curl for downloading registry.yaml, LICENSE, and metadata.json. Clean approach.
src/cli/doctor/mod.rs	Exposes baked_in_registry_repository and baked_in_registry_tag in the doctor JSON output and the human-readable string.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["registry.yaml"] -->|build.rs parses| B["package_registries()"]
    B --> C["registry_files_code()\ncanonical id to YAML"]
    B --> D["registry_aliases_code()\nalias to canonical id"]
    E["metadata.json"] -->|build.rs| F["generate_registry_metadata()"]
    C -->|include!| G["AQUA_STANDARD_REGISTRY_FILES"]
    D -->|include!| H["AQUA_STANDARD_REGISTRY_ALIASES"]
    F -->|include!| I["AQUA_STANDARD_REGISTRY_METADATA"]
    G --> J["baked_registry_file(id)"]
    H --> J
    J --> K["fetch_registry() returns YAML"]
    I --> L["mise doctor output"]

_{Reviews (16): Last reviewed commit: "Merge branch 'main' into codex/aqua-bake..." | Re-trigger Greptile}

jdx · 2026-04-11T17:37:01Z

what's the reason for this @risu729 ?

risu729 · 2026-04-11T17:55:36Z

Sorry I forgot to add this.
I wanted to support custom registries, but the current pkgs/**/*.yaml is not standard, just aqua-registry's choice.
We can continue using it for the default registry as I think they don't change the directory structure soon and the current implementation support the recursive scanning, but I thought it's better to use the same format for baked in registry and the external registry.
I'm gonna work on the custom registry in another PR.

@jdx

## Summary The release PR description was previously listing the aqua-registry release tags rolled into a mise release (e.g. `v4.491.0`, `v4.492.0`, ...), which reads as opaque version noise to users. This restores the pre-[#9043](#9043) behavior of listing the actual added and updated packages, but adapted to work against the merged `registry.yaml` that #9043 introduced. [`scripts/gen-aqua-changelog.sh`](https://github.com/jdx/mise/blob/main/scripts/gen-aqua-changelog.sh) now: - Fetches the previous tag's `registry.yaml` from upstream `aquaproj/aqua-registry`. - Parses both old and new files by canonical package id (`name` → `repo_owner/repo_name` → `path`), matching `crates/aqua-registry/build.rs::canonical_package_id`. - Emits `#### New Packages (N)` and `#### Updated Packages (N)` sections (matching the original pre-#9043 format). - Only links `repo_owner/repo_name` ids to GitHub, so name-only ids like `kiro.dev/kiro-cli` don't render as broken links. YAML parsing is done by inline `python3` (stdlib only — no `PyYAML` / `yq` dependency). The release runner is `ubuntu-latest` which ships python3. ## Sample output (`v4.491.0` → `v4.499.0`) ``` ### 📦 Aqua Registry Updates #### New Packages (28) - [`IBM-Cloud/ibm-cloud-cli-release`](https://github.com/IBM-Cloud/ibm-cloud-cli-release) - [`controlplaneio-fluxcd/flux-operator/flux-operator-mcp`](https://github.com/controlplaneio-fluxcd/flux-operator) - `kiro.dev/kiro-cli` - ... #### Updated Packages (13) - [`gleam-lang/gleam`](https://github.com/gleam-lang/gleam) - [`sigstore/cosign`](https://github.com/sigstore/cosign) - ... ``` ## Test plan - [x] `bash -n scripts/gen-aqua-changelog.sh` - [x] `mise run lint scripts/gen-aqua-changelog.sh` (shellcheck, shfmt, etc. clean) - [x] `./scripts/gen-aqua-changelog.sh v4.498.0 v4.499.0 "###"` — 1 added, 1 updated - [x] `./scripts/gen-aqua-changelog.sh v4.491.0 v4.499.0 "###"` — 28 added, 13 updated - [x] `./scripts/gen-aqua-changelog.sh v4.499.0 v4.499.0 "###"` — silent exit 0 - [x] `./scripts/gen-aqua-changelog.sh "" v4.499.0 "###"` — silent exit 0 - [x] `./scripts/gen-aqua-changelog.sh v0.0.0-bogus v4.499.0 "###"` — fails curl, exits 0 with stderr (matches prior fail-soft contract; `xtasks/release-plz` wraps the call in `|| true`) 🤖 Generated with [Claude Code](https://claude.com/claude-code)  --- > [!NOTE] > **Medium Risk** > Touches release/changelog generation logic and introduces custom YAML parsing, which could fail or mis-detect package changes if the registry format shifts, but does not affect runtime product behavior. > > **Overview** > Switches `scripts/gen-aqua-changelog.sh` from enumerating upstream `aqua-registry` release tags (via `gh`) to diffing the old vs current `registry.yaml` and printing **New Packages** and **Updated Packages** sections for the changelog. > > The script now fetches the old tag’s `registry.yaml` via `curl`, parses both YAMLs with an inline `python3` stdlib parser to compute canonical package IDs, and only adds GitHub links when `repo_owner/repo_name` is present to avoid broken links for name-only entries. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 6b93531. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>  --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

gemini-code-assist Bot reviewed Apr 11, 2026

View reviewed changes

Comment thread crates/aqua-registry/Cargo.toml

risu729 force-pushed the codex/aqua-baked-registry-yaml branch from 7ad7f50 to 47fcd35 Compare April 11, 2026 07:21

risu729 changed the title ~~Refactor aqua registry baking~~ refactor(backend): bake aqua registry from merged yaml Apr 11, 2026

risu729 changed the title ~~refactor(backend): bake aqua registry from merged yaml~~ refactor(aqua): bake aqua registry from merged yaml Apr 11, 2026

refactor(backend): bake aqua registry from merged yaml

b8bf855

risu729 force-pushed the codex/aqua-baked-registry-yaml branch from 47fcd35 to b8bf855 Compare April 11, 2026 07:31

risu729 marked this pull request as draft April 11, 2026 07:46

chore(aqua): restore vendored pkgs

262e51c

greptile-apps Bot reviewed Apr 11, 2026

View reviewed changes

Comment thread xtasks/release-plz Outdated

Comment thread scripts/gen-aqua-changelog.sh Outdated

Comment thread scripts/gen-aqua-changelog.sh Outdated

Comment thread src/cli/doctor/mod.rs Outdated

risu729 added 16 commits April 11, 2026 18:49

refactor(aqua): generate baked registry maps

22c9862

refactor(aqua): preserve static package ids

29ee7ca

docs(aqua): describe upstream registry

aee1531

chore(aqua): remove stale vendored attribute

0ad3164

refactor(aqua): use direct registry count

3b814f8

refactor(aqua): avoid package alias type

4fd6231

chore(aqua): pin registry release tag

3a3d8c5

chore(aqua): clarify registry changelog source

60ccfef

chore(aqua): link registry changelog tags

d91441b

chore(aqua): link registry changelog authors

e856da9

chore(aqua): keep registry changelog authors

cf5b2dd

chore(aqua): list included registry releases

b7940f4

feat(aqua): expose baked registry metadata

77e2b68

refactor(aqua): return results from registry build script

916eccf

refactor(aqua): use eyre in registry build script

f5486c1

chore(aqua): warn on vendored registry changes

dabfb14

risu729 marked this pull request as ready for review April 11, 2026 17:29

greptile-apps Bot reviewed Apr 11, 2026

View reviewed changes

Comment thread crates/aqua-registry/src/registry.rs Outdated

risu729 added 3 commits April 12, 2026 13:41

test(aqua): avoid hardcoded registry metadata tag

b051f6c

fix(aqua): preserve path-only baked packages

357b19c

Merge branch 'main' into codex/aqua-baked-registry-yaml

e77a5ac

risu729 mentioned this pull request Apr 15, 2026

feat(aqua): compress baked registry blobs with zstd #9131

Closed

risu729 added 2 commits April 19, 2026 03:43

Merge branch 'main' into codex/aqua-baked-registry-yaml

f9d9697

Merge branch 'main' into codex/aqua-baked-registry-yaml

2cb3f84

jdx merged commit b114eb5 into jdx:main Apr 23, 2026
35 checks passed

mise-en-dev mentioned this pull request Apr 23, 2026

chore: release 2026.4.20 #9311

Merged

risu729 deleted the codex/aqua-baked-registry-yaml branch April 23, 2026 13:57

BrewTestBot mentioned this pull request Apr 24, 2026

mise 2026.4.20 Homebrew/homebrew-core#279288

Merged

jdx mentioned this pull request Apr 29, 2026

feat(release): list aqua package additions/updates in changelog #9471

Merged

7 tasks

BrewTestBot mentioned this pull request Apr 29, 2026

mise 2026.4.27 Homebrew/homebrew-core#280081

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(aqua): bake aqua registry from merged yaml#9043

refactor(aqua): bake aqua registry from merged yaml#9043
jdx merged 23 commits intojdx:mainfrom
risu729:codex/aqua-baked-registry-yaml

risu729 commented Apr 11, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 11, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdx commented Apr 11, 2026

Uh oh!

risu729 commented Apr 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

risu729 commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Notes

Future follow-ups

Tests

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdx commented Apr 11, 2026

Uh oh!

risu729 commented Apr 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

risu729 commented Apr 11, 2026 •

edited

Loading

greptile-apps Bot commented Apr 11, 2026 •

edited

Loading