Add automatic Ruby download support using rv binaries#1668
Add automatic Ruby download support using rv binaries#1668is-alnilam wants to merge 8 commits intoj178:masterfrom
Conversation
Current Ruby support can only use Ruby interpreters which are already installed on the system, although it goes to great lengths to find interpreters installed by a variety of Ruby managers. This change adds support for installing new interpreters using the binaries delivered by the `rv` team. `rv` only provide installers for versions of Ruby still actively supported (so they don't offer version 3.1, for example), and only build for a subset of all Ruby-supported platforms. If users need an unsupported version of Ruby or wish to use an unsupported platform, they will be prompted to download and install a version of Ruby manually. `rv` bundles are named according to the platform, currently including these components in the filename: - x86_64_linux - arm64_linux - x86_64_linux_musl - arm64_linux_musl - ventura (used for macOS on x86_64) - arm64_sonoma (used for macOS on 64-bit ARM) If and when upstream `rv` changes these names, the detection code will need to be updated to match. In particular, this includes the use of macOS codenames, as if `rv` stop releasing a 'sonoma' package, this will block installing the macOS versions of Ruby. Currently `rv` seem to be attempting to keep these codenames, as they already rename their x86_64 builds from 'sequoia' (macOS 15) to 'ventura' (macOS 13). Adding a new CPU architecture (such as RISC-V) would also need changes, but wouldn't break existing platform support. Ruby versions are found by querying the GitHub Releases API, searching the options returned for an installer that matches the platform and version requirements, then, if found, downloading and unpacking into the `prek` tools folder. The `PREK_RUBY_MIRROR` environment variable can be used to point to a different source for installers, for example to support mirrors or air-gapped CI environments. Mirrors need to follow the GitHub URL patterns, but note that although the GitHub hostname changes between `api.github.com` and `github.com` as needed, any non-GitHub mirror server will not be remapped in this manner. Where Ruby is being downloaded from GitHub (either from the upstream `rv` or a mirror), this remapping does occur, and any `GITHUB_TOKEN` will be sent with the requests. This both limits impact of rate limiting, and also allows a private GitHub repository to be used (e.g. for a vetted subset of `rv` rubies to be mirrored). Note that GitHub tokens will only be sent to mirrors which are hosted on GitHub. To allow for passing the `GITHUB_TOKEN` in download requests, the generic `download_and_extract` function is now a wrapper over a version which takes an extension function, with the default function not extending the request. The Ruby code will add the GitHub token if the request is to GitHub.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1668 +/- ##
==========================================
+ Coverage 90.35% 91.62% +1.27%
==========================================
Files 96 96
Lines 18661 19077 +416
==========================================
+ Hits 16861 17480 +619
+ Misses 1800 1597 -203 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
📦 Cargo Bloat ComparisonBinary size change: +0.42% (23.8 MiB → 23.9 MiB) Expand for cargo-bloat outputHead Branch ResultsBase Branch Results |
|
We need to update the Ruby support status on this page: Lines 313 to 317 in 85d6dc6 |
Good point! I'm trying to deal with the CI failures, I'll add that as well. It looks like the Mac issues may be simply that the GitHub API is being rate limited from the CI runner (but I'll have to work out a way around that...) |
Looking for available Ruby versions via the API is subject to rate limiting when a GitHub token is not supplied. This change improves the messaging when GitHub return an error which may suggest that rate limiting is being applied, hinting to the user that they may need to supply a GitHub token.
To avoid being rate-limited when running tests, pass the GitHub token in
to the language tests (used by Ruby to download interpreters). The token
has no special permissions (the workflow already includes `permissions:
{}` at the top level), but this is sufficient to raise the rate limit.
There was a problem hiding this comment.
Pull request overview
This pull request adds automatic Ruby interpreter download support to prek using binaries from the rv-ruby project. Previously, prek could only use Ruby interpreters already installed on the system. With this change, prek can automatically download and install Ruby versions when requested versions aren't available locally.
Changes:
- Implements automatic Ruby download from rv-ruby GitHub releases for supported platforms (macOS and Linux on x86_64 and ARM64)
- Adds
PREK_RUBY_MIRRORenvironment variable for mirror/air-gapped support with GitHub token authentication - Extends download infrastructure with
download_and_extract_withto support request customization for authentication headers
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/languages.md | Updates Ruby documentation to reflect automatic download support and mirror configuration |
| crates/prek/tests/languages/ruby.rs | Adds comprehensive integration test for auto-download and caching behavior; updates error message snapshots |
| crates/prek/src/languages/ruby/version.rs | Adds Display trait implementation for RubyRequest to improve error messages |
| crates/prek/src/languages/ruby/ruby.rs | Integrates installer with download support by passing allows_download flag and tools directory path |
| crates/prek/src/languages/ruby/installer.rs | Core implementation: GitHub release API querying, platform detection, download logic, system Ruby search (including rv paths), and extensive test coverage |
| crates/prek/src/languages/mod.rs | Adds download_and_extract_with wrapper to support request customization (e.g., auth headers) |
| crates/prek-consts/src/env_vars.rs | Adds PREK_RUBY_MIRROR environment variable constant |
| .github/workflows/ci.yml | Propagates GITHUB_TOKEN to language tests to avoid API rate limiting |
| let (base, is_github) = rv_ruby_mirror(); | ||
| let url = if is_github { | ||
| // Rewrite github.com web URL to API URL. | ||
| let path = base.strip_prefix("https://github.com").unwrap(); | ||
| format!("https://api.github.com/repos{path}/releases/latest") |
There was a problem hiding this comment.
The unwrap() call here is fragile. While it should always succeed given that is_github is only true when the URL starts with "https://github.com/", this creates tight coupling between is_github_https and rv_ruby_api_url. Consider using expect() with a descriptive message explaining the invariant, or restructure to make the relationship more explicit.
| let (base, is_github) = rv_ruby_mirror(); | |
| let url = if is_github { | |
| // Rewrite github.com web URL to API URL. | |
| let path = base.strip_prefix("https://github.com").unwrap(); | |
| format!("https://api.github.com/repos{path}/releases/latest") | |
| let (base, mut is_github) = rv_ruby_mirror(); | |
| let url = if is_github { | |
| // Rewrite github.com web URL to API URL. | |
| if let Some(path) = base.strip_prefix("https://github.com") { | |
| format!("https://api.github.com/repos{path}/releases/latest") | |
| } else { | |
| // Defensive fallback: mirror was marked as GitHub but does not have the | |
| // expected prefix. Treat it as a non-GitHub mirror to avoid panicking | |
| // or sending tokens to an unexpected host. | |
| warn!( | |
| "PREK_RUBY_MIRROR is marked as GitHub but does not start with \ | |
| 'https://github.com': {base}" | |
| ); | |
| is_github = false; | |
| format!("{base}/releases/latest") | |
| } |
There was a problem hiding this comment.
Personally I'd probably change this unwrap() to an expect()? It should never be hit, after all...
Current Ruby support can only use Ruby interpreters which are already installed on the system, although it goes to great lengths to find interpreters installed by a variety of Ruby managers. This change adds support for installing new interpreters using the binaries delivered by the
rvteam.rvonly provide installers for versions of Ruby still actively supported (so they don't offer version 3.1, for example), and only build for a subset of all Ruby-supported platforms. If users need an unsupported version of Ruby or wish to use an unsupported platform, they will be prompted to download and install a version of Ruby manually.rvbundles are named according to the platform, currently including these components in the filename:If and when upstream
rvchanges these names, the detection code will need to be updated to match. In particular, this includes the use of macOS codenames, as ifrvstop releasing a 'sonoma' package, this will block installing the macOS versions of Ruby. Currentlyrvseem to be attempting to keep these codenames, as they already rename their x86_64 builds from 'sequoia' (macOS 15) to 'ventura' (macOS 13). Adding a new CPU architecture (such as RISC-V) would also need changes, but wouldn't break existing platform support.Ruby versions are found by querying the GitHub Releases API, searching the options returned for an installer that matches the platform and version requirements, then, if found, downloading and unpacking into the
prektools folder. ThePREK_RUBY_MIRRORenvironment variable can be used to point to a different source for installers, for example to support mirrors or air-gapped CI environments. Mirrors need to follow the GitHub URL patterns, but note that although the GitHub hostname changes betweenapi.github.comandgithub.comas needed, any non-GitHub mirror server will not be remapped in this manner. Where Ruby is being downloaded from GitHub (either from the upstreamrvor a mirror), this remapping does occur, and anyGITHUB_TOKENwill be sent with the requests. This both limits impact of rate limiting, and also allows a private GitHub repository to be used (e.g. for a vetted subset ofrvrubies to be mirrored). Note that GitHub tokens will only be sent to mirrors which are hosted on GitHub.To allow for passing the
GITHUB_TOKENin download requests, the genericdownload_and_extractfunction is now a wrapper over a version which takes an extension function, with the default function not extending the request. The Ruby code will add the GitHub token if the request is to GitHub.Closes #43
Closes #765