Skip to content

fix: parse inline CLI options for ls-remote, overrides mise.toml#8902

Closed
esteve wants to merge 3 commits intojdx:mainfrom
esteve:parse-inline-ls-remote
Closed

fix: parse inline CLI options for ls-remote, overrides mise.toml#8902
esteve wants to merge 3 commits intojdx:mainfrom
esteve:parse-inline-ls-remote

Conversation

@esteve
Copy link
Copy Markdown
Contributor

@esteve esteve commented Apr 4, 2026

This PR enables ls-remote to parse options from the CLI (e.g. mise ls-remote "vfox-pixi:samtools[channels=bioconda]"), overriding options from mise.toml if the latter exists, as introduced in #8875 . Also adds support for parsing TOML arrays (e.g. mise ls-remote 'vfox-pixi:samtools[channels=["conda-forge", "bioconda"]]')

@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 4, 2026

@jdx I realized that ls-remote is meant to be context-free and not rely on mise.toml, which is what I did in #8875 , let me know if the previous behavior is still valid, but I don't think ls-remote should read anything from mise.toml.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refines tool option handling by prioritizing inline CLI options over configuration files in the vfox backend and ensuring the ls-remote command remains context-free by bypassing local configuration. Additionally, the regex for parsing bracketed options was updated to be more precise. I have no feedback to provide as there were no review comments to evaluate.

@jdx
Copy link
Copy Markdown
Owner

jdx commented Apr 4, 2026

I realized that ls-remote is meant to be context-free and not rely on mise.toml.

I don't think so

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 4, 2026

Greptile Summary

This PR fixes two related issues: a regex bug in split_bracketed_opts that caused the greedy first capture group to swallow inner TOML array brackets (e.g. channels=[\"conda-forge\", \"bioconda\"]), and threads an explicit opts: Option<ToolVersionOptions> parameter through the version-listing call chain so that ls-remote can pass inline CLI opts directly to backends, bypassing both the version cache and mise.toml config opts.

Confidence Score: 5/5

Safe to merge — no P0/P1 issues found; previous thread concerns about cached-backend opts have been addressed by threading opts as an explicit parameter

The regex fix is correct and necessary. The opts-threading design is clean and avoids the cached-backend pitfall by never relying on self.ba.opts inside the backend implementation. The only finding is a P2 style inconsistency (missing warn! for invalid versions in the opts path).

No files require special attention; all changes are consistent and well-structured

Important Files Changed

Filename Overview
src/cli/args/backend_arg.rs Regex fix in split_bracketed_opts: [^\[]+ prevents the first group from consuming inner TOML array brackets, enabling correct parsing of channels=["conda-forge", "bioconda"]
src/backend/mod.rs list_remote_versions_with_info and _list_remote_versions_with_opts gain an opts param; cache is correctly bypassed when opts is Some; default _list_remote_versions_with_opts ignores opts for backends that don't support them
src/cli/ls_remote.rs Opts extracted from tool_arg.ba.opts and passed explicitly to list_remote_versions_with_info, sidestepping any cached-backend ba.opts discrepancy
src/backend/vfox.rs Adds _list_remote_versions_with_opts that uses inline opts when Some, otherwise falls back to config.get_tool_opts
src/backend/github.rs Splits _list_remote_versions into a with_opts variant; inline opts override config/ba opts for release listing
src/backend/http.rs fetch_versions accepts explicit opts; inline opts take full precedence over both ba.opts and config opts, preserving the existing version_list_url fallback logic when no opts are supplied
src/backend/s3.rs fetch_versions now accepts opts parameter; inline opts bypass config lookup for S3 version discovery
src/backend/conda.rs list_remote_versions_with_info signature updated to accept opts; delegates to default _list_remote_versions_with_opts which correctly ignores them
src/plugins/core/java.rs list_remote_versions_with_info signature updated to accept opts; delegates to default _list_remote_versions_with_opts which correctly ignores them

Sequence Diagram

sequenceDiagram
    participant User
    participant LsRemote
    participant BackendArg
    participant Backend as Backend (trait)
    participant Cache as VersionCacheManager
    participant VfoxBackend
    participant Config

    User->>LsRemote: mise ls-remote 'vfox-pixi:samtools[channels=["bioconda"]]'
    LsRemote->>LsRemote: run() → run_single(config, plugin)

    note over LsRemote: Extract inline opts from tool_arg.ba.opts
    LsRemote->>BackendArg: ta.ba.opts.clone()
    BackendArg-->>LsRemote: Some(opts) — channels=["bioconda"]

    LsRemote->>Backend: list_remote_versions_with_info(config, Some(opts))

    alt opts is Some — inline opts path
        note over Backend: Cache bypass (results depend on opts)
        Backend->>VfoxBackend: _list_remote_versions_with_opts(config, Some(opts))
        VfoxBackend->>VfoxBackend: inline opts present → use opts.opts directly
        note over VfoxBackend: config.get_tool_opts() NOT called
        VfoxBackend->>VfoxBackend: vfox.backend_list_versions(pathname, tool_name, inline_opts)
        VfoxBackend-->>Backend: Vec<VersionInfo>
        Backend-->>LsRemote: filtered Vec<VersionInfo>
    else opts is None — normal cached path
        Backend->>Cache: get_or_try_init_async(...)
        alt cache miss
            Cache->>Backend: _list_remote_versions(config)
            Backend->>VfoxBackend: _list_remote_versions(config)
            VfoxBackend->>VfoxBackend: delegates to _list_remote_versions_with_opts(config, None)
            VfoxBackend->>Config: get_tool_opts(&ba)
            Config-->>VfoxBackend: ToolVersionOptions from mise.toml
            VfoxBackend-->>Cache: Vec<VersionInfo>
        end
        Cache-->>Backend: cached Vec<VersionInfo>
        Backend-->>LsRemote: Vec<VersionInfo>
    end

    LsRemote->>LsRemote: filter by prefix, print versions
    LsRemote-->>User: version list output
Loading

Reviews (7): Last reviewed commit: "propagate opts param to backends, remove..." | Re-trigger Greptile

Comment thread src/cli/ls_remote.rs Outdated
@jdx jdx marked this pull request as draft April 4, 2026 16:31
@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 4, 2026

I don't think so

So ls-remote should read from mise.toml as well?

I got the impression it's more of a one-off command, since there's no mention of mise.toml in https://mise.jdx.dev/cli/ls-remote.html

Should I consider both cases in this PR (with/without mise.toml)? Currently on main I can only query for a package if the channels have been defined in a project's mise.toml, not outside of it.

@esteve esteve force-pushed the parse-inline-ls-remote branch from 403bbc4 to ac7ab17 Compare April 4, 2026 17:18
@jdx
Copy link
Copy Markdown
Owner

jdx commented Apr 4, 2026

The regex fix in backend_arg.rs looks good — without it, TOML arrays in bracket opts break because the greedy (.+) matches the wrong closing bracket.

However, I have concerns about the other two changes:

ls_remote.rs — forcing empty opts:
ls-remote should still read from mise.toml when no inline opts are provided. If someone has a complex tool like ruby with a bunch of tool options in config, they should still be able to run mise ls-remote ruby without re-specifying everything. Force-inserting empty opts and bypassing the backend cache breaks that.

vfox.rs — inline opts override config opts:
The precedence change (inline > config) is actually correct — if you explicitly pass opts on the CLI they should win. Note that the other backends (github, http, s3) all have the same pre-existing issue where config silently overrides inline opts, but that's out of scope for this PR.

I'd suggest keeping the regex fix and the vfox.rs precedence change, but dropping the ls_remote.rs changes.

This comment was generated by Claude Code.

@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 4, 2026

@jdx I've reenabled reading from mise.toml in ac7ab17

@esteve esteve marked this pull request as ready for review April 4, 2026 17:28
@esteve esteve changed the title fix: parse inline CLI options for ls-remote, do not read options from mise.toml fix: parse inline CLI options for ls-remote, overrides mise.toml Apr 4, 2026
@jdx
Copy link
Copy Markdown
Owner

jdx commented Apr 4, 2026

I dug into this more and my earlier feedback was wrong. The TOOLS cache looks up backends by short name, so the cached backend has its own BackendArg from when it was first created (config/registry load time). Inline CLI opts on the ToolArg.ba are on a different BackendArg and get silently dropped after the cache lookup.

So there's no precedence bug — inline bracket opts on ls-remote simply don't reach the backend at all without bypassing the cache. The ls_remote.rs cache bypass is necessary for inline opts to work, and the vfox.rs check on this.ba.opts only works because the bypassed backend carries the CLI's BackendArg.

That said, this cache bypass + vfox-specific opts check is still vfox-only. If we want inline opts on ls-remote to work for other backends (github, http, s3), they'd need similar treatment. Could this be generalized — e.g., passing opts through to _list_remote_versions rather than each backend reading them independently?

This comment was generated by Claude Code.

@esteve esteve force-pushed the parse-inline-ls-remote branch 2 times, most recently from 849a182 to 440e638 Compare April 4, 2026 17:36
@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 4, 2026

@jdx ls_remote.rs changes removed in the latest revision.

@jdx jdx marked this pull request as draft April 4, 2026 17:38
Comment thread src/backend/vfox.rs Outdated
@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 4, 2026

So there's no precedence bug — inline bracket opts on ls-remote simply don't reach the backend at all without bypassing the cache. The ls_remote.rs cache bypass is necessary for inline opts to work, and the vfox.rs check on this.ba.opts only works because the bypassed backend carries the CLI's BackendArg.

@jdx I've re-added the cache bypass in ls_remote.rs in 96c9927

That said, this cache bypass + vfox-specific opts check is still vfox-only. If we want inline opts on ls-remote to work for other backends (github, http, s3), they'd need similar treatment. Could this be generalized — e.g., passing opts through to _list_remote_versions rather than each backend reading them independently?

Probably, I don't know enough about the other backends, for now I'm only focusing on this so that I can complete my vfox-pixi backend.

@esteve esteve marked this pull request as ready for review April 4, 2026 18:00
@jdx
Copy link
Copy Markdown
Owner

jdx commented Apr 4, 2026

The vfox-only approach isn't going to work — we need inline opts to be handled generically so all backends (github, http, s3, etc.) benefit from the same fix. Rather than each backend independently checking ba.opts, the opts should be passed through from ls_remote.rs into the backend's _list_remote_versions call so backends don't need to know where the opts came from.

The regex fix in backend_arg.rs is good and can stay as-is.

This comment was generated by Claude Code.

@jdx
Copy link
Copy Markdown
Owner

jdx commented Apr 4, 2026

To expand on my previous comment — the ls_remote.rs cache bypass is a band-aid for a deeper problem: inline opts on BackendArg get silently dropped at the TOOLS cache boundary everywhere, not just in ls-remote. The cache keys on ba.short only, and BackendArg's Hash/Eq impls ignore opts entirely. This needs to be solved generically rather than per-call-site.

The regex fix in backend_arg.rs is solid — that should stay regardless.

This comment was generated by Claude Code.

@jdx jdx marked this pull request as draft April 5, 2026 14:28
@esteve esteve force-pushed the parse-inline-ls-remote branch 2 times, most recently from 179efb9 to e9ad707 Compare April 6, 2026 10:47
@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 6, 2026

@jdx I've removed the vfox-specific code and added a function called _list_remote_versions_with_opts that by default just calls _list_remote_versions, but can be overriden by the backends that use opts (i.e. conda, github, http, s3 and vfox)., the rest of the backends are unchanged. I did this because it felt cleaner and to minimize the changes to the other backends, but another approach would be to add an opts parameters to _list_remote_versions that is ignored in the backends that do not use opts.

@esteve esteve marked this pull request as ready for review April 6, 2026 10:57
@esteve esteve force-pushed the parse-inline-ls-remote branch from e9ad707 to 8f78778 Compare April 7, 2026 11:57
@esteve
Copy link
Copy Markdown
Contributor Author

esteve commented Apr 13, 2026

@jdx anything else that should be addressed? Would you prefer I add opts to _list_remote_versions and update all the backends or is the current approach with _list_remote_versions_with_opts ok?

@jdx
Copy link
Copy Markdown
Owner

jdx commented Apr 27, 2026

going to pass for now, sorry

@jdx jdx closed this Apr 27, 2026
jdx pushed a commit that referenced this pull request May 5, 2026
## What

- Add a shared backend option resolver that preserves where each
effective option came from.
- Resolve backend options in this order, with later layers overriding
earlier ones:
  1. Registry backend defaults.
2. Backend alias `[...]` options from `[tool_alias]` / `[alias]` backend
entries.
  3. Config options selected by the existing tool config lookup rules.
4. Inline backend options from the current tool spec, such as
`tool[...]`.
- Apply that resolved option path consistently in GitHub/GitLab/Forgejo,
HTTP, S3, UBI, vfox backend plugins, Conda channel resolution, SPM
provider/API URL resolution, Java release type listing, versions-host
cache filtering, fuzzy version matching, `minimum_release_age` handling,
runtime args, and install request initialization.
- Preserve inline backend options across the short-name backend cache,
so commands like `ls-remote 'github:owner/repo[api_url=...]'` do not
accidentally use a cached installed backend without those inline
options.
- Preserve config options when runtime args include inline overrides,
while still letting inline options win for the one command.
- Switch Go install to use the resolved request options.
- Fix shorthand parsing for URL-valued inline options such as
`tool[api_url=https://example/api/v3]`.

## Follow-up

The versions-host behavior for locally overridden backend options has
been split into #9568. That PR is stacked on this one and should be
rebased after this merges.

## Relationship to #9315

This PR does not change which config entry is eligible for a tool. That
behavior comes from #9315:

- Registry short names should not inherit config from their resolved
full backend key.
- Configured aliases can fall back to resolved full-backend config when
there is no alias-specific entry.
- Alias-specific `[tools.<alias>]` config is more specific than
`[tools."<resolved-backend>"]` config.

This PR builds on that lookup and changes what happens after config has
been selected: registry defaults, backend alias options, config options,
and inline options are merged through one resolver, with provenance
retained for callers that need to distinguish where an option came from.

## Relationship to #8902

#8902 was a narrower `ls-remote` fix for inline backend options. This PR
covers the same installed-tool cache failure mode generically: when a
caller supplies inline backend options, mise builds a backend from that
caller `BackendArg` instead of returning a short-name cache entry that
was loaded from install state without the inline options.

## Example

Given config selected through an alias fallback:

```toml
[tool_alias]
hw = "github:jdx/mise-test-fixtures[api_url=https://api.github.com]"

[tools."github:jdx/mise-test-fixtures"]
version = "1.0.0"
asset_pattern = "definitely-not-a-real-hello-world-asset"
```

Inline options should still win for this invocation:

```sh
mise install 'hw[asset_pattern=hello-world-1.0.0.tar.gz,bin_path=hello-world-1.0.0/bin]@1.0.0'
```

Before this PR, the resolved full-backend config could replace the
inline `asset_pattern`, and alias `[...]` options were only visible as
part of a resolved string. After this PR, alias options are their own
layer and inline options are applied over the selected config options.

## Why

Backend-level code had several config-or-inline branches. Depending on
the path:

- Inline options could be ignored when config existed for the same tool
or resolved backend.
- Inline options could be dropped at the installed-tool backend cache
boundary.
- Config options could be dropped when runtime args included inline
options.
- Backend alias `[...]` options were not represented as a distinct
layer.
- Backend-specific list/install paths could read raw backend args
instead of the resolved tool options.
- Registry defaults were not always part of the same overlay path as
config and inline options.

This PR centralizes that overlay so config-aware paths use the same
precedence, while preserving source information separately from the
effective option values.

## Validation

- `cargo fmt --all`
- `git diff --check`
- `cargo test --all-features tool_opts`
- `cargo test --all-features opts_with_config`
- `cargo test --all-features test_git_provider_from_ba`
- `mise run test:e2e e2e/backend/test_github_ls_remote_inline_opts`

*This PR was updated by an AI coding assistant.*

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants