You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hermes can list its providers and models four ways today — cli.py interactive picker, tui_gateway/server.pymodel.options, hermes_cli/web_server.py/api/model/options, hermes_cli/main.py:2250 switcher — but none of them are scriptable. Every entry point is either interactive (TTY required) or HTTP (gateway running). There is no way for a cron job, a CI script, an external dashboard, or a Hermes-based agent to ask "what providers exist on this host, which are configured, what models can I switch to" without parsing free-form text or maintaining a reverse-engineered mental model of hermes_cli/auth.py:PROVIDER_REGISTRY.
The dashboard at /api/model/options already returns this exact data as JSON — it just isn't reachable from the CLI.
Concrete pain
Cron job verifying a key is valid before a batch runs has to spawn an interactive hermes, parse hermes doctor's free-form output, or import internals.
Observability layer can't answer "of 33 supported providers, which 12 are working on this host" without standing up the gateway HTTP server.
Multi-host fleet check is ssh + hermes model + eyeball + repeat instead of ssh + hermes providers list --json | jq | aggregate.
Hermes-based AI agent introspecting its own environment can't scrape an answer — has to query OpenAI/Anthropic/etc. directly and bypass the framework it's running on.
Plugin authors validating a new plugins/model-providers/<name>/ have to launch the picker and scroll, instead of hermes providers list | grep <name>.
Debugging "why isn't my provider showing up" needs reading code; with a scriptable surface, hermes providers list would show a row with auth_state: unconfigured, env_vars: TOGETHER_API_KEY (not set) in five seconds.
Five open PRs reinvent the same traversal
Each of the following touches inventory traversal for one narrow purpose, with no shared function:
While researching the fix I verified these against main at d62808c37. They are not blockers for the inventory CLI, but should be tracked separately so they don't get lost:
Four registries don't fully overlap.providers.list_providers() = 33 plugin profiles. hermes_cli.auth.PROVIDER_REGISTRY = 33 (different set — excludes openrouter, custom; includes lmstudio, tencent-tokenhub). hermes_cli.models.CANONICAL_PROVIDERS = 35 (superset). _PROVIDER_MODELS = 32 (yet another set: includes moonshot, openai; missing custom, ollama-cloud, openrouter, qwen-oauth). Iterating over any single one drops real providers from the operator-visible inventory.
profile.fetch_models() returns models the runtime rejects. OpenRouter raw = 367. provider_model_ids("openrouter") (used by the picker) = 34 tool-call-filtered. The 333-model gap = models that don't support tool-calling. The plugin's own docstring says: "Tool-call capability filtering is applied by hermes_cli/models.py via fetch_openrouter_models() → _openrouter_model_supports_tools(), not here." Anyone (including agents introspecting via get_provider_profile) calling profile.fetch_models() directly gets the unfiltered list and runtime errors on most picks.
profile.fallback_models is smaller than _PROVIDER_MODELS[slug] for 28 of 33 providers.anthropic curated = 8, profile.fallback_models = 0. copilot = 17/0. bedrock = 10/0. nous = 24/2 — and the 2 in profile aren't in curated.
provider_model_ids("nous") without credentials bypasses the remote catalog. Returns 24 (static _PROVIDER_MODELS). get_curated_nous_model_ids() returns 32 (with the remote manifest). Same gap may exist for other manifest-backed providers.
list_authenticated_providers does unconditional live fetches before per-provider gating: fetch_models_dev() (HTTP to models.dev/api.json, cached 1h), fetch_ollama_cloud_models() (always), fetch_lmstudio_models() (when LM_BASE_URL set or current = lmstudio). Callers assuming "this is offline-safe" hit the network.
Proposed scope of this tracking issue
A non-interactive, scriptable, TTY-free CLI surface that consolidates the four current surfaces:
hermes models list [--provider NAME] [--json] [--all] [--no-live] [--offline]
hermes models status [--provider NAME] [--json] [--probe] [--offline]
hermes providers list [--json] [--all] [--offline]
Default behaviour matches the existing picker/dashboard (configured-only, network calls allowed, cached). --offline synthesises from in-repo static data only — no HTTP, no remote catalogs, no credential-pool probing — for cron/CI use. --probe is opt-in and explicitly mutually exclusive with --offline.
TUI / dashboard migration to consume the shared inventory module. Module ships with a public merge_canonical_with_authenticated helper for them to opt in later.
The gap
Hermes can list its providers and models four ways today —
cli.pyinteractive picker,tui_gateway/server.pymodel.options,hermes_cli/web_server.py/api/model/options,hermes_cli/main.py:2250switcher — but none of them are scriptable. Every entry point is either interactive (TTY required) or HTTP (gateway running). There is no way for a cron job, a CI script, an external dashboard, or a Hermes-based agent to ask "what providers exist on this host, which are configured, what models can I switch to" without parsing free-form text or maintaining a reverse-engineered mental model ofhermes_cli/auth.py:PROVIDER_REGISTRY.The dashboard at
/api/model/optionsalready returns this exact data as JSON — it just isn't reachable from the CLI.Concrete pain
hermes, parsehermes doctor's free-form output, or import internals.ssh + hermes model + eyeball + repeatinstead ofssh + hermes providers list --json | jq | aggregate.plugins/model-providers/<name>/have to launch the picker and scroll, instead ofhermes providers list | grep <name>.hermes providers listwould show a row withauth_state: unconfigured, env_vars: TOGETHER_API_KEY (not set)in five seconds.Five open PRs reinvent the same traversal
Each of the following touches inventory traversal for one narrow purpose, with no shared function:
/modelsslash (gateway) — callsprovider_model_ids()directly + own formatting + own current-provider detection.model.picker_modeconfig to hide unconfigured providers from the picker./free-modelsfor OpenRouter — special case ofmodels list --provider=openrouter --filter=free.hermes doctorskips/modelshealth checks for providers without that endpoint.Four issues converge on the same gap:
/modelsslash command for listing models.providersthe single source of truth for model registry #16622 — Makeproviders:config the source of truth for the model registry.Empirical findings worth tracking separately
While researching the fix I verified these against
mainatd62808c37. They are not blockers for the inventory CLI, but should be tracked separately so they don't get lost:Four registries don't fully overlap.
providers.list_providers()= 33 plugin profiles.hermes_cli.auth.PROVIDER_REGISTRY= 33 (different set — excludesopenrouter,custom; includeslmstudio,tencent-tokenhub).hermes_cli.models.CANONICAL_PROVIDERS= 35 (superset)._PROVIDER_MODELS= 32 (yet another set: includesmoonshot,openai; missingcustom,ollama-cloud,openrouter,qwen-oauth). Iterating over any single one drops real providers from the operator-visible inventory.profile.fetch_models()returns models the runtime rejects. OpenRouter raw = 367.provider_model_ids("openrouter")(used by the picker) = 34 tool-call-filtered. The 333-model gap = models that don't support tool-calling. The plugin's own docstring says: "Tool-call capability filtering is applied by hermes_cli/models.py via fetch_openrouter_models() → _openrouter_model_supports_tools(), not here." Anyone (including agents introspecting viaget_provider_profile) callingprofile.fetch_models()directly gets the unfiltered list and runtime errors on most picks.profile.fallback_modelsis smaller than_PROVIDER_MODELS[slug]for 28 of 33 providers.anthropiccurated = 8,profile.fallback_models= 0.copilot= 17/0.bedrock= 10/0.nous= 24/2 — and the 2 in profile aren't in curated.provider_model_ids("nous")without credentials bypasses the remote catalog. Returns 24 (static_PROVIDER_MODELS).get_curated_nous_model_ids()returns 32 (with the remote manifest). Same gap may exist for other manifest-backed providers.list_authenticated_providersdoes unconditional live fetches before per-provider gating:fetch_models_dev()(HTTP tomodels.dev/api.json, cached 1h),fetch_ollama_cloud_models()(always),fetch_lmstudio_models()(whenLM_BASE_URLset or current = lmstudio). Callers assuming "this is offline-safe" hit the network.Proposed scope of this tracking issue
A non-interactive, scriptable, TTY-free CLI surface that consolidates the four current surfaces:
Default behaviour matches the existing picker/dashboard (configured-only, network calls allowed, cached).
--offlinesynthesises from in-repo static data only — no HTTP, no remote catalogs, no credential-pool probing — for cron/CI use.--probeis opt-in and explicitly mutually exclusive with--offline.Fixing this lands as:
model.allowlist/model.picker_mode(feat(cli): add picker_mode config to restrict model picker #19526).providersthe single source of truth for model registry #16622 — needs write-side symmetry (hermes modelwriting toproviders.<slug>.models); filing as separate ~50 LOC follow-up.Out of scope (tracked separately)
hermes model(Makeprovidersthe single source of truth for model registry #16622). ~50 LOC follow-up; schema fixtures already exist intests/hermes_cli/test_model_catalog.py.merge_canonical_with_authenticatedhelper for them to opt in later./modelsgateway slash command unification (feat(gateway): add /models slash command for model discovery #3503). After this lands, comment on feat(gateway): add /models slash command for model discovery #3503 proposing migration.References
providersthe single source of truth for model registry #16622.