Skip to content

feat(dashboard): display + edit provider max-token limit (#6209)#6238

Merged
houko merged 2 commits into
mainfrom
feat/6209-provider-max-token
Jun 19, 2026
Merged

feat(dashboard): display + edit provider max-token limit (#6209)#6238
houko merged 2 commits into
mainfrom
feat/6209-provider-max-token

Conversation

@houko

@houko houko commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements #6209: the Providers (供应商) page now displays each provider's max-token limit, and the value is editable from the provider config drawer.

Interpretation of "最大token"

"最大token限制" is read as the per-request max-output-token limit of the provider's representative model.

  • The headline value is max_tokens override when the user has set one, otherwise the model's catalog max_output_tokens.
  • The representative model is the provider's default model (default_model_for_provider), falling back to the first catalog model for the provider.

This interpretation was chosen over context_window deliberately. The title says "限制" (a limit you impose) and "支持在UI界面修改" (editable), and max_tokens is the only per-model numeric value with an existing, registry-sync-safe, fully reversible persistence path (ModelOverridesmodel_overrides.json) that works for every provider — builtin and custom alike. context_window is shipped in the upstream librefang-registry and is overwritten on registry sync, so editing it is not safely persistable, and ModelOverrides carries no context_window field. Using max_tokens keeps the displayed value and the edited value identical (consistent between card and drawer).

Changes

Backend (crates/librefang-api/src/routes/providers.rs):

  • New provider_max_output_tokens(catalog, provider_id) helper resolves the representative model's effective output limit (override over catalog).
  • list_providers, providers_snapshot, and get_provider now emit max_output_tokens per provider. The value is computed once while the catalog snapshot is live so it stays consistent with the rest of the entry. The list/snapshot endpoints compute it for all providers in one pass (no per-card extra query).

Frontend (crates/librefang-api/dashboard/):

  • ProviderItem type gains max_output_tokens (src/api.ts).
  • Provider card shows a "max tokens" stat in both grid and list views, formatted with formatCompact (e.g. 16.4K) with the exact number on hover; renders - when unknown (src/pages/ProvidersPage.tsx).
  • New ProviderMaxTokensSection in the config drawer: a number input bound to the representative model's max_tokens override (model switchable when the provider has more than one model). Clearing the field deletes the override and reverts to the catalog default. It persists through the existing useUpdateModelOverrides mutation — all API access stays in the query/mutation hooks, no inline fetch/api.*.
  • i18n keys added to en.json, zh.json, uk.json.

CHANGELOG: entry under [Unreleased].

Backend change needed?

Yes — the per-provider max_output_tokens field is added to the provider list/snapshot/detail responses so the card can show it without N per-card queries. Editing itself needed no new endpoint; it reuses PUT/DELETE /api/models/overrides/{provider}:{model}.

Verification

Dashboard (crates/librefang-api/dashboard):

  • pnpm typecheck — clean.
  • pnpm lint — exit 0, no errors and no new warnings in ProvidersPage.tsx (only pre-existing warnings in unrelated files).
  • pnpm build✓ built in ~0.9s.
  • pnpm test:i18n-parityzh.json at parity with en.json; the 6 uk.json "extra" keys it flags (mcp.*_count_few, network.*_count_few, prompts.*_count_few) are pre-existing on origin/main and unrelated to this change. The 7 max_tokens* keys added here are present in all three locales.

Backend (via the repo's Dockerfile.rust-dev dev image; no native cargo on this host):

  • cargo check -p librefang-api --lib — clean.
  • cargo clippy -p librefang-api --lib --tests -- -D warnings — clean.
  • cargo fmt -p librefang-api -- --check — clean.
  • cargo test -p librefang-api --test providers_routes_test39 passed; 0 failed, including the new provider_max_output_tokens_reflects_catalog_then_override, which asserts the list/detail endpoints expose the catalog max_output_tokens, reflect a max_tokens override after PUT, and revert to the catalog default after the override is deleted.

Closes #6209

Surface each provider's max-token limit on the Providers page and make it editable from the config drawer.

The headline value is the provider's representative model's per-request max-output-token cap: the user's `max_tokens` override when set, otherwise the model's catalog `max_output_tokens`.
The representative model is the provider's default model (`default_model_for_provider`), falling back to the first catalog model.

Backend: `provider_max_output_tokens` resolves the value once per catalog snapshot and `list_providers` / `providers_snapshot` / `get_provider` emit it as `max_output_tokens`.
Editing reuses the existing per-model override endpoint (`PUT /api/models/overrides/{provider}:{model}`), which composes over the catalog default and is registry-sync-safe, so no new persistence path is introduced.

Frontend: a "max tokens" stat is added to the provider card (grid + list) and a "max tokens" editor to the config drawer; clearing the field drops the override and reverts to the catalog default.
Display strings are translated in en / zh / uk.

Closes #6209
@github-actions github-actions Bot added size/L 250-999 lines changed area/docs Documentation and guides labels Jun 19, 2026

houko commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Automated review — 1 CI failure fixed, 1 style fix applied (commit 6688065)

CI failure (Dashboard build job):

ProvidersPage.test.tsx has a strict vi.mock("../lib/queries/models", () => ({...})) factory that only listed useModels.
This PR added useModelOverrides to that same module, but did not update the factory.
Vitest throws at render time: No "useModelOverrides" export is defined on the "../lib/queries/models" mock.

Fix: added useModelOverrides: vi.fn(() => ({ data: undefined, isLoading: false })) to the factory (same pattern as useCredentialPools in the providers mock).

Style fix (CLAUDE.md — "never write multi-line comment blocks; one short line max"):

The 10-line comment block before ProviderMaxTokensSection was reduced to one line capturing the non-obvious design decision (why max_tokens not context_window).


Generated by Claude Code

@houko
houko merged commit 0567183 into main Jun 19, 2026
33 checks passed
@houko
houko deleted the feat/6209-provider-max-token branch June 19, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation and guides size/L 250-999 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

供应商默认展示最大token限制, 并支持在UI界面修改

2 participants