Skip to content

feat(kernel): budget-aware fallback chain — skip exhausted providers, try next in chain #4807

Description

@DaBlitzStein

Summary

When a provider's budget is exhausted, the pre-dispatch gate (#4800/#4801) rejects the entire call before the FallbackDriver gets a chance to try alternative providers. A Moonshot-exhausted agent with an ollama fallback configured gets blocked entirely instead of falling back.

Proposed design

Inject a ProviderBudgetChecker trait (defined in librefang-llm-driver, the trait crate) into FallbackChain. Before each provider attempt, check its budget. If exhausted, skip to next provider. Stamp actual_provider on CompletionResponse so usage is recorded against the provider that actually served the request.

Key components:

  1. ProviderBudgetChecker trait in librefang-llm-driver (avoids circular dep)
  2. FallbackChain checks budget per-provider before calling driver
  3. KernelBudgetChecker implements the trait using MeteringEngine
  4. actual_provider: Option<String> on CompletionResponseAgentLoopResultUsageRecord
  5. Remove the pre-dispatch provider gate from messaging.rs (the chain handles it internally now)

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/kernelCore kernel (scheduling, RBAC, workflows)area/runtimeAgent loop, LLM drivers, WASM sandboxenhancementNew feature or requesthas-prA pull request has been linked to this issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions