Summary
When a provider's budget is exhausted, the pre-dispatch gate (#4800/#4801) rejects the entire call before the FallbackDriver gets a chance to try alternative providers. A Moonshot-exhausted agent with an ollama fallback configured gets blocked entirely instead of falling back.
Proposed design
Inject a ProviderBudgetChecker trait (defined in librefang-llm-driver, the trait crate) into FallbackChain. Before each provider attempt, check its budget. If exhausted, skip to next provider. Stamp actual_provider on CompletionResponse so usage is recorded against the provider that actually served the request.
Key components:
ProviderBudgetChecker trait in librefang-llm-driver (avoids circular dep)
FallbackChain checks budget per-provider before calling driver
KernelBudgetChecker implements the trait using MeteringEngine
actual_provider: Option<String> on CompletionResponse → AgentLoopResult → UsageRecord
- Remove the pre-dispatch provider gate from
messaging.rs (the chain handles it internally now)
Related
Summary
When a provider's budget is exhausted, the pre-dispatch gate (#4800/#4801) rejects the entire call before the
FallbackDrivergets a chance to try alternative providers. A Moonshot-exhausted agent with an ollama fallback configured gets blocked entirely instead of falling back.Proposed design
Inject a
ProviderBudgetCheckertrait (defined inlibrefang-llm-driver, the trait crate) intoFallbackChain. Before each provider attempt, check its budget. If exhausted, skip to next provider. Stampactual_provideronCompletionResponseso usage is recorded against the provider that actually served the request.Key components:
ProviderBudgetCheckertrait inlibrefang-llm-driver(avoids circular dep)FallbackChainchecks budget per-provider before calling driverKernelBudgetCheckerimplements the trait usingMeteringEngineactual_provider: Option<String>onCompletionResponse→AgentLoopResult→UsageRecordmessaging.rs(the chain handles it internally now)Related
BudgetGatedDriverbut wiring was incomplete)