Summary
check_provider_budget() exists in librefang-kernel-metering but is only called post-call via check_all_and_record(). Neither send_message_full_with_upstream nor send_message_ephemeral checks the per-provider token/cost budget before dispatching to the LLM driver.
This means:
- Streaming calls that have already consumed tokens cannot be rejected — the post-call check fires after the response finishes, and the usage is recorded anyway ("recording usage anyway" path)
- An exhausted provider keeps getting hit until the fallback chain or the caller gives up, wasting tokens on every attempt
- Retries multiply the waste — each retry re-sends the full prompt payload to a provider whose budget is already blown
Observed impact (production)
On a Rodela instance (2026-05-07), the Moonshot provider's 250k hourly token budget was exhausted at 300k. After a daemon restart:
- The new daemon inherited the exhausted counter from SQLite
clone_petroserver workflow failed immediately (3 retries, all rejected post-call)
- A Telegram auto-reply leaked through via streaming: 16,924 tokens consumed past the limit (
"Post-call quota check failed (streaming); recording usage anyway")
- The counter rose to 317k — all subsequent calls blocked for hours
Root cause
messaging.rs has pre-dispatch gates for:
- Global USD budget (
reserve_global_budget) ✓
- Per-agent token quota (
check_quota_and_reserve) ✓
- Per-provider budget → MISSING
The closed PR #4757 attempted to fix this with BudgetGatedDriver/ProviderBudgetGate but was rejected because the wiring was never connected (reviewer comment: types defined but never constructed).
Proposed fix
Add check_provider_budget() calls pre-dispatch in both:
send_message_full_with_upstream — after check_quota_and_reserve, with reservation rollback on failure
send_message_ephemeral — after the suspended-agent check
This is a minimal 27-line insertion (no new types, no new crates) that uses the existing MeteringEngine::check_provider_budget which already has unit test coverage.
Related
Summary
check_provider_budget()exists inlibrefang-kernel-meteringbut is only called post-call viacheck_all_and_record(). Neithersend_message_full_with_upstreamnorsend_message_ephemeralchecks the per-provider token/cost budget before dispatching to the LLM driver.This means:
Observed impact (production)
On a Rodela instance (2026-05-07), the Moonshot provider's 250k hourly token budget was exhausted at 300k. After a daemon restart:
clone_petroserverworkflow failed immediately (3 retries, all rejected post-call)"Post-call quota check failed (streaming); recording usage anyway")Root cause
messaging.rshas pre-dispatch gates for:reserve_global_budget) ✓check_quota_and_reserve) ✓The closed PR #4757 attempted to fix this with
BudgetGatedDriver/ProviderBudgetGatebut was rejected because the wiring was never connected (reviewer comment: types defined but never constructed).Proposed fix
Add
check_provider_budget()calls pre-dispatch in both:send_message_full_with_upstream— aftercheck_quota_and_reserve, with reservation rollback on failuresend_message_ephemeral— after the suspended-agent checkThis is a minimal 27-line insertion (no new types, no new crates) that uses the existing
MeteringEngine::check_provider_budgetwhich already has unit test coverage.Related
AgentMessagelets any authenticated peer spend the receiver's LLM budget — no per-peer rate limit, no per-peer cap #3876 (OFP peer budget spending)