Skip to content

fix(kernel): provider budget checked post-call only — streaming leaks tokens past exhausted budgets #4800

Description

@DaBlitzStein

Summary

check_provider_budget() exists in librefang-kernel-metering but is only called post-call via check_all_and_record(). Neither send_message_full_with_upstream nor send_message_ephemeral checks the per-provider token/cost budget before dispatching to the LLM driver.

This means:

  1. Streaming calls that have already consumed tokens cannot be rejected — the post-call check fires after the response finishes, and the usage is recorded anyway ("recording usage anyway" path)
  2. An exhausted provider keeps getting hit until the fallback chain or the caller gives up, wasting tokens on every attempt
  3. Retries multiply the waste — each retry re-sends the full prompt payload to a provider whose budget is already blown

Observed impact (production)

On a Rodela instance (2026-05-07), the Moonshot provider's 250k hourly token budget was exhausted at 300k. After a daemon restart:

  • The new daemon inherited the exhausted counter from SQLite
  • clone_petroserver workflow failed immediately (3 retries, all rejected post-call)
  • A Telegram auto-reply leaked through via streaming: 16,924 tokens consumed past the limit ("Post-call quota check failed (streaming); recording usage anyway")
  • The counter rose to 317k — all subsequent calls blocked for hours

Root cause

messaging.rs has pre-dispatch gates for:

  • Global USD budget (reserve_global_budget) ✓
  • Per-agent token quota (check_quota_and_reserve) ✓
  • Per-provider budget → MISSING

The closed PR #4757 attempted to fix this with BudgetGatedDriver/ProviderBudgetGate but was rejected because the wiring was never connected (reviewer comment: types defined but never constructed).

Proposed fix

Add check_provider_budget() calls pre-dispatch in both:

  1. send_message_full_with_upstream — after check_quota_and_reserve, with reservation rollback on failure
  2. send_message_ephemeral — after the suspended-agent check

This is a minimal 27-line insertion (no new types, no new crates) that uses the existing MeteringEngine::check_provider_budget which already has unit test coverage.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/kernelCore kernel (scheduling, RBAC, workflows)area/runtimeAgent loop, LLM drivers, WASM sandboxbugSomething isn't workingneeds-infoIssue lacks reproduction steps, logs, or version info

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions