feat(extension): add cost-guard — budget enforcement and cost alerts#17449
feat(extension): add cost-guard — budget enforcement and cost alerts#17449miloudbelarebia wants to merge 4 commits intoopenclaw:mainfrom
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (status.level === "exceeded" && config.hardStop) { | ||
| return { | ||
| prependContext: [ | ||
| "[Cost Guard] BUDGET EXCEEDED.", | ||
| `Daily limit of $${status.dailyLimit.toFixed(2)} reached ($${status.dailyUsed.toFixed(2)} used).`, | ||
| "Responses are blocked until the budget resets.", | ||
| ].join(" "), | ||
| }; | ||
| } |
There was a problem hiding this comment.
Exceeded message always references daily limit
The "exceeded" status can be triggered by the monthly limit (monthlyUsed >= config.monthlyBudgetUsd) or a per-provider limit, but this message always says "Daily limit of $X reached". When only the monthly budget is exceeded (e.g. daily spend is $2 with a $5 daily budget, but monthly spend is $50 with a $50 monthly budget), the user sees a misleading message.
The same issue applies to the warning block below (lines 48-55), which only references dailyPercent and daily budget even though the warning may have been triggered by monthlyPercent >= config.warningThreshold.
Consider checking which limit was actually exceeded and showing the appropriate message, e.g.:
const dailyExceeded = dailyUsed >= config.dailyBudgetUsd;
const monthlyExceeded = monthlyUsed >= config.monthlyBudgetUsd;
const limitMsg = dailyExceeded
? `Daily limit of $${status.dailyLimit.toFixed(2)} reached ($${status.dailyUsed.toFixed(2)} used).`
: `Monthly limit of $${status.monthlyLimit.toFixed(2)} reached ($${status.monthlyUsed.toFixed(2)} used).`;
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/cost-guard/index.ts
Line: 38:46
Comment:
**Exceeded message always references daily limit**
The `"exceeded"` status can be triggered by the monthly limit (`monthlyUsed >= config.monthlyBudgetUsd`) or a per-provider limit, but this message always says "Daily limit of $X reached". When only the monthly budget is exceeded (e.g. daily spend is $2 with a $5 daily budget, but monthly spend is $50 with a $50 monthly budget), the user sees a misleading message.
The same issue applies to the warning block below (lines 48-55), which only references `dailyPercent` and daily budget even though the warning may have been triggered by `monthlyPercent >= config.warningThreshold`.
Consider checking which limit was actually exceeded and showing the appropriate message, e.g.:
```
const dailyExceeded = dailyUsed >= config.dailyBudgetUsd;
const monthlyExceeded = monthlyUsed >= config.monthlyBudgetUsd;
const limitMsg = dailyExceeded
? `Daily limit of $${status.dailyLimit.toFixed(2)} reached ($${status.dailyUsed.toFixed(2)} used).`
: `Monthly limit of $${status.monthlyLimit.toFixed(2)} reached ($${status.monthlyUsed.toFixed(2)} used).`;
```
How can I resolve this? If you propose a fix, please make it concise.| pruneTimer = setInterval(() => { | ||
| tracker.pruneOldEntries(); | ||
| }, PRUNE_INTERVAL_MS); |
There was a problem hiding this comment.
Missing .unref() on prune interval timer
This setInterval will keep the Node.js event loop alive indefinitely, preventing clean process shutdown even after stop() is called on other services (if stop() on this service isn't reached). The nostr extension's seen-tracker.ts handles this correctly by calling pruneTimer.unref(). Consider adding the same here:
| pruneTimer = setInterval(() => { | |
| tracker.pruneOldEntries(); | |
| }, PRUNE_INTERVAL_MS); | |
| pruneTimer = setInterval(() => { | |
| tracker.pruneOldEntries(); | |
| }, PRUNE_INTERVAL_MS); | |
| pruneTimer.unref(); |
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/cost-guard/src/service.ts
Line: 74:76
Comment:
**Missing `.unref()` on prune interval timer**
This `setInterval` will keep the Node.js event loop alive indefinitely, preventing clean process shutdown even after `stop()` is called on other services (if `stop()` on this service isn't reached). The `nostr` extension's `seen-tracker.ts` handles this correctly by calling `pruneTimer.unref()`. Consider adding the same here:
```suggestion
pruneTimer = setInterval(() => {
tracker.pruneOldEntries();
}, PRUNE_INTERVAL_MS);
pruneTimer.unref();
```
How can I resolve this? If you propose a fix, please make it concise.
extensions/cost-guard/src/tracker.ts
Outdated
| checkBudget(config: CostGuardConfig): BudgetStatus { | ||
| const dailyUsed = sumTotal(todayEntries()); | ||
| const monthlyUsed = sumTotal(monthEntries()); | ||
| const todayProviders = sumByProvider(todayEntries()); | ||
| const monthProviders = sumByProvider(monthEntries()); |
There was a problem hiding this comment.
Redundant full-array iteration in checkBudget
checkBudget calls todayEntries() and monthEntries() independently for both sumTotal and sumByProvider, resulting in 4 separate filter+iterate passes over the entire entries array. The same issue exists in summary() (lines 217-225) which also makes 4 passes. Under high-volume usage, this becomes a performance bottleneck called on every diagnostic event (line 60 of service.ts).
Consider computing today/month entries once and reusing them:
const today = todayEntries();
const month = monthEntries();
const dailyUsed = sumTotal(today);
const monthlyUsed = sumTotal(month);
const todayProviders = sumByProvider(today);
const monthProviders = sumByProvider(month);
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/cost-guard/src/tracker.ts
Line: 155:159
Comment:
**Redundant full-array iteration in `checkBudget`**
`checkBudget` calls `todayEntries()` and `monthEntries()` independently for both `sumTotal` and `sumByProvider`, resulting in 4 separate filter+iterate passes over the entire `entries` array. The same issue exists in `summary()` (lines 217-225) which also makes 4 passes. Under high-volume usage, this becomes a performance bottleneck called on every diagnostic event (line 60 of `service.ts`).
Consider computing today/month entries once and reusing them:
```
const today = todayEntries();
const month = monthEntries();
const dailyUsed = sumTotal(today);
const monthlyUsed = sumTotal(month);
const todayProviders = sumByProvider(today);
const monthProviders = sumByProvider(month);
```
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.
|
Clean approach — piggybacking on the existing diagnostic event stream keeps the surface area small. One pattern worth flagging from production: the highest-cost failure mode isn't steady-state overspend — it's retry amplification. A single failed LLM call with large context can retry 3–4x before any One lightweight option might be checking an estimated cost immediately before dispatch (even a rough input_tokens × model_rate calculation). If the estimate exceeds remaining budget, skip the call and emit a diagnostic rather than letting the retry loop run. Also worth thinking about concurrent runs — two executions could pass the budget check at the same time and both dispatch. Even a minimal guard around the remaining balance would prevent that edge case without changing the overall structure. What's the intended recovery path when |
Add a new extension that monitors API costs in real time via diagnostic events and enforces daily/monthly budget limits. - Service listens to model.usage events via onDiagnosticEvent() - In-memory tracker accumulates spend per provider, per day/month - before_agent_start hook injects budget warnings into agent context - message_sending hook blocks responses when budget exceeded (hard stop) - /cost slash command shows current spend and budget status - Configurable: daily/monthly limits, warning threshold, per-provider caps Addresses community feedback about unexpected high API costs. Related: openclaw#12299, openclaw#8485, openclaw#14779, openclaw#9812 Co-Authored-By: Claude Opus 4.6 <[email protected]>
- Show correct limit in exceeded/warning messages: detect whether daily, monthly, or per-provider budget triggered the status and display the appropriate message instead of always showing daily - Add pruneTimer.unref() to prevent the interval from keeping the Node.js event loop alive after other services stop - Cache todayEntries()/monthEntries() results in checkBudget() and summary() to avoid redundant array iterations (4 → 2 passes) Co-Authored-By: Claude Opus 4.6 <[email protected]>
CI was failing because the lockfile was not updated after adding the cost-guard extension workspace. Ran pnpm install to sync. Co-Authored-By: Claude Opus 4.6 <[email protected]>
The plugin validation system scans all extensions and requires each to have an openclaw.plugin.json manifest. Without it, the config.plugin-validation tests fail because loadPluginManifest() returns an error diagnostic, causing the entire validation to report ok=false. Co-Authored-By: Claude Opus 4.6 <[email protected]>
miloudbelarebia
force-pushed
the
feat/cost-guard-extension
branch
from
e03f62a to
ff888b5
Compare
3 participants
Summary
cost-guardthat monitors API costs in real time and enforces budget limitsmodel.usagediagnostic events viaonDiagnosticEvent()(same pattern asdiagnostics-otel)before_agent_starthook injects budget warnings into agent context when approaching limitsmessage_sendinghook blocks responses when budget exceeded (configurable hard stop)/costslash command shows current spend and budget statusMotivation
Multiple community members have reported unexpected high API costs:
The codebase has excellent cost tracking infrastructure (diagnostic events, session-cost-usage) but zero enforcement — costs are calculated and emitted but never checked against any budget.
This extension fills that gap by adding a lightweight enforcement layer via the existing plugin system.
Configuration
Files
extensions/cost-guard/index.tsextensions/cost-guard/package.jsonextensions/cost-guard/src/config.tsextensions/cost-guard/src/tracker.tsextensions/cost-guard/src/service.tsextensions/cost-guard/src/format.tsChange Type (select all)
Scope (select all touched areas)
Linked Issue/PR
User-visible / Behavior Changes
New
/costcommand available when extension is enabled. Budget warnings injected into agent context. Optional hard stop when budget exceeded.Security Impact
No security impact. Extension only reads diagnostic events (read-only) and uses standard plugin hooks. No new API keys, no external network calls, no file system writes.
Repro + Verification
cost-guardextension in config/costto verify spend trackingEnvironment
tsc --noEmitoxfmt --checkHuman Verification (required)
src/plugins/types.tsdiagnostics-otelpattern for service + event subscriptionmemory-lancedbpattern for config schema + hooks + commandsRisks and Mitigations
session-cost-usage.ts. The tracker only needs real-time data for enforcement.Test plan
/costcommand returns formatted outputLocal Validation
pnpm check(tsgo): ✅ passespnpm test: ✅ passes (including plugin validation tests)oxfmt --checkScope
New extension at
extensions/cost-guard/— 7 files, self-contained, no modifications to core codebase.AI Assistance
AI-assisted (Claude Code) for codebase exploration, pattern discovery, and drafting. The architecture decisions (diagnostic event subscription, budget enforcement via hooks), implementation, and Greptile review fixes are my own work.
Testing level: Fully tested — all CI checks pass (check, bun tests, node tests, windows tests).
Greptile Summary
This PR adds a new
cost-guardextension that provides real-time API cost monitoring and budget enforcement. The extension subscribes tomodel.usagediagnostic events, tracks spending in-memory per provider/day/month, and uses lifecycle hooks to inject warnings and block responses when budgets are exceeded.Key changes:
extensions/cost-guard/with 7 files (index, service, tracker, config, format, package.json, plugin manifest)before_agent_starthook injects budget warnings into agent context at configurable threshold (default 80%)message_sendinghook blocks responses when budget exceeded (optional hard stop)/costslash command shows current spend and budget status with provider breakdowndiagnostics-otel(event subscription) andmemory-lancedb(config schema)Implementation quality:
.unref()to avoid blocking process shutdown (line 77 of service.ts)The extension is self-contained with no modifications to core codebase. All CI checks pass including TypeScript compilation, formatting, and tests.
Confidence Score: 5/5
Last reviewed commit: e03f62a