Skip to content

Commit efc20e7

Browse files
authored
Merge pull request #6538 from lidge-jun/codex/spend-capacity-guard
fix(spend): refuse unbooked dispatches under applicable ceilings
2 parents dd9a980 + c46febd commit efc20e7

10 files changed

Lines changed: 171 additions & 31 deletions

File tree

‎docs-site/src/content/docs/ja/reference/configuration/server.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,3 +199,7 @@ This takes effect on the **next routed `ocx claude` launch**, injecting `CLAUDE_
199199
Claude Code **2.1.257 or newer** is required for FORCE. Plugin and built-in agents (including Explore/Plan) and per-call model arguments are overridden. Forks and subagent skills with `model: inherit` keep the main conversation model. The main loop and Haiku/small-fast sidecars are unaffected. Existing roster files remain available.
200200

201201
The dashboard warns about old or unknown CLI versions, unavailable targets, and either variable already present in `settings.json` → `env` (which overrides launch env). Detection is read-only and server-local: it cannot inspect another launch shell, another machine, or project-local settings. An unknown result is not proof of force support.
202+
203+
## トークン予約と上限
204+
205+
`spend.root.maxTokens`、`spend.identity.maxTokens`、`spend.pool.maxTokens` のいずれかがリクエストに適用される場合、追跡容量の不足などでトークン予約を記録できなければ送信を拒否します。適用される上限がないリクエストは観測のみを続けます。

‎docs-site/src/content/docs/ko/reference/configuration/server.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,3 +256,7 @@ This takes effect on the **next routed `ocx claude` launch**, injecting `CLAUDE_
256256
Claude Code **2.1.257 or newer** is required for FORCE. Plugin and built-in agents (including Explore/Plan) and per-call model arguments are overridden. Forks and subagent skills with `model: inherit` keep the main conversation model. The main loop and Haiku/small-fast sidecars are unaffected. Existing roster files remain available.
257257

258258
The dashboard warns about old or unknown CLI versions, unavailable targets, and either variable already present in `settings.json` → `env` (which overrides launch env). Detection is read-only and server-local: it cannot inspect another launch shell, another machine, or project-local settings. An unknown result is not proof of force support.
259+
260+
## 토큰 예약과 한도
261+
262+
요청에 `spend.root.maxTokens`, `spend.identity.maxTokens`, `spend.pool.maxTokens` 중 하나가 적용되면 추적 용량 부족 등으로 토큰 예약을 기록할 수 없을 때 전송을 거부합니다. 적용되는 한도가 없는 요청은 계속 관측만 합니다.

‎docs-site/src/content/docs/reference/configuration/server.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ runs helper features around provider requests.
2929
| `usageLedgerMaxBytes?` | `number` | unset | Opt-in ceiling in bytes for `usage.jsonl`. Absent means the request history grows without limit, which stays the default. See [usage history size](#usage-history-size). |
3030
| `appOwnedMemoryBudgetMb?` | `number` | `256` | Cap in MiB for evictable app-owned logs, caches, blobs, and continuation payloads. Range 64–4096; not an RSS cap. |
3131
| `metricsExport.enabled?` | `boolean` | `false` | Enable process-local aggregate request metrics at authenticated `GET /api/metrics`. Restart required; disabled mode returns 404 and starts no exporter activity. |
32-
| `spend?` | `{ root?: { maxTokens?: number }; identity?: { maxTokens?: number }; pool?: { maxTokens?: number }; retentionDays?: number }` | unset | Durable token ceilings, off unless you write one. Each scope bounds settled spend plus in-flight reservations plus unresolved spend: `root` is one task including its whole fan-out, `identity` is one account across every task it serves, and `pool` is one provider pool. They intersect, so a request is admitted only when all three have room — which is what holds a ceiling against a client that mints a new task id per request. A reservation is the request's whole input plus its enforceable output ceiling, counted as if every cached prefix misses. Observe-only mode still journals, so every server owns the state directory's single-writer lease; an explicit sibling must use a separate `OPENCODEX_HOME`. Spend survives an ordinary process restart when its writes reached the filesystem, but the journal does not promise survival across host power loss because each append is not fsynced. Raising or removing the value is what grants more. `maxTokens` must be a positive integer (0 would refuse everything), `retentionDays` is 1–365 and defaults to 7, and an unknown key in this section is rejected rather than ignored. A refusal is a local HTTP 429 carrying `x-opencodex-local-refusal: workflow_spend_exhausted`, and its message names the scope and the ceiling; no provider is contacted. |
32+
| `spend?` | `{ root?: { maxTokens?: number }; identity?: { maxTokens?: number }; pool?: { maxTokens?: number }; retentionDays?: number }` | unset | Durable token ceilings, off unless you write one. Each scope bounds settled spend plus in-flight reservations plus unresolved spend: `root` is one task including its whole fan-out, `identity` is one account across every task it serves, and `pool` is one provider pool. They intersect, so a request is admitted only when all three have room — which is what holds a ceiling against a client that mints a new task id per request. A reservation is the request's whole input plus its enforceable output ceiling, counted as if every cached prefix misses. Observe-only mode still journals, so every server owns the state directory's single-writer lease; an explicit sibling must use a separate `OPENCODEX_HOME`. Spend survives an ordinary process restart when its writes reached the filesystem, but the journal does not promise survival across host power loss because each append is not fsynced. Raising or removing the value is what grants more. `maxTokens` must be a positive integer (0 would refuse everything), `retentionDays` is 1–365 and defaults to 7, and an unknown key in this section is rejected rather than ignored. A refusal is a local HTTP 429 carrying `x-opencodex-local-refusal: workflow_spend_exhausted`, and its message names the scope and the ceiling; no provider is contacted. With an applicable ceiling, dispatch is also refused if its token reservation cannot be booked, including full tracking capacity. Requests without an applicable ceiling remain observe-only. |
3333

3434
| `codexAutoStart?` | `boolean` | `true` | Let the Codex shim run `ocx ensure` before launching Codex. False makes ensure a no-op. |
3535
| `codexShimAutoRestore?` | `boolean` | `true` | Restore an installed shim after a completed external Codex update replaces it. Environment opt-out: `OPENCODEX_CODEX_SHIM_AUTO_RESTORE=0`. |

‎docs-site/src/content/docs/ru/reference/configuration/server.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,3 +247,7 @@ This takes effect on the **next routed `ocx claude` launch**, injecting `CLAUDE_
247247
Claude Code **2.1.257 or newer** is required for FORCE. Plugin and built-in agents (including Explore/Plan) and per-call model arguments are overridden. Forks and subagent skills with `model: inherit` keep the main conversation model. The main loop and Haiku/small-fast sidecars are unaffected. Existing roster files remain available.
248248

249249
The dashboard warns about old or unknown CLI versions, unavailable targets, and either variable already present in `settings.json` → `env` (which overrides launch env). Detection is read-only and server-local: it cannot inspect another launch shell, another machine, or project-local settings. An unknown result is not proof of force support.
250+
251+
## Резервирование токенов и лимиты
252+
253+
Если к запросу применяется `spend.root.maxTokens`, `spend.identity.maxTokens` или `spend.pool.maxTokens`, отправка отклоняется, когда резерв токенов нельзя учесть, в том числе из-за заполнения хранилища отслеживаемых записей. Запросы без применимого лимита остаются в режиме наблюдения.

‎docs-site/src/content/docs/zh-cn/reference/configuration/server.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,3 +213,7 @@ This takes effect on the **next routed `ocx claude` launch**, injecting `CLAUDE_
213213
Claude Code **2.1.257 or newer** is required for FORCE. Plugin and built-in agents (including Explore/Plan) and per-call model arguments are overridden. Forks and subagent skills with `model: inherit` keep the main conversation model. The main loop and Haiku/small-fast sidecars are unaffected. Existing roster files remain available.
214214

215215
The dashboard warns about old or unknown CLI versions, unavailable targets, and either variable already present in `settings.json` → `env` (which overrides launch env). Detection is read-only and server-local: it cannot inspect another launch shell, another machine, or project-local settings. An unknown result is not proof of force support.
216+
217+
## 令牌预留与额度限制
218+
219+
如果请求受 `spend.root.maxTokens`、`spend.identity.maxTokens` 或 `spend.pool.maxTokens` 限制,无法记录令牌预留时会拒绝发送,包括跟踪容量已满的情况。没有适用额度限制的请求仍保持仅观察模式。

‎scripts/test-layout/layout.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1552,6 +1552,7 @@
15521552
"responses-snapshot-repair.test.ts": "responses",
15531553
"responses-sparse-terminal-tool-scope.test.ts": "responses",
15541554
"responses-spend-ledger-wiring.test.ts": "responses",
1555+
"responses-spend-capacity-guard.test.ts": "responses",
15551556
"responses-spill-acl-recovery.test.ts": "responses",
15561557
"responses-spill-inspection.test.ts": "responses",
15571558
"responses-spill-orphan-sweep.test.ts": "responses",

‎src/server/responses/request-spend.ts‎

Lines changed: 30 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
import { randomUUID } from "node:crypto";
22
import type { RequestSendObserver } from "../../lib/request-execution-budget";
3-
import { sharedSpendLedger, type SpendReservationLedger } from "../../lib/spend-reservation-ledger";
3+
import { sharedSpendLedger, type SpendReservationLedger, type SpendScopes } from "../../lib/spend-reservation-ledger";
44
import { SpendLedgerOwnerError } from "../../lib/spend-ledger-owner";
55
import { markLocalRequestLogRefusal, type RequestLogContext } from "../request-log";
6-
import { recordWorkflowRefusalEvent, workflowDenialSummary } from "../../lib/workflow-budget";
6+
import { recordWorkflowRefusalEvent, workflowDenialSummary, type WorkflowDenial } from "../../lib/workflow-budget";
77

88
/** The terminal usage a request reported, in the only two fields the ledger books. */
99
export interface TerminalSpendUsage {
@@ -81,44 +81,44 @@ export function createRequestSpendTracker(
8181
// short of its limit forever and refuse nothing.
8282
const alreadySent = options?.alreadySent === true;
8383
const sendId = randomUUID();
84-
const decision = ledger().reserve({
84+
const bookedLedger = ledger();
85+
const scopes: SpendScopes = {
86+
...(rootId !== undefined ? { rootId } : {}),
87+
// The existing privacy-safe log label is aliased again by the ledger.
88+
...(logCtx.accountLogLabel !== undefined ? { identityId: logCtx.accountLogLabel } : {}),
89+
...(logCtx.provider !== undefined ? { poolId: logCtx.provider } : {}),
90+
};
91+
const policy = bookedLedger.policy;
92+
const enforced = (scopes.rootId !== undefined && policy.root.maxTokens !== undefined)
93+
|| (scopes.identityId !== undefined && policy.identity.maxTokens !== undefined)
94+
|| (scopes.poolId !== undefined && policy.pool.maxTokens !== undefined);
95+
const decision = bookedLedger.reserve({
8596
sendId,
86-
scopes: {
87-
...(rootId !== undefined ? { rootId } : {}),
88-
// Already the privacy-safe label the request log uses, and the ledger aliases it
89-
// again on the way to disk. A raw credential never reaches either.
90-
...(logCtx.accountLogLabel !== undefined ? { identityId: logCtx.accountLogLabel } : {}),
91-
...(logCtx.provider !== undefined ? { poolId: logCtx.provider } : {}),
92-
},
97+
scopes,
9398
inputTokens: logCtx.spendInputEstimateTokens ?? logCtx.usageLogInputTokens ?? 0,
9499
outputCeilingTokens: logCtx.spendOutputCeilingTokens ?? 0,
95100
...(alreadySent ? { alreadySent: true } : {}),
96101
});
97102
if (!decision.reserved) {
103+
// An applicable ceiling cannot authorize an unbooked send, including when tracking
104+
// capacity is full. Unconfigured/nonapplicable requests remain observe-only, and a
105+
// physical send reported after dispatch cannot be refused retroactively.
106+
if (alreadySent || !enforced) return true;
98107
refusals += 1;
99108
const denial = decision.denial;
100-
// A ceiling refuses, and so does a ledger that cannot make the reservation durable
101-
// under one. That second case is the whole reason this store is on disk: admitting a
102-
// send whose record a restart would forget is how an exhausted budget comes back with
103-
// a fresh allowance, and the ledger raises those two denials ONLY when a limit is
104-
// configured -- so an install that configured nothing is still never refused here.
105-
// Capacity and a duplicate send id stay permissive: they say the ledger cannot account
106-
// for this send, which is a degradation to report, not an outage to cause.
107-
if (denial.reason === "reserve-not-durable" || denial.reason === "journal-corrupt") {
108-
return alreadySent;
109-
}
110-
if (denial.reason !== "spend-limit-exceeded") return true;
111-
// This send is refused, and the dispatch path that asked will report an exhausted send
112-
// budget -- from there, that is all it can see. The row is where an operator actually
113-
// looks, so the ceiling is named on it here: a locally assigned code wins in
114-
// addFinalRequestLog, so the request that CROSSED the ceiling reads as a spend refusal
115-
// rather than as the ordinary budget exhaustion it would otherwise be indistinguishable
116-
// from. The event ring gets the same pair so /api/workflow-budget agrees with the row.
117-
const detail = { scope: denial.scope, limit: denial.limit, projected: denial.projected };
118-
const summary = workflowDenialSummary("workflow-spend-exhausted", detail);
109+
const reason: WorkflowDenial = denial.reason === "duplicate-send-id"
110+
? "workflow-send-replayed"
111+
: denial.reason === "reserve-not-durable" || denial.reason === "journal-corrupt"
112+
? "workflow-spend-undurable"
113+
: denial.reason === "tracking-capacity-exhausted"
114+
? "workflow-tracking-exhausted"
115+
: "workflow-spend-exhausted";
116+
const detail = denial.reason === "spend-limit-exceeded"
117+
? { scope: denial.scope, limit: denial.limit, projected: denial.projected } : undefined;
118+
const summary = workflowDenialSummary(reason, detail);
119119
markLocalRequestLogRefusal(logCtx, summary.code);
120120
logCtx.errorCode = summary.code;
121-
recordWorkflowRefusalEvent(rootId, "workflow-spend-exhausted", Date.now(), detail);
121+
recordWorkflowRefusalEvent(rootId, reason, Date.now(), detail);
122122
return false;
123123
}
124124
live.push(sendId);

‎structure/transports/responses-spend.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,14 @@ so a second restart has nothing to redo.
130130
booking, the settlement split, the refund, a ceiling that refuses a dispatch rather than
131131
describing it afterwards, and the restart.
132132

133+
The request tracker checks the current policy against the exact root, identity and pool scopes
134+
on each charge. With an applicable ceiling, any refused booking prevents a new dispatch,
135+
including full tracking capacity or a duplicate send id; the log uses the existing specific
136+
workflow refusal reason. Requests without an applicable ceiling remain observe-only. Reports
137+
of sends that already left stay permissive and do not count as local refusals; this guard does
138+
not promise complete accounting when those post-dispatch bookings fail.
139+
`tests/responses/responses-spend-capacity-guard.test.ts` covers these boundaries.
140+
133141
The default policy still sets no token ceiling on any scope, so an unconfigured install accounts
134142
and reports without refusing. An operator turns enforcement on with the `spend` section in
135143
config.json, which `src/lib/spend-reservation-ledger.ts` resolves through

‎tests/fixtures/test-layout-expected.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1564,6 +1564,7 @@
15641564
"responses-snapshot-repair.test.ts": "responses",
15651565
"responses-sparse-terminal-tool-scope.test.ts": "responses",
15661566
"responses-spend-ledger-wiring.test.ts": "responses",
1567+
"responses-spend-capacity-guard.test.ts": "responses",
15671568
"responses-spill-acl-recovery.test.ts": "responses",
15681569
"responses-spill-inspection.test.ts": "responses",
15691570
"responses-spill-orphan-sweep.test.ts": "responses",

0 commit comments

Comments
 (0)