Repository navigation
Description
Context
TokenOps caps spend for a whole agent workflow and enforces the cap before every model call. The pieces this issue touches:
- run: one whole workflow, not one request. It shares a
run_idand one ledger.tokenops_run()opens or joins a run. wrap_complete(): wraps the model call. It is the enforcement point.- Ledger: records spend and holds the halt flag. Amounts are
Micros, integer micro-USD ($1.00 == 1_000_000). - Governor: runs the policies (for example
cost_budget) built for the run. Halt: raised to abort a run. It extendsBaseExceptionon purpose (control/core.py:246) so a framework'sexcept Exceptioncannot swallow it.
TokenOps does not record spend itself. It learns what a call cost from Chronicle, the record-and-replay library it depends on. The order inside one wrap_complete call (control/integration.py):
governor.pre_call(request)(:267): checks the budget. Before the provider is contacted.governor.ledger.admit(seg)(:275): marks a call in flight.traced(...)(:283):tracediswrap_llm(boundary_id, dispatch)(:251). Chronicle calls the provider, records the crossing, then calls the session hookon_crossing.crossing.on_crossing(control/crossing.py:134) callsemit_observation(:171), which callsGovernor.observe(control/boundary.py:132).observecallsledger.record(obs)(control/engine.py:311): this is the only place spend is written.if controls.retry(:284): retry decision, using what step 4 recorded.
This is deliberate. The module docstring says the observation comes only through the hook, "do not also emit_observation here (double-billing)" (integration.py:14).
Problem
Setting CHRONICLE_ENABLED=0 silently turns off budget enforcement.
Chronicle documents that variable as a way to disable LIVE recording for A/B timing runs (Chronicle README.md:633). When it is off, Chronicle's wrappers return before step 3's hooks fire (chronicle/boundary.py:147, :171), so step 4 never happens and the ledger never sees any spend.
Repro (setup for reg, attr, run_scope and the run's ledger entry is as in tests/test_llm_boundary_precall.py; budget cap 100 micros; the fake provider costs 4000 micros per call under the test price function toy_price in tests/conftest.py:24, which charges 10 micros per input token and 30 per output token; 5 calls through wrap_complete):
cfg = {"governance": {"budgets": [{"id": "b", "limit_micros": 100, "dimension": "run"}],
"policies": {"cost_budget": {"budget": "b"}}}}
controls = ApplyControls()
gov = build_governor(cfg, toy_price, controls)
dispatch = lambda p, m, messages, max_output_tokens=None, **kw: ModelResponse(
content="x", input_tokens=100, output_tokens=100)
governed = wrap_complete(gov, controls, attr, provider="openai", model="gpt-4o-mini",
dispatch=dispatch, service="agent")
with run_scope(reg, SpanContext(span_id="s", service="agent")), \
_governance_scope(gov, attr, provider="openai", model="gpt-4o-mini"):
for _ in range(5):
governed("openai", "gpt-4o-mini", [{"role": "user", "content": "hi"}])Observed:
CHRONICLE_ENABLED unset: model calls made=1, halted=Halt, ledger cost_micros=4000
CHRONICLE_ENABLED=0: model calls made=5, halted=None, ledger cost_micros=0
What the user sees versus what is happening. They see nothing: no exception, no log line, no Halt, five successful model calls. What is happening is that pre_call still runs, but it compares a cap against a ledger that stays at zero, so it always passes. The spend was real (20,000 micros against a 100 micro cap) and is not recorded anywhere.
Blast radius. Every provider, and every policy that reads spend from the ledger (cost_budget is the one the repro exercises). Anyone who sets the variable in a shell or .env file for a Chronicle benchmark, then runs a governed agent, gets no enforcement. I verified this by running wrap_complete. I did not run wrap_stream (it has the same wrap_llm shape, integration.py:392) or the hook-only path where an @boundary(kind="llm") is governed by crossing.on_enter (crossing.py:57); by reading chronicle/boundary.py the second one never reaches on_enter at all when the switch is off.
Nothing in the code marks this as deliberate: the docstring at integration.py:14 says spend arrives through the hook, and does not say what happens when the hook cannot fire.
Proposed approach
Refuse to start a governed run when the hooks TokenOps depends on cannot fire. Fail at the start of the run, not on the first call.
# control/run.py, top of tokenops_run(), next to install_crossing_hook() (run.py:175)
import chronicle
if not chronicle.is_enabled():
raise GovernanceUnavailable(
"CHRONICLE_ENABLED is off. TokenOps records spend through Chronicle's on_crossing "
"hook, so no spend would be recorded and budgets would not be enforced. "
"Unset CHRONICLE_ENABLED to run governed."
)GovernanceUnavailableis a newRuntimeErrorsubclass exported fromtokenops.control. It is not aBaseExceptionlikeHalt: it is raised on enteringwith tokenops_run():before any agent code runs, so nothing can swallow it.- Add the repro above as a regression test in
tests/usingmonkeypatch.setenv("CHRONICLE_ENABLED", "0"), asserting the raise. Add a second test asserting no raise when the variable is unset. - Document the requirement in
docs/architecture.mdnext to the description of the crossing hook.
What stays the same: with CHRONICLE_ENABLED unset or on, nothing changes. No new config, no migration.
Open question (recommendation attached). Should there be an opt-out for someone who deliberately wants to run without accounting? I recommend no opt-out. That case is already covered by not calling tokenops_run, and an opt-out flag would be the way an accidental .env value goes unnoticed again.
This is the TokenOps half of a two-part fix. The root cause is in Chronicle, where one switch turns off both recording and interceptor hooks: theagentplane/chronicle#60. When that lands, this guard should be relaxed to a check that hooks are registered rather than that Chronicle is enabled.
Tradeoffs / non-scope
- Accepted: anyone who had set
CHRONICLE_ENABLED=0alongsidetokenops_runwill now get an error instead of an unenforced run. That is the desired outcome, since those runs had no cap. - Accepted: the guard runs once at the start of a run. If the variable is flipped mid-run (
is_enabled()reads the environment on every call), it will not catch that. I judge that not worth a per-call check. - Rejected: have
wrap_completerecord its own spend so the ledger no longer needs the hook. It would fix thewrap_completepath, but it reverses the single-observation rule atintegration.py:14, needs a second skip mechanism in the hook to avoid double-billing, and leaves the hook-only path unenforced. Whether TokenOps should own its accounting is a design discussion that belongs in its own issue. - Out of scope: the Chronicle-side change (CHRONICLE_ENABLED=0 also disables on_enter/on_crossing/on_leave, so switching off recording silently switches off consumers built on the hooks chronicle#60). Calls that never enter the governed wrapper at all (Fail-closed when a run has no governed dispatch, instead of silently running ungoverned #109, which is about raw
completebypassing governance, a different cause with the same symptom of silent non-enforcement). The two governance paths doing different things (hook versuswrap_complete): needs a new issue.
Activity
Yashika-1705 commentedon Sep 26, 2026
Hi @susheem-k, I'd like to work on this!
I can implement the proposed fail-closed check:
GovernanceUnavailablefromtokenops.controlchronicle.is_enabled()check at the start oftokenops_run()CHRONICLE_ENABLED=0and the default unset casedocs/architecture.md