Repository navigation
Reconcile divergent 'run' vs 'trace' (and 'span') terminology across TokenOps / Chronicle / control-plane #113
Description
Problem
The three repos use inconsistent vocabulary for the same concepts, and the one place they are tied together is about to be deleted (see #55, #60, and the tokenops.server removal work).
| Concept | TokenOps | Chronicle | control-plane |
|---|---|---|---|
| Top-level unit of work | run (run_id, RunRegistration/Record/State, tokenops_run) |
trace (trace_id, begin_trace) |
both, unjoined — runs / run_registrations / ledger_* and traces / envelopes, separate schemas in one SQLite file, no FK |
| One boundary crossing | step (BoundaryStep) / observation (Observation) |
envelope (Envelope, envelope_id) |
envelopes row |
span_id |
per agent-hop id, minted new_id("span") in attribution.py — one per agent per run |
alias for envelope_id — one per crossing (every LLM/tool call) |
stores envelope_id, indexes parent_envelope_id |
| parent link | parent_span_id / header X-TokenOps-Parent-Span-Id |
parent_span_id = parent_envelope_id |
parent_envelope_id |
| ordering | RunState.window list order |
sequence (int), invocation_index |
sequence column |
| custom attributes | user_dims, tags, boundary_tags, RunRecord.dims (4 names) |
dims (flat str→str; trace-level copied onto every span) |
dims on runs and traces and envelopes; trace search indexes dims.{session_id,message_id,user_id} (Chronicle conventions, not TokenOps') |
Divergences that bite
run!=trace, coupled by one line.run_id == trace_idis set only inmount_run_registration(src/tokenops/control/http.py):reset_session().begin_trace(reg.run_id). The control-plane'sregister_rundeliberately drops this ("Chronicle session coupling intentionally omitted — agents own their traces").Envelope.trace_idotherwise defaults to a freshuuid4(). Aftertokenops.server/mount_run_registrationis removed there is no guaranteed join key between ledger rows (run_id) and envelope rows (trace_id).- "span" spans two granularities. TokenOps span = a whole agent invocation (coarse, header-propagated). Chronicle span = one boundary crossing (fine, = envelope). A TokenOps span contains many Chronicle spans;
X-TokenOps-Parent-Span-Idandparent_envelope_idare different trees. - One event, three names: Chronicle
Envelope-> tokenopsObservation(crossing hook) ->BoundaryStep(Ledger.record, adds price + cum_spent). No doc names the pipeline. - control-plane "one file" doesn't join —
store.pyandenvelope_store.pyshare no id.
Desired outcome
- One canonical id that spans the ledger and the envelope stream. Options:
- A — plane mandates
trace_id == run_idat registration (re-add the coupling plane-side). Simplest; blocks a child agent having its own trace. - B (preferred) — stamp
run_idas a dim on every envelope (Chronicle already propagates trace-level dims down); plane indexesdims.run_id; join =envelopes.dims.run_id -> runs.run_id. Keepstrace_idindependent, one extra index. - C — never join (rejected; blocks deriving the ledger from the envelope stream).
- A — plane mandates
- Naming: rename the coarse per-hop id to
agent_span/hop_id; reserve "span" for the envelope/OTel level. Collapse the four attribute names where possible. (Overlaps Standardize policy names across code/YAML/UI/docs #53.) - Docs: name the
Envelope -> Observation -> BoundarySteppipeline indocs/architecture.md; add a glossary mapping the three vocabularies.
Must not be dropped NOW (cheap, in the decoupling PR)
When mount_run_registration / tokenops.server is removed, preserve the run_id <-> trace binding — whatever registers the run (plane, or SDK on the entry hop) must either call begin_trace(run_id) or stamp run_id into the Chronicle session dims. 1–3 lines; do not merge the removal without it.
Everything else here (schema, indexes, join queries, UI correlation, renames) is an inexpensive later rewrite and does not block the HTTP hard-dependency work.
Activity
susheem-k commentedon Sep 10, 2026
Update from the remote-only planning discussion: TokenOps will drop its own
span_identirely (no minting, noX-TokenOps-Parent-Span-Idheader, stripspan_id/parent_span_idfromSpanContext/Observation/BoundaryStep, stop writingruns.parent_span). It drives no ledger keying and no policy decision — it only ever duplicated Chronicle's envelope chain.That removes the "span means two things" half of this issue. "span" becomes unambiguously the Chronicle envelope. What's left here is just: give the
run_id-keyed ledger and thetrace_id-keyed envelope stream a shared join key — option B: stamprun_idas a dim on every envelope and index it.Cross-hop linkage in TokenOps is now
X-TokenOps-Run-Idonly.