Repository navigation
Description
Problem
ledger_inflight is a single integer per segment_key:
ledger_admit(segment_key)→count = count + 1ledger_complete(segment_key)→count = MAX(0, count - 1)ledger_inflight(segment_key)→count
(control_plane/store.py ledger_admit / ledger_complete; mirrored in tokenops HttpStore.)
It is read by concurrency_cap at pre_call to decide admit vs reject/queue.
This works today only because every admit/complete is synchronous and serialized
(one governed call at a time per process, SqliteStore serializes on an RLock). It is
not safe once admit/complete are batched, buffered, retried, or a process crashes —
which is exactly the direction of the remote-only / events:batch work.
Failure examples
A. Lost complete on crash → permanent phantom in-flight
- Agent process admits call X →
count = 1. - LLM call returns; process crashes before
completeflushes. countstays1forever.concurrency_cap(max_concurrent=1)now rejects every
future call on that segment, for every process, until someone manually clears it.
MAX(0, count-1)only guards the low side; nothing caps or repairs the high side.
B. Retried batch → double count (no idempotency)
- Client buffers
[admit(seg)], flush POST times out, client retries. - Plane applied it twice →
count = 2for one real call. concurrency_caprejects a call that should have been admitted.
A retriedcompleteis the mirror:countgoes too low, masking a real in-flight call,
soconcurrency_capadmits past the limit.
C. Reordering across a buffer
- Call X
completeis buffered; call Yadmitflushes first. - Between the two flushes, a third process reads
inflightand sees an inflated count.
Bounded by flush interval, but the value is transiently wrong in the unsafe direction
for a reject decision.
D. Idempotency key on a bare counter is ambiguous
To dedupe, the plane must remember, per idempotency_key, whether that key was a +1
or a -1, and refuse to reapply — extra state that a counter does not naturally carry.
Sketch of a fix (for later — NOT this plan)
Model inflight as a set of active call-ids:
admit(segment_key, call_id)→INSERT OR IGNOREcomplete(segment_key, call_id)→DELETEinflight(segment_key)→SELECT COUNT(*)- TTL sweep for call-ids older than N minutes (crash cleanup).
Naturally idempotent, crash-tolerant, and safe to read slightly stale. Alternative:
keep the counter but make complete and reject-mode admit hard flush barriers that
never ride a buffer.
Scope
Explicitly out of scope for the remote-only control-plane plan
(scratch/remote-only-control-plane-plan.md). That plan assumes synchronous
admit/complete for now and only leaves room for batching. This issue tracks making
the inflight model actually safe before that room is used.
Activity