Repository navigation
Commit 4067004
authored
feat(spend): require one writer lease per state directory for the spend journal (#5157)
* feat(spend): require one writer lease per state directory for the spend journal
The ledger documented a guarantee covering one live proxy process, while the CLI
deliberately supports starting a sibling on a different port under the same state
directory. Nothing serialized the two. That is not a theoretical overlap: every
Responses-serving process is a potential journal writer, including one with no
configured ceiling, because a ceiling decides whether to refuse rather than
whether to write. Two processes could mint different salts, interleave appends
and compaction, and a second process's construction replay declares the first
one's open reservations lost while it is still settling them.
Startup now takes a state-directory writer lease before any listener can serve,
so the exported startServer path is covered rather than only the CLI preflight,
and the ledger asserts that lease before replay, append or compaction. Siblings
remain supported with independent state directories; a second writer on the same
one is refused with a clear message.
Ownership is an OS-held SQLite write transaction for the process lifetime, which
is the pattern this repository already uses for its other cross-process locks.
There is deliberately no stale-owner reclamation: PID reuse, container PID
namespaces, restarts and power loss all defeat a PID or timestamp check, and a
TTL can evict a live process that was merely paused. Busy means a live owner;
any other failure to establish the lock is ambiguous and fails closed.
Diagnostics are scalar only on the authenticated health route - ownership,
initialized, configured, degraded and bounded counters - and reading them
constructs nothing. /healthz is unchanged.
* fix(spend): bind journal mutations to a live lease, not to the handle
Pre-publication review found three gaps in the writer lease.
The guard sat where the shared ledger was handed out, so a caller holding an
existing ledger or journal facade could keep mutating after the final release,
once another process legitimately owned the home. Ownership is now asserted at
the mutations themselves: every reservation state change, every file-backed
append and compaction rewrite, and salt creation. A retained handle refuses to
write, which the new negative case pins with a second owner in place.
Two differently configured homes could alias one journal or salt through a link
while holding separate owner databases, so both could write the same file. The
backing files must now be regular, single-link, self-owned files; links, owner
files and unusable owner databases are refused. This is the cooperative
configuration contract, not a claim of isolation against a hostile same-UID
process.
The unsupervised restart starts its replacement before the parent exits. With a
zero busy timeout and every owner error terminal, the child would exit busy while
the parent still owned the home, and the parent would then exit leaving nothing
serving. The deferred restart child now carries a one-use parent marker, accepted
only when it matches its actual parent, and waits a bounded five seconds for
ownership. An ordinary sibling still fails closed immediately.
* fix(spend): release the ledger singleton with the lease that owned it
The binding outlived ownership, so a process that served one state directory and
then legitimately served another was refused for a conflict it no longer had.
Every test process that starts servers against per-test homes is that shape, and
so is a restart handoff inside one process.
Releasing the last lease now discards the in-memory ledger along with the
binding. Nothing is lost: the journal on disk is the durable record and the next
construction replays it, which is what a restart already does. Two homes owned
at the same time are still refused, which is the invariant that matters.
* fix(spend): prove exact ownership on every accounting read and change
Review found a retained ledger coming back to life. The guard compared the state
directory, which cannot tell "still the ownership I was built under" from "the
same directory, owned again since" - so a handle kept across a release and a
reacquire resumed with totals from before the gap, over a journal another writer
may have appended to. Ownership now has an identity: each acquisition that takes
the lock mints one, the ledger captures it at construction, and every accounting
read and change proves that exact value. A handle from the previous ownership is
refused; a fresh handle replays what happened in between.
Reads are covered as well as writes, because reporting figures from a journal
this process no longer owns is the same error with a quieter symptom.
The file-backed journal and salt writers no longer accept an absent owner check.
The parameter is required, so a caller that owns its own temporary file writes
that decision down instead of inheriting it by omission.
A link whose target does not exist read as "no file" through existsSync, so the
safety check was skipped and the append created that target elsewhere. Entry
presence is now decided with lstat, which sees the link itself.
* fix(spend): make ownership proof something only the owner module can mint
The previous round kept adding checks at each caller, and the review was right
that this converged on one boundary rather than a set of gaps. A required guard
supplied by the caller proves nothing, because the caller can supply one that
does nothing - which is exactly what the fixtures here did.
Production journal and salt storage is now minted by the owner module from the
directory it actually owns. The mint takes a file name rather than a path, so
nothing outside chooses the destination; the returned value carries the exact
ownership it was minted under and re-proves it on every read, append, rewrite,
salt read and salt create; and a brand means a look-alike object is refused on
identity rather than on shape. There is no longer an exported entrypoint that
writes a caller-chosen path.
Every ledger member now proves ownership too, including knows and the policy,
degraded, persistFailures and corruptRecords getters. Reporting a figure from a
journal this process no longer owns is the same error as writing one.
The generic in-memory and injected factory stays usable without any of this and
does not pretend to enforce process ownership. The file-journal cases that cover
real persistence, hardening and compaction now take a real lease over a throwaway
state directory and go through the production entrypoints, rather than being
replaced by a stand-in.
Also corrects the reserve request shape in the reacquire regression to scopes
with input and output-ceiling tokens, and the stale comment claiming production
never discards the singleton.
* fix(spend): key ownership proof on identity, not on a marker the token carries
A marker on the object was still forgeable. Spreading a real token copies its
own symbol, so { ...minted, path: elsewhere, assert() {} } passed the brand check
and then ran the caller's replacement guard against the caller's path - the same
no-op guard the required-callback version allowed, reached another way.
The token now carries nothing at all. Its path and the ownership it was minted
under live in a module-private table keyed by the token's identity, which a copy
cannot reproduce because a copy is a different object. Callers ask this module
for the path and for the check rather than reading either off the value they were
handed.
The negative case covers all three shapes: a bare look-alike, a hand-built object
with the right fields, and a spread of a genuine token with the path redirected
and the guard replaced. It also asserts the redirected file was never created,
and that a genuine token still works so the refusal is about identity rather
than refusing everything. A separate case pins that a real token stops working
once its ownership ends.
* fix(spend): take ownership on demand instead of refusing to serve
Hosted CI showed the guard was asserting the wrong rule. Requiring ownership to
touch the journal is right; requiring it to have been taken by startServer is
not, and handleResponses is an equally supported entry point that never calls
startServer - 72 test files exercise exactly that shape, and an embedder can too.
The result was a 502 wherever a turn reached its first physical send without a
server having started.
The shared ledger now takes the lease itself when nobody holds one. The guarantee
becomes "no unowned writer" rather than "no writer outside one entry point", and
it is not weakened: a directory another process owns is refused exactly as
before, and the call fails closed with it. The lease lives as long as a server's
would and returns through the same release path.
Applying a policy no longer demands ownership either. Recording a value touches
no journal; only reconfiguring a ledger that already exists does, because that
ledger is a live view of an owned directory.
* fix(spend): drop the implicit lease and own the directory in the fixtures instead
The on-demand lease was wrong twice over. Its premise - that handleResponses is
an equally supported entry point - does not hold: the package exports only the
root, src/index.ts exports startServer and not this handler, and every production
caller is server-internal beneath that lease. A direct import of an internal
module in a test is not a public contract. It also leaked: the lease took its own
reference, so server.stop left one behind, blocking final cleanup and the
sequential directory switch the design allows.
Ownership is required again, with no exception carved for callers that skipped
startServer. The cases that dispatch without a server now take the real lease
through a shared helper and release it after each case, so the production rule is
exercised rather than relaxed for tests.
The exact-token, startup and restart boundaries are unchanged.
* test(spend): order the lease release ahead of each fixture's cleanup
Hook registration order differed between the two fixtures, so neither FIFO nor
LIFO execution could guarantee the lease closed before the directory holding it
was removed - a failed removal on Windows, an unlinked live database on POSIX.
The helper now returns an idempotent release and each fixture calls it first in
its own teardown, so the ordering is stated where it matters instead of inferred
from where a hook happened to be registered.
Adds the same treatment to the Claude native-affinity fixture, which CI showed
reaching the ledger through combo dispatch.
* test(spend): let a failed lease release fail the case
The release swallowed whatever lease.release() threw. A rollback or close that
fails is a defect in the thing under test, and hiding it leaves a green run over
a lease that never let go - the exact state the single-writer rule exists to
prevent. The reset stays in finally so the next case still starts from a
discarded singleton.
* test(spend): own the state directory in the six fixtures CI named
Every dispatch charges the ledger: reserveDispatch consults the spend observer
before it books, so any case that reaches prepareAdapterExchange touches the
shared journal. These six call the internal handler directly and never take the
lease startServer takes, so the ledger refused and the cases saw 502/529 instead
of their own contract.
Each one takes the real lease at the end of its own beforeEach, after its home is
in place, and releases it at the top of its own afterEach, before that home is
removed. Ordering is stated at the call site because hook registration order
differs between these fixtures and is a contract in neither direction.
Diagnosed from the hosted shard logs at 654bcee, not guessed: claude-inbound-
cache-stabilize (shard 1), main-account-hard-lock-auth (shard 2), reserve-auth-
context and reserve-dispatch (shard 3), github-copilot-account-origin and
kiro-auth-context-continuation (shard 4). Shards stop at the first failed batch,
so this is a partial view by construction and the remaining fixtures are being
inventoried rather than discovered one run at a time.
* test(spend): own the state directory in the merged send-budget count rows
These rows arrived with #5152 after this branch was cut, so no CI run has ever
seen them against the ownership rule. Every one of them dispatches through the
handler directly and therefore charges the shared ledger.
The lease is taken per row, not per file. Two rows install their own
OPENCODEX_HOME and the rest inherit the preload sandbox, and a lease is bound to
the directory in effect when it was taken, so a block-level hook would either
bind the wrong directory or conflict with the rows that swap one in. The two
own-home rows drop it inside their existing finally, ahead of the environment
restore and the directory removal.
The file's afterEach drops it as a backstop. Without that, a row that throws
mid-assertion leaves the lease behind and the next row's different home reports
an ownership conflict instead of the failure that actually happened.
Local checks: NOT RUN.
* test(spend): take the writer lease in every fixture that dispatches directly
Thirty-one fixtures call a response handler without going through startServer,
so none of them held the spend-journal writer lease and every dispatch came back
as "Spend-ledger ownership is required before the shared ledger can be used".
CI could only ever show a handful of these at a time, because a shard stops at
its first failed batch, so the set was derived by tracing the dispatching call
blocks rather than by re-running until the next one appeared.
Placement is stated at each call site, never inferred from hook order:
- A fixture that installs its own OPENCODEX_HOME takes the lease after that
assignment and drops it before the environment is restored and the directory
removed. An open lease inside a directory being deleted fails the removal on
Windows and leaves an unlinked live database on POSIX.
- A fixture that inherits the preload sandbox home takes the lease at the
dispatch and drops it in the file's afterEach, so a row that throws
mid-assertion cannot leave a lease behind and make the next row's different
home read as an ownership conflict.
- Files that mix both shapes get neither a file-level nor a block-level lease.
Each dispatching case owns its own.
Cases that never reach a physical dispatch are deliberately untouched:
admission refusals, management-API rows, in-memory ledger rows and direct
persistence rows. No assertion, expected value, mock, fixture or timeout is
changed anywhere in this commit, and no file-size cap moves.
Five fixtures sit at exactly their ratchet cap and are NOT in this batch:
openai-responses-passthrough, responses-compaction-routing, usage/request-log,
responses-custom-tool-repair and responses-undeclared-tool-guard. A cap only
moves down, so they cannot take an additive lease and need an extraction first.
That is its own change rather than blank-line churn smuggled into this one.
Local checks: NOT RUN.
* test(spend): finish the lease migration and remove the image suite's own home
Twenty more fixtures dispatch directly and inherit the preload sandbox home, so
each takes the writer lease at its dispatch and drops it in its own afterEach.
Three files that had no afterEach get one whose only job is that drop: a case
that throws mid-assertion would otherwise leave the lease behind and make the
next case's home read as an ownership conflict instead of reporting its own
failure.
Cases that never reach a physical dispatch stay lease-free, and the boundary is
drawn at the seam rather than by file. v2-agent-message-failfast keeps its bare
post() for the two rows that assert dispatch never happens and routes the rest
through a dispatchPost() that takes the lease; abort-race leaves the build-time
abort and buildRequest-throw rows alone; opencode-go-session-header leaves the
policy-fallback rows, whose injected runCore answers without an adapter.
The image activation suite also now removes the home it owns. Taking the lease
creates the state directory, and that suite names a fresh one per run, so
before this it left a directory behind on every run. Release, then remove, then
restore: the removal has to happen while OPENCODEX_HOME still names it.
No assertion, expected value, mock, fixture or timeout changes, and no cap
moves. Local checks: NOT RUN.
* test(spend): make room for the lease in the five fixtures at their cap
A file-size cap only ever moves down, so these five could not take an additive
lease. The repository answer to that is an extraction, not deleted blank lines
or two statements on one line, and each one here removes more than the lease
costs.
Four move a helper to a sibling module, verbatim, so the cases that called it
read the same behaviour through a different name: the SSE stream reader and
builder out of the undeclared-tool guard (nine files had written their own copy
of readAll), the request-log row builder, the key-auth URL builder out of the
Responses passthrough, and the config/request/upstream fixtures out of
compaction routing.
responses-custom-tool-repair is split instead, because its twelve dispatching
cases are one contiguous block and no helper in it is worth enough lines. They
move to responses-custom-tool-repair-dispatch.test.ts unchanged, registered in
both scripts/test-layout/layout.json and tests/fixtures/test-layout-expected.json,
with the fixtures both halves need in a shared helper so the two copies cannot
drift. The original keeps the cases that drive the compat functions directly and
no longer dispatches at all, so it needs no lease.
Compaction routing takes the lease per dispatching describe rather than per file:
five of its cases install a home inside the case body, and each of those drops
the block lease and takes one for its own directory, then drops that before the
directory is removed. The one describe that only exercises a pure function takes
no lease.
One placement is worth naming. In the passthrough file the shared call arrow
takes the lease inside its body, not beside it: beside it the acquire would run
while the describe was being collected, and the first case's teardown would drop
it for every case after.
No assertion, expected value, mock, fixture or timeout changes, and no cap moves.
Local checks: NOT RUN.
* test(spend): release each turn's body before the lease that served it
Review found cases that dispatched, asserted on headers or on what the upstream
stub recorded, and then discarded the Response without touching its body. The
body is a live stream, so the case finished with a reader still attached and the
lease was dropped underneath it. That is the state the single-writer rule exists
to keep from happening, and it is invisible until something downstream reports a
pending handle instead of the assertion that actually failed.
Each of these now consumes or cancels the body after its assertions and before
it returns: the annotation and DeepSeek inbound drive helpers, the two FastWire
characterization paths, both service-tier paths, and the six image-bridge cases
that only ever read a header. The 400 case is untouched because it already reads
its body.
No assertion, expected value, mock, fixture or timeout changes. Local checks: NOT RUN.
* test(spend): read the gateway attempt row's body before its lease is released
The row asserts on the request-log metadata only, so the turn's body was still
attached when the finally released the lease that served it. Reading it keeps
the case's own assertions untouched and leaves nothing pending behind them.
Bounded fixture hygiene. Nothing about the production body lifetime changes, and
no claim is made that the static buffer leaks on its own.
Local checks: NOT RUN.
* test(spend): repair the three failures the first hosted run found
Hosted CI at 4de35b4 reported three distinct problems, and only one of them was
a missing lease.
The Lab activation guard was the serious one. Its SERVE_ANCHOR is the literal
text of the statement that creates the listener, and this branch assigns that
listener from spendLedgerLifecycle.track(Bun.serve(...)) so the lease can roll
the listener back on a failed start. The anchor stopped matching, indexOf
returned -1, and the guard failed loudly rather than measuring an empty string,
which is exactly what it was written to do. The anchor now names the real
statement, so it starts the same window it always did, and the new body-level
call is registered for review rather than skipped: track binds the listener's
stop, records a rollback closure and returns the same server, with no await. The
lifecycle's release() is not registered because it is called inside the async
stop wrapper, which this scan skips as a nested function. Nothing in the guard is
relaxed.
The second was a regression this branch introduced. main-account-hard-lock-auth
took the lease in a file-level beforeEach, and two of its describes spawn a child
that runs startServer against the same home. The child then failed with
SPEND_LEDGER_OWNER_BUSY, which is the ownership contract working correctly and
the case failing for a reason it is not about. The lease is now taken only by the
two cases that dispatch in-process. No other fixture in this batch spawns a
child; that was checked rather than assumed.
The third was an inventory gap. The first pass searched for handleResponses and
its compact and policy-fallback siblings, and missed handleChatCompletions and
handleNativeChatCompletions, so the chat fixtures were never leased. Eight more
files take it now, and the search is by the full handler set rather than by one
of them.
ws-upstream needed room first: its runtime pin and the two wrappers that bind it
move to a sibling helper, verbatim, because the file was at its cap.
Files whose handler calls stop before a physical dispatch are deliberately
untouched: OAuth admission refusals, unknown-provider routing, the policy
fallback rows with an injected runCore, and the reasoning-envelope rows that
answer 404 for a deliberately absent model.
Assertion counts are unchanged in every file; the deletions in this diff are
re-indentation where a body was wrapped in try/finally. No cap moves.
Local checks: NOT RUN.
* test(spend): lease the four mixed fixtures the transitive audit found
All four run a real server in most of their cases and dispatch directly in a
few, so none of them can take a file-wide lease: the lease is taken by the
cases that dispatch in-process and by nothing else.
cursor-effort-rows takes it per case, inside the try that already stops the mock
upstream, and releases it first in that finally. claude-messages-endpoint takes
it inside invokeMessages, beside the turn-admission lease it already holds, and
releases it in the same finally. chat-completions-endpoint and
server-combo-failover-e2e take it at each direct dispatch and drop it at the top
of the afterEach they already have, ahead of the home restore and removal.
Where a case here also starts a server, the acquire is a second reference on the
same directory rather than a competing owner, so the server keeps serving and
the release only drops the reference this case added. That is different from a
CHILD process, which cannot acquire at all while the parent holds the lease; the
one fixture in that shape was repaired separately.
server-combo-failover-e2e needed room first: it was one line under its cap, so
its five upstream response builders move to a sibling helper, verbatim.
Assertion counts are unchanged in all four files. No cap moves.
Local checks: NOT RUN.
* test(spend): lease the websocket steering path at its shared fixture
The steering rows did not fail with an ownership message. They failed with
'fixture condition timed out' after roughly a second, because the websocket
handler dispatches through the real request path: the turn was refused, the
response.created frame never arrived, and the wait expired naming nothing. Two
dozen rows across three files reported it that way.
The lease is taken where the turn begins, not row by row. beginInjection in the
shared native-injection fixture covers ws-native-result-continuations and
ws-steering-stability, and installInjectionFixture drops it in the afterEach it
already registers. ws-native-steering has its own begin() and hooks and gets the
same treatment there.
responses-snapshot-repair-server has one row that calls the handler directly
while every other row starts a real server, so that row takes the lease itself
and the file drops it in teardown.
Assertion counts unchanged. No cap moves. Local checks: NOT RUN.
* test(spend): fix the audit batch and the two ownership-identity traps
The critical one first: a scripted insertion put takeSpendHome() between a
ternary's condition and its question mark in server-combo-failover-e2e, which is
not valid TypeScript. It now sits above the statement. An AST syntax screen over
every changed file finds no remaining parse error.
Two failures had the same root and neither announced itself as an ownership
problem. The websocket fixtures took the lease in begin() and beginInjection(),
but the failing rows call downstream() and injectionClient() directly, so the
lease moved down to the seam where the client actually dispatches. And the
subagent streaming rows force process.platform to win32 to reach the eager-relay
path; ownership identity lowercases the state directory on win32, so the lease
taken under the real platform stopped matching the directory the dispatch checks
the moment the override landed, and the turn was refused with no terminal at
all. Those rows now retake the lease under the platform they are pretending to
run on and give it back before restoring the descriptor.
Release ordering is corrected wherever teardown can still settle a turn. The
steering fixtures close their synthetic clients, then the upstream sockets, then
run the shutdown hooks, and only then release. The cursor rows stop their fake
upstreams first. The combo suite stops its listeners and flushes response state
first, and still releases before its home is removed. The Claude endpoint case
now holds one lease across both of its turns instead of taking a fresh one per
invocation, and drops it after both upstreams are down.
Six files that dispatch through the shared agent-task-recovery post() helper
take the lease too. One of them installs its own OPENCODEX_HOME inside a
describe, so its lease is taken there rather than at file level, for the same
reason as the platform case: a lease binds the directory in effect when it was
taken. The sparse-JSON snapshot-repair row was still missing one.
No assertion, expected value, mock, fixture or timeout changes; no cap moves.
Local checks: NOT RUN. The syntax screen is a pure AST read and is not a
substitute for hosted typecheck or tests.
* test(codex): give the overlapping successor its own state directory
Both hard-kill cases launch the successor while the owner is still listening, on
purpose: that overlap is what proves the contended snapshot, the denied main
admission, the takeover after the kill and the auth-temp scrub. They shared one
OPENCODEX_HOME, and the spend journal allows one writer per state directory, so
the successor was refused before it ever bound and the case reported a startup
failure instead of the transition it is about.
The successor now gets its own config directory under the same fixture root,
written with the same helper and the same account shape. CODEX_HOME is
unchanged, and the native-main lock, the recovery journal and the vault all
derive from that, so every assertion still observes the same shared native
state. This is the pattern the file already uses for its other deliberately
overlapping child a few cases earlier; it is now a named helper that also
restores the parent's OPENCODEX_HOME after writing.
The launch stays before the kill and the lease stays required. Nothing about the
cross-process ownership contract changes.
Local checks: NOT RUN.
* test(spend): wait for the turns to finish before giving the lease back
Two lifetime gaps the review found, both about teardown racing a turn that is
still accounting.
handler.close only STARTS the websocket pump cancellation. The fixtures closed
their synthetic clients and released immediately, so a reader could still be
settling against a journal nobody owned. Both fixtures now wait, on the bounded
seam they already use, until the socket has dropped its stream cancel and its
native control, then run the shutdown hooks, then release, then restore. The
teardowns are async for that reason.
The combo suite leases every dispatch but its status-only rows never read the
transformed body they get back. The shared helpers now hand each turn through a
tracker, and teardown cancels every body that is still unread and unlocked
before the listeners stop, the response state is flushed and the lease is given
back. Rows that do read their body are unaffected, because a consumed or locked
body is skipped.
Pure in-memory rows that never dispatch are deliberately still lease-free.
Assertion counts unchanged, no cap moves, and an AST syntax screen over every
changed file is clean. Local checks: NOT RUN.
* test(spend): track the wrapper each logged helper returns, not the raw turn
The three logged helpers tracked the response they got from the handler and then
returned a deferred-request-log wrapper built around it. The wrapper locks the
raw body, so teardown skipped the raw as locked and never saw the wrapper at
all: the body that rows actually hold was the one left unread. Each helper now
tracks what it returns.
The cancellation no longer swallows its error. A body that refuses to cancel is
a real defect and fails the case, but only after every other turn and every
listener has still been given its chance to close, so one bad body cannot leave
the rest of the suite holding ports or a lease.
Assertion counts unchanged, no cap moves, AST syntax screen clean.
Local checks: NOT RUN.
* test(routing): narrow the eager-relay platform claim to the relay decision
The four streaming rows reported an empty terminal list and read as a relay
defect. They were not. The eager relay is reachable only on win32 and darwin, so
the rows overwrote process.platform globally to reach it, and that redirects far
more than the relay: every filesystem, ACL and state-directory decision in the
process follows it. The spend-ledger owner lowercases its home on win32, and on
a case-sensitive filesystem the lowercased temp directory is a DIFFERENT
directory, so the send could not be reserved and the turn delivered no terminal
at all. Retaking the lease under the fake platform could not fix that, because
the directory it then owned was not the one the case was using.
The claim is now narrowed to the two calls that actually choose the relay path,
through an internal test seam in the delivery module. No config key reaches it,
and ownership, home casing and the win32 policy itself are untouched.
The rows also say more than they did. The status, the event-stream content type
and the relay path itself are asserted before the callback is inspected, so a
turn that never delivered now says so instead of presenting as a missing
callback, and the legacy-tee and eager-relay variants each prove which path they
actually took rather than assuming the streamMode was honoured.
Assertion coverage grows; nothing is relaxed. AST syntax screen clean, no cap
moves. Local checks: NOT RUN.
* fix(spend): close three failure paths in the ledger's own storage
All three are on paths the happy case never reaches, which is why a green suite
said nothing about them.
A fresh state directory was never claimed. Config ownership refuses to claim a
directory that already has contents, and the owner database lives inside that
directory, so creating the database first left a new home with no owner marker
and no manifest at all. Nothing recorded the database or its sidecars, and a
later uninstall could not remove them. The paths are now registered while the
directory is still empty, and the regression proves an uninstall takes the whole
directory back afterwards.
An entry that could not be inspected read as absent. Only ENOENT means absent;
a permission denial or an I/O error means we do not know, and answering no file
skipped the file-safety assertion and appended to whatever was actually there.
It now refuses, and it refuses in the module's own vocabulary rather than
handing a client the errno and path of a state file. The ledger already treats a
journal it cannot make durable as a degradation rather than an outage, so this
surfaces as reserve-not-durable and does not fail the request.
A failed compaction left its temp behind. The name carries random bytes, so a
validate, harden or rename that threw left a uniquely named file and the next
attempt made another: repeated failures accumulated rather than overwriting one
fixed name. Only that exact temp is removed, only on the failure path, so the
original journal and the primary error both survive.
The failed-start rollback also stopped waiting for its own listeners. Bun's
Server.stop(true) resolves once connections are closed, and the rollback
discarded that promise, so the state directory was handed back while a listener
could still be serving. It now holds the lease until every stop has settled,
while staying synchronous and returning void, because startServer must not
become async. Rollback failures stay contained so the startup error that caused
them is still the one reported.
Not covered by a test: the compaction failure path has no deterministic lever
without an injection seam, so the cleanup is asserted by inspection only.
Local checks: NOT RUN. AST syntax screen clean, no cap moves.
* test(spend): prove the storage and rollback failure paths instead of asserting them
The previous commit fixed three failure paths and shipped one of them with no
test, which is how a failure path stays broken. These are the regressions.
Compaction now has a narrow fault seam for its own filesystem steps, because
there is no portable way to make a validate, harden or rename fail on demand. It
is an internal test contract: no config key reaches it and it defaults to
absent. Four cases drive it. A failure at validate, harden or rename, repeated
three times, leaves no compaction residue and an unchanged journal; a write that
stops partway leaves neither residue nor a truncated journal; and a candidate
name that already belongs to something else is left exactly as it was.
That last case drove a real change. The temp was created by a combined write, so
whether the entry was ours had to be inferred from which error the write threw.
It is now an exclusive open first, so ownership is a fact: EEXIST means the name
is not ours and is never removed, and every failure after the open is cleaned
because the entry is provably ours, including a short write. The descriptor is
closed on the failure path too.
The rollback has its own file now. Two listeners whose stops stay pending prove
the lease is still held after both are asked, still held when only one has
settled, and returned once both have; a listener whose stop rejects proves the
others are still stopped and the directory is still returned. Both also pin the
newest-first order.
failedStartStops is typed as returning void or a promise, which is what the
rollback awaits. The old annotation compiled while statically erasing the
promise; the runtime closure returned it either way, so this corrects the
contract rather than a behaviour.
New file registered in both layout maps. Local checks: NOT RUN.
* test: repair the two source-anchor and platform assumptions CI found
Both are test-side. No production guard moves.
The F4 bind oracle pinned the public serve call by its exact old text, and the
listener is now handed to a lifecycle registrar as it is created. The assertion
is wrapper-aware without giving anything up: the serve call's argument object is
still pinned exactly, so the public bind takes bindHost and nothing else, and
server is still what that call is assigned to, through at most one registrar
call. Both negative assertions about a hardcoded loopback host are unchanged.
The streaming pair claimed win32, and on win32 a turn needing a client rewrite
takes the eager relay unconditionally under #864. So the legacy half could never
be legacy there, and the marker assertion I added last round was reporting that
honestly: status and content type passed, the path was eager either way. The
pair now claims darwin, which is the platform where the configured mode actually
decides: legacy-tee resolves to tee, eager-relay to the eager relay, and both
halves keep their health and terminal-callback coverage. The alternative would
have been to weaken the marker assertion, which would have hidden exactly the
thing it was added to prove.
AST syntax screen clean; the new anchor was checked against the current source
and still rejects a hardcoded 127.0.0.1 bind. Local checks: NOT RUN.
* test(spend): close the three fixture gaps the review named
The startup rollback case asserted ownership was returned in the same turn as
the throw, which only held while the rollback discarded its stop promise. It now
waits for the directory to come back, then reacquires, and awaits the blocking
listener's own stop in its teardown. The assertion that the start actually threw
is unchanged.
The partial-write case was not partial. The fault threw before any byte landed,
so it proved cleanup of an empty file rather than of a short write. The fault
now writes a real prefix into the entry the exclusive create already made, and
asserts the prefix is there, before failing with ENOSPC. That is the residue the
cleanup has to remove, and the journal and the absence of residue are still
asserted across repeated attempts.
The unreadable-entry contract had only a POSIX chmod case, which is skipped on
Windows and proves nothing as root. A narrow stat step on the same internal
fault seam now proves it everywhere, for both EACCES and EIO, with only the
journal's own inspection failing: the salt stays readable and every other
filesystem step is real. It checks the refusal is the module's typed error and
that nothing was reset - no truncation, no new entries, and the salt still
mints. The chmod case stays as real-filesystem evidence where it can run.
Also worth recording: the enforce-target failure on the previous head was a
concurrency cancellation, not a gate refusal.
AST syntax screen clean. Local checks: NOT RUN.
* test: keep websocket teardown failure-safe and drain the store-default turns
Two public review findings, both correct against the current source.
The websocket fixtures awaited a bounded completion wait before the rest of
their teardown. When that wait gave up, the hook stopped there: remaining
sockets stayed open, the shutdown hooks never ran, the writer lease was never
returned and the replaced globals were never restored, so one slow turn poisoned
every case after it. The wait is now inside a try and everything else is in the
finally, in the same order as before. The failure still propagates, and the
comment says why that matters: a wait that expired is not evidence the turn
settled, only that the fixture could not prove it settled. No timeout moved.
The store-default rows ask for a stream and then read only the captured upstream
request, so the turn's own body was left live while the teardown handed back the
lease. They drain it now. An earlier review of mine reported every body as
consumed; that was wrong, and the source is what settles it.
AST syntax screen clean. Local checks: NOT RUN.
* test: finish the websocket teardown so one failure cannot strand the rest
My previous attempt wrapped the whole client loop in one try, which left two
gaps the review caught. A wait that gave up aborted the loop, so every client
after it kept its socket open for the next case to inherit. And the lease
release can itself throw, which took the global restore down with it.
Each step is attempted now and the first failure is kept: every client is closed
and waited on, every upstream socket is closed, the shutdown hooks run, and the
lease is released, each guarded so a failure in one does not skip the others.
Restoring the replaced globals sits in an outer finally, so it happens whatever
else went wrong. The collected failure is thrown afterwards.
That last part is the point: a collected failure still fails the case. It means
the fixture could not prove the turn settled, not that it settled. No timeout
moved and the bounded completion assertion is unchanged.
AST syntax screen clean. Local checks: NOT RUN.
* test(server): fix the three late-batch failures the shard finally reached
These were not new. A shard stops at its first failed batch, so batch 26 only
became visible once the earlier batches passed. All three are fixture-side.
The management-auth ACL case timed out icacls for the state directory as well as
the management token file. Only the token file was ever load-bearing for its
claim, and the assertion that the state's source is "environment" is what proves
which path answered. The directory is hardened with required: true by the
spend-journal owner during startServer, which refuses rather than soft-failing,
because an unverified ACL on a directory holding a secret is not something to
proceed past. The stub is narrowed to the token file and the claim is unchanged.
Activation E overwrote process.platform globally to reach the relay path. That
also changes state-directory identity, which is lowercased on win32 and so names
a different directory on a case-sensitive filesystem, and the harness server's
own writer lease stopped matching. The retry was refused before it could reach
the second account, which is why the case saw acct-pool-a alone. It uses the
narrow relay seam now, so the platform claim reaches the relay decision and
nothing else. Same root cause as the subagent rows, different file.
The combo failover row dispatches through the handler rather than the harness
server, so it takes the lease itself and drains its response before releasing.
No production policy moved. Assertions are unchanged; the drain is the only
addition. Local checks: NOT RUN.
* test(spend): fix two macOS-only failures in my own storage regressions
All four Linux shards passed; macOS found two things Linux could not.
The seam-driven case compared the faulted entry against a path it built itself.
The owned home is the REAL path of the directory, and on macOS the temp root is
a symlink, so the equality never held and the fault silently did nothing: the
read succeeded and the case failed asking why nothing threw. It matches on the
entry name now, which is path-shape independent and still leaves the salt alone.
The chmod case asserted the exact refusal message. Which gate notices first is
platform-dependent: where the directory cannot be traversed at all, the
ownership check cannot resolve it before the entry is ever inspected, so the
refusal arrives from there instead. Both are the same module's refusal, so the
case pins the type and the absence of a leaked path, and the seam-driven case
below it pins the exact message on every platform. That is what the seam was
added for.
Also batched, since the review asked for it conditionally and the condition
holds: client close and the completion wait now have separate guards in both
websocket teardowns. close() runs the production handler, so a throw there would
have skipped that client's wait as well as reporting its own failure.
Local checks: NOT RUN.
* test(spend): make the contention matrix test the modes it names
The acceptance mapping found a labelling gap and it was real. The matrix
iterated holder/contender mode pairs, but contenderMode reached only the test
name and the marker suffix: busyError() acquired with nothing configured, so
the contender had no mode at all. Both rows varied only the holder. No other
fixture covered it either; every other spawned holder in this suite is observe
and none configures a contender.
The contender now records its own policy before it tries to acquire, which is
exactly the state a second instance starts in: configured one way or the other
and not yet a writer. Recording a policy touches no journal while this process
owns nothing, so this adds configuration without adding a second writer.
The matrix is all four combinations. The rule under test is that ownership does
not depend on either side's ceiling, and a matrix missing observe/observe and
enforced/enforced was not testing the claim its names made.
One case is new rather than restored: a ceiling turned on while another process
still owns the directory. It proves the transition changes what would be
refused, never who may write, and that the ownership refusal is identical on
both sides of it.
Every real-process and timeout assertion is unchanged, and no production code
moves. The gate was already unconditional; what was missing was the proof.
Local checks: NOT RUN.1 parent 57b1df7 commit 4067004
132 files changed
Lines changed: 4463 additions & 1434 deletions
File tree
- docs-site/src/content/docs
- fr/reference/cli
- ja/reference/cli
- ko/reference/cli
- reference
- cli
- configuration
- ru/reference/cli
- tr/reference/cli
- zh-cn/reference/cli
- zh-tw/reference/cli
- scripts/test-layout
- src
- cli
- lib
- server
- index
- management
- responses
- structure
- transports
- tests
- adapters
- anthropic
- openai
- claude-integration
- codex-integration
- fixtures
- helpers
- images
- lab
- lib
- oauth
- providers
- cursor
- github-copilot
- kiro
- responses
- routing
- server
- service
- usage
- web-search
- windows
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
26 | | - | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
29 | | - | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
30 | | - | |
31 | | - | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
32 | 34 | | |
33 | 35 | | |
34 | 36 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
0 commit comments