Skip to content

fix(control): refuse NewSession while a turn is running#6077

Merged
SivanCola merged 4 commits into
esengine:main-v2from
SivanCola:fix/new-session-running-guard
Jul 6, 2026
Merged

fix(control): refuse NewSession while a turn is running#6077
SivanCola merged 4 commits into
esengine:main-v2from
SivanCola:fix/new-session-running-guard

Conversation

@SivanCola

Copy link
Copy Markdown
Collaborator

Summary

  • ClearSession (and Compact) refuse to run while a turn is in flight, but NewSession had no such guard. Both the Submit /new path (go c.NewSession()) and the bot gateway invoke it asynchronously, so a mid-turn /new could swap the executor session out from under the run loop's direct session reads and race the turn-end snapshot onto the fresh path.
  • Mirror ClearSession's running guard in Controller.NewSession ("cannot start a new session while a turn is running"). Callers already surface errors: TUI notices SlashNewFailed, Submit notices "new session failed", serve returns the error, desktop propagates to the frontend.
  • The bot gateway cancels the in-flight turn right before rotating, and that cancel is asynchronous — it now waits a bounded window (5s/10ms, same shape as the desktop's waitControllerStopped) for the turn to unwind, and on failure replies 新会话创建失败 instead of claiming a new session started while silently keeping the old one.

Verification

  • New test TestNewSessionRefusesWhileTurnRunning: refusal while running (path unrotated, executor session untouched), successful rotation once the turn stops.
  • go test ./internal/control ./internal/cli ./internal/serve ./internal/bot — all green.

Cache impact

Cache-impact: none — session lifecycle guard only; no provider request, system prompt, or tool schema changes.
Cache-guard: not applicable; existing NewSession/ClearSession controller tests plus the new guard test cover the change.
System-prompt-review: N/A

Follow-up to the /new triage around #5699.

@SivanCola SivanCola requested a review from esengine as a code owner July 6, 2026 07:30
@github-actions github-actions Bot added v2 Go rewrite (1.x) — main-v2 branch, active development agent Core agent loop (internal/agent, internal/control) labels Jul 6, 2026
@SivanCola SivanCola force-pushed the fix/new-session-running-guard branch from 6640ea3 to 544a60e Compare July 6, 2026 08:11
ClearSession and Compact both guard against rotating the session out
from under a running turn, but NewSession did not. Both the Submit
"/new" path (go c.NewSession()) and the bot gateway invoke it
asynchronously, so a mid-turn /new could swap the executor session
while the run loop reads it directly and race the turn-end snapshot
onto the fresh path.

Mirror ClearSession's running guard in NewSession. The bot gateway
cancels the in-flight turn before rotating, and that cancel is
asynchronous — give it a bounded window to unwind (same 5s/10ms shape
as the desktop's waitControllerStopped) and report the failure to the
chat instead of claiming a new session started.
@SivanCola SivanCola force-pushed the fix/new-session-running-guard branch from 544a60e to 395aa3d Compare July 6, 2026 08:20
The running guard checked c.running once at entry, released c.mu, and
only swapped the executor session after Snapshot(). A turn could start
during that snapshot window (running was false at the check) via
runGuarded/RunTurn and then have its live session replaced by
SetSession — the exact race the guard meant to prevent, just moved later.

Replace the point-in-time check with a rotation gate: beginRotation
atomically refuses when a turn is running or another rotation holds the
gate, and stays held (c.rotating) from before Snapshot() through the
swap. runGuarded and RunTurn now also refuse to start while c.rotating,
so turn-start and session-rotation are mutually exclusive in both
directions and the run loop's session reference cannot change under it.

Regression test forces the interleaving deterministically by parking
NewSession inside Snapshot()'s recovery callback, then asserting a
concurrent RunTurn is refused, running stays false, and the session is
not mutated until the rotation completes. Verified it fails without the
c.rotating checks.
@SivanCola

Copy link
Copy Markdown
Collaborator Author

Confirmed the P1 and fixed it in a6f96dc.

The reviewer is right: the entry-time if c.running check was released before Snapshot(), and a turn starting inside that window (running still false) would then have its session replaced by the later SetSession — the guard only closed the "already running at call time" case, not "turn starts mid-rotation".

Fix: a rotation gate instead of a point-in-time read.

  • beginRotation() atomically refuses when c.running OR another rotation is in progress, and sets c.rotating under c.mu; it stays held from before Snapshot() through the swap (NewSession and ClearSession both use it via defer endRotation()).
  • runGuarded and RunTurn now also refuse to start while c.rotating. So turn-start and session-rotation are mutually exclusive in both directions — a rotation can't begin under a live turn, and a turn can't begin under an in-flight rotation. The run loop's session reference can no longer change under it.

Regression test TestNewSessionRefusesTurnStartedDuringSnapshot forces exactly the reported interleaving deterministically: it parks NewSession inside Snapshot()'s recovery callback (still holding the gate), then asserts a concurrent RunTurn is refused with ErrTurnRunning, running stays false, and the session is not mutated until the rotation completes. I verified the test fails without the c.rotating checks (panics at the RunTurn-refused assertion) and passes with them.

Verification:

  • go test ./internal/control ./internal/bot ./internal/cli ./internal/serve ./internal/acp — green
  • go test -race ./internal/control -run 'TestNewSessionRefuses|TestConcurrentSnapshots|TestRunTurn|TestClearSession' — green
  • go vet ./internal/control ./internal/bot — clean

NewSession/ClearSession closed their turn↔swap TOCTOU window with the
rotation gate, but the same bare-Running()-then-mutate-later shape lived
in every other interactive session-mutating entry point:

- forkNamed: Running() check, then Snapshot(), then SetSession
- Branch:    read running, then Snapshot(), then SetSession
- SwitchBranch: read running, then Branches()/LoadSession, then SetSession
- Rewind:    Running() check, then Session().Replace + SnapshotRewrite
- Compact:   Running() check, then CompactNow rewrites the live session

All are dispatched from submit()'s synchronous slash-command path (or
Fork wrappers), so a user can trigger them while a turn streams — the
guard existed for exactly that reason, but a turn starting after the
check and before the mutation slipped through, replacing/rewriting the
session under the run loop.

Route all five through beginRotation()/endRotation(), same as
NewSession/ClearSession: the gate is held from before the snapshot/load
through the swap, and runGuarded/RunTurn already refuse to start while
c.rotating. Each keeps its original "while a turn is running" message on
the running path and reports errRotationInProgress for the newly-closed
window. None of these methods is called from inside a turn body, so the
gate is never re-entered (no deadlock).

TestSessionMutationsRefuseWhileRotating asserts all five (plus RunTurn)
refuse while a rotation holds the gate and the live session is untouched,
and that the gate cannot be claimed while a turn runs. Verified it fails
if the rotating check is removed.
@SivanCola

Copy link
Copy Markdown
Collaborator Author

Extended the fix to close the same TOCTOU window in every other session-mutating entry point (252ff75).

Auditing all live-session mutations (SetSession/Session().Replace/CompactNow) turned up the identical bare-Running()-then-mutate-later shape in five more interactive commands, all dispatched from submit()'s synchronous slash-command path so they can race a streaming turn:

method shape
forkNamed (/branch N, Fork) Running()Snapshot()SetSession
Branch (/branch) read running → Snapshot()SetSession
SwitchBranch (/switch) read running → Branches()/LoadSessionSetSession
Rewind (/rewind) Running()Session().Replace + SnapshotRewrite
Compact (/compact) Running()CompactNow rewrites live session

All five now go through the same beginRotation()/endRotation() gate, held from before the snapshot/load through the swap; runGuarded/RunTurn already refuse while c.rotating. Each keeps its original "cannot X while a turn is running" message on the running path and reports errRotationInProgress for the window this closes.

Safety checks I verified:

  • No re-entrancy / deadlock: none of these is called from inside a turn body — all callers are the slash-command dispatch and Fork wrappers. Their internal Snapshot/Branches/SnapshotRewrite calls touch only snapshotMu, never rotating.
  • Lock order unchanged: beginRotation holds c.mu only briefly to set the flag, then releases; the gate is a flag, not a held mutex, so no interaction with snapshotMu/c.mu ordering.
  • Error shapes preserved: rewindFail wrapping, agent.BranchInfo{} returns, and every user-facing message are byte-identical on the running path.

Regression test TestSessionMutationsRefuseWhileRotating asserts all five (plus RunTurn) refuse while a rotation holds the gate with the live session untouched, and that the gate can't be claimed while a turn runs. Verified it fails when the rotating check is removed.

Verification: go test ./internal/control ./internal/bot ./internal/cli ./internal/serve ./internal/acp green; go test -race ./internal/control green; desktop build+tests green; go vet clean.

SummarizeFrom/SummarizeUpTo still used the bare Running() check before
rewriting the live session — and their window is the widest of the
class: the summarizer performs a provider round-trip before
Session.Replace, leaving seconds for a turn to start and then have the
log replaced under it. Reachable from desktop summarize actions, the
TUI rewind menu, and serve /summarize.

Route summarizeAt through beginRotation()/endRotation() from the
boundary read through the post-rewrite snapshot, keeping the original
"cannot summarize while a turn is running" message on the running path.

Also strengthen TestSessionMutationsRefuseWhileRotating: every gated
method now asserts the errRotationInProgress sentinel instead of any
non-nil error, so each method's wiring is independently
regression-proof (a bare-Running() revert previously slipped past the
weaker checks by failing later for an unrelated reason). Verified the
summarize assertions fail with the gate removed.
@SivanCola

Copy link
Copy Markdown
Collaborator Author

Confirmed and fixed in 203ab34. The finding is correct — and it was the worst instance of the class: summarizeAt rewrites the live session after a provider round-trip, so the unguarded window was seconds wide, not microseconds.

Why my previous audit missed it: I enumerated direct mutation sites in controller.go (SetSession / Session().Replace / CompactNow), but summarize delegates the Session.Replace to the agent layer (Agent.SummarizeFrom/UpTo in internal/agent/compact.go). I have now completed the audit from the opposite direction — every executor method the controller calls, classified by whether it mutates the session:

  • SetSession ×7, CompactNow, SummarizeFrom, SummarizeUpTo — all now behind the rotation gate.
  • PruneStaleToolResults — mutates the session, but its only caller is maybeColdResumePrune on the Resume path (session load, no concurrent turn on this controller), same class as Resume itself.
  • ReplaceTodoState/SeedTodoState/RebuildTodoState — todo sidecar state, not the message log.

So the gated set is now closed over all user-triggerable live-session mutations: NewSession, ClearSession, forkNamed, Branch, SwitchBranch, Rewind, Compact, SummarizeFrom/UpTo.

Fix mirrors the others: beginRotation()/defer endRotation() from the boundary read through the post-rewrite SnapshotRewrite, original "cannot summarize while a turn is running" message preserved on the running path.

Test hardening: your finding also exposed that my TestSessionMutationsRefuseWhileRotating assertions were too weak — several checked only err != nil, which a bare-Running() revert can satisfy by failing later for an unrelated reason (e.g. missing boundary). Every gated method now asserts the errRotationInProgress sentinel specifically, plus the summarize entries and the "cannot summarize while a turn is running" running-path message. Reverse-verified: reverting summarizeAt to the bare check fails the test at exactly the SummarizeFrom assertion.

Verification: go test ./internal/control ./internal/bot ./internal/cli ./internal/serve ./internal/acp green; go test -race ./internal/control over the rotation tests green; desktop tests green; go vet clean.

@SivanCola SivanCola merged commit e38623e into esengine:main-v2 Jul 6, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Core agent loop (internal/agent, internal/control) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant