-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
fix: session write lock race — async release can delete newly-acquired lock #57019
Copy link
Copy link
Closed
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:data-lossCan lose, corrupt, or silently drop user/session/config data.Can lose, corrupt, or silently drop user/session/config data.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:data-lossCan lose, corrupt, or silently drop user/session/config data.Can lose, corrupt, or silently drop user/session/config data.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Session write lock race: release can destroy a freshly acquired lock
Summary
In
src/agents/session-write-lock.ts,releaseHeldLockdeletes the session key from the in-memoryHELD_LOCKSMap (line 149) before starting the async file cleanup (handle.close()+fs.rm(), lines 150-161). This opens a race window where a concurrentacquireSessionWriteLockfor the same session can acquire a new lock, only for the old release's deferredfs.rmto delete the new lock file out from under it.Affected platforms
getProcessStartTime()returnsnullon non-Linux platforms, so lock files never containstarttime. This meansshouldTreatAsOrphanSelfLock(line 389) always reaches its!HELD_LOCKS.has()check for same-PID contention./proc/<pid>/statis unreadable (containers with restricted/proc, or unusual security policies), since a validstarttimein the payload causesshouldTreatAsOrphanSelfLockto returnfalseearly (line 398-399).Step-by-step race scenario
Precondition: Process P holds a write lock on session S. Lock file exists at
S.jsonl.lockwithpid = P.pidand nostarttime(macOS/Windows).release()for session S — entersreleaseHeldLock.HELD_LOCKS.delete(normalizedSessionFile)executes synchronously. The Map no longer has session S.handle.close()begins but has not resolved.acquireSessionWriteLockfor session S (e.g., reentrant agent writing to the same transcript). A microtask or concurrent call from the same event loop tick.fs.open(lockPath, "wx")fails withEEXIST— the old lock file still exists on disk.readLockPayloadreads the old lock file. Payload haspid = process.pid, nostarttime.shouldTreatAsOrphanSelfLockis called:pid === process.pid— true (same process)hasValidStarttime— false (macOS, no/proc)!HELD_LOCKS.has(normalizedSessionFile)— true (deleted at step 2)reclaimDetailsis constructed withstale: trueforced and"orphan-self-pid"added to reasons.shouldReclaimContendedLockFilereturnstrue(stale is true, reasons include"orphan-self-pid"which doesn't match the mtime-fallback filter).fs.rm(lockPath, { force: true })— deletes the old lock file.continuere-enters the loop.fs.open(lockPath, "wx")succeeds — new lock file created, written, stored inHELD_LOCKS.handle.close()completes, then line 157:fs.rm(held.lockPath, { force: true })executes — deleting the new lock file.Result: Process P believes it holds the session write lock (entry exists in
HELD_LOCKSwith a valid handle), but the lock file no longer exists on disk. Any other process can nowopen("wx")the same path and obtain a concurrent "lock" on the same session.Impact
.jsonlsession file, interleaving or overwriting each other's JSON lines.HELD_LOCKSentry still exists, the handle is still open (pointing at a now-unlinked inode on Linux, or a deleted-but-cached file on macOS/Windows).HELD_LOCKSentries but doesn't verify the lock file still exists on disk.Suggested fixes
Option A — "releasing" guard set (minimal change):
Add a
RELEASING_LOCKS: Set<string>alongsideHELD_LOCKS. InreleaseHeldLock, add the key toRELEASING_LOCKSbefore deleting fromHELD_LOCKS, and remove it fromRELEASING_LOCKSafter the asyncfs.rmcompletes. InshouldTreatAsOrphanSelfLock, returnfalseifRELEASING_LOCKS.has(normalizedSessionFile).Option B — move delete after async IO (simpler but changes semantics):
Move
HELD_LOCKS.delete(normalizedSessionFile)to after thefs.rmcompletes (inside the async IIFE, after line 157). This keeps the entry visible inHELD_LOCKSduring the entire cleanup, soshouldTreatAsOrphanSelfLockreturnsfalseand the acquirer waits/retries normally.The downside is that
HELD_LOCKS.sizeremains non-zero during cleanup, which keeps the watchdog running slightly longer, but this is harmless.Option B is simpler and more correct — the lock should be considered "held" until cleanup actually finishes, not just until cleanup starts.