Repository navigation
Commit db1f1bc
committed
fix: keep a one byte anchor so Spark never drops the task entry mid task
Spark's ExecutionMemoryPool removes a task's entry when its balance hits
zero, and an acquire parked inside Spark indexes that entry on wake. The
previous fix held one byte back from a zeroing release while acquires
were in flight and paid it back later. That withheld byte starved
Spark's minimum share check, which grants a parked request only when the
freed bytes cover it in full, so the waiter slept forever; and a release
planned before a late acquire parked could still remove the entry.
The pool now asks for one extra byte with every acquire that starts
before the anchor lands, keeps the first one until the pool drops, and
sends every release whole. The task's balance never returns to zero mid
task, so both failure modes are impossible by construction. The test
double now models ExecutionMemoryPool's entry lifecycle, share rules and
wait loop, and covers the minimum share scenario, the late acquire race
and the in flight anchor race.1 parent eb410e5 commit db1f1bc
1 file changed
Lines changed: 606 additions & 249 deletions
0 commit comments