Build a single ThreadPoolExecutor per workflow run when the host passes no
executor instead of one per DAG wave: create it lazily on the first wave
that runs, dispatch each wave with a flat map (flat_dispatch) so scheduling
matches the historical executor-less path, and shut it down in finally
before stream-pipeline close so wave tasks drain in the same order the
per-wave with-blocks did. Host-provided pools keep batched dispatch and are
never built or shut by the engine, so the HTTP server and streaming
pipeline are unchanged.

Worker threads live for exactly one run. Framework context stays isolated
per task through context snapshots and the unconditional rebinds in
safe_execute_step; third-party threading.local set inside one step may be
seen by later steps of the same run (the reuse profile hosted pools have
had since roboflow#1717), and cannot cross runs because the threads are destroyed
at run end.

On a six-round counterbalanced benchmark, pool constructions fall from the
wave count to one and median run latency drops from 1.10 to 0.71 ms on a
10-step chain (-35%) and from 5.08 to 3.01 ms on a 50-step chain (-41%),
with single-wave, host-provided, and compute-heavy cells unchanged within
noise. The 16 regression tests fail on unpatched base code with 10
failures (pool counts, dispatch shape, shutdown ordering, early-
termination teardown) and pass here; suites report 6599 passed, 113
skipped (workflows package) and 1238 passed, 30 skipped (root server
workflows).

Build a single ThreadPoolExecutor per workflow run when the host passes no
executor instead of one per DAG wave: create it lazily on the first wave
that runs and dispatch each wave flat (flat_dispatch - submit the whole
wave, collect results) so scheduling matches the historical executor-less
path. A step error drains in-flight siblings before leaving
run_steps_in_parallel, exactly where the per-wave with-blocks used to
join, and the pool shuts down in a nested finally before stream-pipeline
close so pipeline cleanup still runs if shutdown itself raises.
Host-provided pools keep batched dispatch and are never built or shut by
the engine, so the HTTP server and streaming pipeline are unchanged.

Worker threads live for exactly one run. Framework context stays isolated
per task through context snapshots and the unconditional rebinds in
safe_execute_step; third-party threading.local set inside one step may be
seen by later steps of the same run (the reuse profile hosted pools have
had since roboflow#1717), and cannot cross runs because the threads are destroyed
at run end.

On a six-round counterbalanced benchmark, pool constructions fall from the
wave count to one and median run latency drops from 1.10 to 0.71 ms on a
10-step chain (-35%) and from 5.08 to 3.01 ms on a 50-step chain (-41%),
with single-wave, host-provided, and compute-heavy cells unchanged within
noise. The 17 regression tests fail on unpatched base code with 11
failures (pool counts, dispatch shape, shutdown ordering,
early-termination teardown) and pass here; suites report 6607 passed, 113
skipped (workflows package) and 1238 passed, 30 skipped (root server
workflows).

Build a single ThreadPoolExecutor per workflow run when the host passes no
executor instead of one per DAG wave: create it lazily on the first wave
that runs and dispatch each wave flat (flat_dispatch - submit the whole
wave, collect results) so scheduling matches the historical executor-less
path. A step error drains in-flight siblings before leaving
run_steps_in_parallel, exactly where the per-wave with-blocks used to
join, and the pool shuts down in a nested finally before stream-pipeline
close so pipeline cleanup still runs if shutdown itself raises.
Host-provided pools keep batched dispatch and are never built or shut by
the engine, so the HTTP server and streaming pipeline are unchanged.

Worker threads live for exactly one run. Framework context stays isolated
per task through context snapshots and the unconditional rebinds in
safe_execute_step; third-party threading.local set inside one step may be
seen by later steps of the same run (the reuse profile hosted pools have
had since roboflow#1717), and cannot cross runs because the threads are destroyed
at run end.

On a six-round counterbalanced benchmark, pool constructions fall from the
wave count to one and median run latency drops from 1.10 to 0.71 ms on a
10-step chain (-35%) and from 5.08 to 3.01 ms on a 50-step chain (-41%),
with single-wave, host-provided, and compute-heavy cells unchanged within
noise. The 17 regression tests fail on unpatched base code with 11
failures (pool counts, dispatch shape, shutdown ordering,
early-termination teardown) and pass here; suites report 6607 passed, 113
skipped (workflows package) and 1238 passed, 30 skipped (root server
workflows).

…uns (#3052)

Build a single ThreadPoolExecutor per workflow run when the host passes no
executor instead of one per DAG wave: create it lazily on the first wave
that runs and dispatch each wave flat (flat_dispatch - submit the whole
wave, collect results) so scheduling matches the historical executor-less
path. A step error drains in-flight siblings before leaving
run_steps_in_parallel, exactly where the per-wave with-blocks used to
join, and the pool shuts down in a nested finally before stream-pipeline
close so pipeline cleanup still runs if shutdown itself raises.
Host-provided pools keep batched dispatch and are never built or shut by
the engine, so the HTTP server and streaming pipeline are unchanged.

Worker threads live for exactly one run. Framework context stays isolated
per task through context snapshots and the unconditional rebinds in
safe_execute_step; third-party threading.local set inside one step may be
seen by later steps of the same run (the reuse profile hosted pools have
had since #1717), and cannot cross runs because the threads are destroyed
at run end.

On a six-round counterbalanced benchmark, pool constructions fall from the
wave count to one and median run latency drops from 1.10 to 0.71 ms on a
10-step chain (-35%) and from 5.08 to 3.01 ms on a 50-step chain (-41%),
with single-wave, host-provided, and compute-heavy cells unchanged within
noise. The 17 regression tests fail on unpatched base code with 11
failures (pool counts, dispatch shape, shutdown ordering,
early-termination teardown) and pass here; suites report 6607 passed, 113
skipped (workflows package) and 1238 passed, 30 skipped (root server
workflows).

Co-authored-by: Paweł Pęczek <[email protected]>