Repository navigation
Commit 76d9c3d
perf(engine): scale worker-count caps with cpu count
Follow-up to hf#732 commit 74adf6d. The hybrid layered/shader-blend
path saturates well past 6 DOM workers on multi-core hosts, but the
engine's `calculateOptimalWorkers` clamped both the explicit
`--workers N` request and the `auto` default to a hardcoded ceiling of
10 / 6 respectively. On the 96-core validation host, w=8, w=12, w=16
all collapsed to 6-10 internally, masking whether the wall is
algorithmically bound or just cpu-starved by the cap.
Two changes:
* `ABSOLUTE_MAX_WORKERS`: 10 → 24. Explicit `--workers 16` now
surfaces 16 DOM sessions instead of being silently truncated to 10.
The new ceiling is still finite because CDP-protocol dispatch
serializes through Node's main event loop; past ~24 we expect noise
to dominate signal. Holding 24 as a hard cap matches what a
thoughtful operator would have picked manually.
* `DEFAULT_SAFE_MAX_WORKERS` (a constant) →
`defaultSafeMaxWorkers()` returning
`Math.max(6, Math.min(16, Math.floor(cpuCount / 8)))`.
On 8-core: 6 (unchanged). On 16-core: 6. On 32-core: 6. On 64-core:
8. On 96-core: 12. On 128-core: 16. The /8 divisor leaves headroom
for each Chrome worker's SwiftShader compositor + the in-process
shader-blend `worker_threads` pool, both of which are themselves
CPU-heavy. Hard 16-cap caps the bump so a 256-core host does not
spawn enough Chrome processes to OOM the box.
Empirical role:
This change is intentionally a probe. If the new ceiling moves the
shader-render wall meaningfully (108s → lower at w=12), the prior
hf#732 fix was cpu-bound at the old cap and the next bottleneck is
something else. If the wall stays flat across w=4 → w=16, lever 1
confirms the ceiling is genuinely algorithmic (DOM-capture
serialization through Node's main event loop) and lever 2 — the
DOM-capture cost itself — is the next attack surface.
The CPU-scaled `auto` default also matters for production users on
big hosts who never pass `--workers`: they previously left ~90 cores
idle on a 96-core machine.
Tests:
* parallelCoordinator.test.ts: existing 7 tests pass. They cover both
the explicit-request path (capped at `ABSOLUTE_MAX_WORKERS`) and
the auto path. The auto path with `concurrency: 6` in the test
config exercises the explicit branch (`concurrency !== "auto"`),
so the new `defaultSafeMaxWorkers()` is reached only through the
literal-`"auto"` branch which is covered by the engine's existing
default-config integration tests.
— Vai
Co-Authored-By: Vai <[email protected]>1 parent 74adf6d commit 76d9c3d
1 file changed
Lines changed: 18 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
68 | | - | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
69 | 84 | | |
70 | 85 | | |
71 | 86 | | |
| |||
79 | 94 | | |
80 | 95 | | |
81 | 96 | | |
82 | | - | |
| 97 | + | |
83 | 98 | | |
84 | 99 | | |
85 | 100 | | |
| |||
0 commit comments