perf(v8-runtime): parked warm session workers by NathanFlurry · Pull Request #215 · rivet-dev/secure-exec

NathanFlurry · 2026-07-02T18:53:13Z

Pre-created V8 session workers (thread + snapshot-restored isolate, keyed by
snapshot digest + heap limit) are claimed at session create via a warm hint,
taking isolate_new (~4.2ms) and blob_clone off the per-exec critical path.
Workers are never reused across guests (each exec still gets a virgin
isolate); a wrong hint falls back to the existing lazy-create path. Capacity
per key via AGENTOS_V8_WARM_ISOLATES (default 2, 0 disables); refill after
claim; pool transitions logged; warm_worker_hit/miss counters in the session
phases output.

Seeding is deliberately conservative: auto mode stays snapshot-only and only
AGENTOS_WASM_SNAPSHOT_RUNNER=block seeds workers (wasm-runner key + default
node key). Reason: creating isolates on background threads while other
isolates execute guest WebAssembly deterministically SIGSEGVs the pinned V8
130 (WasmCodePointerTable::AllocateUninitializedEntry inside Isolate::New,
even under the isolate lifecycle lock — the race is against V8-internal wasm
threads). Backtraces + notes in the rusty-v8-upgrade todo; enabling default
seeding rides that upgrade.

Warm floors (block, this host): node -e '' 17 -> 14.4ms p50, wasm true
33 -> ~30ms (warm_worker_hit 64/67, isolate_new down to 3 background calls);
pwd 42.6 -> 37.5ms, printf-0b 44.7 -> 40.2ms, date 115 -> 111ms. wasm suite
8.9s (was 96s, pool+snapshot amortize debug isolates); v8-runtime tests and
bench:gate green.

…s via event The sidecar drains process output before emitting process_exited and the frame stream + event pump are FIFO, so once the exit event is observed no trailing output can follow it; the host-side quiet-turn drain (2 turns at 10ms) only remains for the snapshot-poll fallback exit path. Also stop awaiting signal-state refreshes on the exec critical path (start + first output) — kill paths await the in-flight refresh instead — and clean up the parked refresh in finishProcess so fallback exits release it too. Warm host-driven exec p50: node -e '' 30ms -> ~17ms, wasm true 52ms -> 39ms; whole wasm-command-floor lane down ~13ms/row; bench:gate green. New regression test: 10 sequential fast-exit 64KiB stdout captures.

Bake the constant wasm runner + wasi shim (~300KB of JS previously recompiled per exec) into the per-process V8 userland snapshot via guest_runtime.snapshot_userland_code, keyed and cached process-wide. Mode env AGENTOS_WASM_SNAPSHOT_RUNNER=auto|block|off: auto probes the snapshot cache without blocking (async warm kicked once per process) and falls back to the byte-identical inline runner until ready, so the cold path is unchanged. The sync-RPC glue is re-evaluated per exec (the snapshot-baked copy cannot bind session bridge fns). Module bytes are now base64-encoded once per module in a bounded, fingerprint-validated cache (64 entries, warn on evict, debug log reports per-entry and cumulative cache bytes) instead of fs::read+encode per exec. Adds AGENTOS_V8_SESSION_PHASES timers (snapshot_get / blob_clone / isolate_new / user_code_execute) and a measured NOTE against EagerCompile at snapshot build (moves cost to isolate deserialize). Warm wasm floor p50: true 39->33ms, pwd 48->43ms, ls-empty 81->75ms, date-version 127->115ms (52/62/96/140 before PR #212); cold first-exec improved; wasm suite green; bench:gate green. Remaining floor is isolate-per-exec + module decode — pooling tracked as Stage C.

NathanFlurry · 2026-07-02T18:53:19Z

Stack for rivet-dev/secure-exec

fix(wasm): external-command shell redirects write through to their targets #227 msnxyvqr · 2026-07-02 23:29:36
perf(wasi): cut the shim's path_filestat_get cost 10x #226 opqrluul · 2026-07-02 23:07:48
fix(website): pin docs-theme to the published v0.3.1 tag #224 urvqxsnp · 2026-07-02 21:47:26
perf(http2): retain loop rides the net_socket event push #223 zzqtkzpn · 2026-07-02 21:47:25
perf(net): event-driven loopback socket wakes #221 kpyplspp · 2026-07-02 21:28:07
test(sidecar): fs.promises metadata semantics coverage #220 vrsntkur · 2026-07-02 21:07:22
feat(benchmarks): full ecosystem command rows (spec 4.1) #219 soqurlqr · 2026-07-02 20:34:14
perf(fs): readdir raw fast path + stream writev coalescing (JS lane) #218 xlprnvwv · 2026-07-02 20:29:28
fix(sidecar): JS-lane reads see kernel-VFS state #217 nxztyxvs · 2026-07-02 19:47:03
perf(execution): raw wasm module bytes + eager userland compile #216 xusommqr · 2026-07-02 19:08:35
perf(v8-runtime): parked warm session workers #215 zssyplrp · 2026-07-02 18:53:13 👈
fix(ci): install build-tools workspace deps in the bench workflow #214 yuvtorxt · 2026-07-02 18:53:11
feat(execution): snapshot-baked wasm runner userland #213 osswtnlq · 2026-07-02 18:14:10
fix(core): drop the fixed 10ms trailing-output drain when exit arrives via event #212 qtwvlzny · 2026-07-02 17:46:37
main

Get stack: forklift get 215
Push local edits: forklift submit
Merge when ready: forklift merge 215

The filtered install (--filter @secure-exec/benchmarks...) omitted packages/build-tools, so every bench-gate run died in the v8-bridge build script (missing Node dependencies) before gating anything — the PR gate has never actually run green on CI.

Pre-created V8 session workers (thread + snapshot-restored isolate, keyed by snapshot digest + heap limit) are claimed at session create via a warm hint, taking isolate_new (~4.2ms) and blob_clone off the per-exec critical path. Workers are never reused across guests (each exec still gets a virgin isolate); a wrong hint falls back to the existing lazy-create path. Capacity per key via AGENTOS_V8_WARM_ISOLATES (default 2, 0 disables); refill after claim; pool transitions logged; warm_worker_hit/miss counters in the session phases output. Seeding is deliberately conservative: auto mode stays snapshot-only and only AGENTOS_WASM_SNAPSHOT_RUNNER=block seeds workers (wasm-runner key + default node key). Reason: creating isolates on background threads while other isolates execute guest WebAssembly deterministically SIGSEGVs the pinned V8 130 (WasmCodePointerTable::AllocateUninitializedEntry inside Isolate::New, even under the isolate lifecycle lock — the race is against V8-internal wasm threads). Backtraces + notes in the rusty-v8-upgrade todo; enabling default seeding rides that upgrade. Warm floors (block, this host): node -e '' 17 -> 14.4ms p50, wasm true 33 -> ~30ms (warm_worker_hit 64/67, isolate_new down to 3 background calls); pwd 42.6 -> 37.5ms, printf-0b 44.7 -> 40.2ms, date 115 -> 111ms. wasm suite 8.9s (was 96s, pool+snapshot amortize debug isolates); v8-runtime tests and bench:gate green.

NathanFlurry added 2 commits July 2, 2026 10:46

This was referenced Jul 2, 2026

fix(core): drop the fixed 10ms trailing-output drain when exit arrives via event #212

Merged

feat(execution): snapshot-baked wasm runner userland #213

Merged

fix(ci): install build-tools workspace deps in the bench workflow #214

Merged

NathanFlurry force-pushed the stack/fix-ci-install-build-tools-workspace-deps-in-the-bench-workflow-yuvtorxt branch from a1fb4da to 8eae9c4 Compare July 2, 2026 19:08

NathanFlurry force-pushed the stack/perf-v8-runtime-parked-warm-session-workers-zssyplrp branch from 54fd916 to e831166 Compare July 2, 2026 19:08

This was referenced Jul 2, 2026

perf(execution): raw wasm module bytes + eager userland compile #216

Merged

fix(sidecar): JS-lane reads see kernel-VFS state #217

Merged

NathanFlurry added 2 commits July 2, 2026 13:04

NathanFlurry force-pushed the stack/fix-ci-install-build-tools-workspace-deps-in-the-bench-workflow-yuvtorxt branch from 8eae9c4 to fc5d69a Compare July 2, 2026 20:05

NathanFlurry force-pushed the stack/perf-v8-runtime-parked-warm-session-workers-zssyplrp branch from e831166 to 7e0fa8e Compare July 2, 2026 20:05

NathanFlurry changed the base branch from stack/fix-ci-install-build-tools-workspace-deps-in-the-bench-workflow-yuvtorxt to main July 2, 2026 23:42

NathanFlurry merged commit 7e0fa8e into main Jul 2, 2026
3 checks passed

NathanFlurry deleted the stack/perf-v8-runtime-parked-warm-session-workers-zssyplrp branch July 2, 2026 23:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(v8-runtime): parked warm session workers#215

perf(v8-runtime): parked warm session workers#215
NathanFlurry merged 4 commits into
mainfrom
stack/perf-v8-runtime-parked-warm-session-workers-zssyplrp

NathanFlurry commented Jul 2, 2026 •

edited

Loading

Uh oh!

NathanFlurry commented Jul 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

NathanFlurry commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NathanFlurry commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

NathanFlurry commented Jul 2, 2026 •

edited

Loading

NathanFlurry commented Jul 2, 2026 •

edited

Loading