perf: cut create-session latency via agent-SDK heap snapshot#124
Merged
Conversation
…YNC_RPC_TRACE). Finding: ~50 fs RPCs/session, not a storm; bulk tax is 855-file module loading via the inline module-reader path.
…TRACE). Finding: loadPiSdkRuntime is evaluate-dominated (~510ms run vs 115ms resolve/load/format/compile for 500 mods) — code-cache buys only ~38ms; real lever is isolate reuse / post-init V8 snapshot.
…CRPC_LAT). ROOT CAUSE: 5,100 round-trips x 139us = 709ms = all of loadPiSdkRuntime. VM tax is IPC round-trips (frame I/O over pipe per host_call), not V8 compute. Lever: cut round-trip count (combine per-module resolve/load/format) and/or per-call latency.
…ModuleReader trait (owned in v8-runtime, no circular dep) + ModuleResolveState.guest_reader + Phase-5 load fast-path that reads source directly when a reader is present, else bridge IPC. Inert until the higher crate plumbs the reader (iter6); compiles, gate green, sessionCreate unchanged. (iter4 structural verdict folded in; see ~/.agents notes.)
… reads module source DIRECTLY (step 1 LOAD). ModuleReaderHandle newtype rides RuntimeCommand -> dispatch -> SessionCommand::SetModuleReader -> session thread; sidecar impls GuestModuleReader over HostDirModuleReader (cloned per call site). VALIDATED: round-trips 5100->4600, sessionCreate 1498->1463, gate green, session creates correctly. Resolve+format (the bulk) is iter9.
…wraps reader+cache, exact bridge ModuleResolver semantics). Round-trips 4600->2725, sessionCreate 1463->1444, gate GREEN (resolution correct). Finding: 139us/call was mostly the resolution/read WORK not IPC overhead, so wall-clock barely moved (loadPiSdkRuntime 633->615). Cumulative steps1+2: 1498->1444 (~3.6%), loadPiSdkRuntime 708->615 (~13%). VM tax is largely intrinsic (compile/evaluate/openat2), not IPC.
railway-app
Bot
temporarily deployed
to
secure-exec / secure-exec-pr-124
June 24, 2026 23:12
Destroyed
railway-app
Bot
temporarily deployed
to
rivet-frontend / secure-exec-pr-124
June 24, 2026 23:12
Destroyed
|
🚅 Deployed to the secure-exec-pr-124 environment in rivet-frontend
🚅 Deployed to the secure-exec-pr-124 environment in secure-exec
|
…tocol + VM-config wiring Review hardening: - H-1: process.versions wrapper in SNAPSHOT_USERLAND_PREP now defers to the bridge's live getter post-restore (falls back to the static identity only while it throws during snapshot creation) instead of a permanent static pin, so restored sessions read real bridge-provided versions, not frozen constants. - M-1: bounded-by-default heap cap (DEFAULT_HEAP_LIMIT_MB=128) applied in both create_isolate and create_isolate_from_snapshot when no explicit limit is given; the OOM guard is always installed so a guest heap bomb terminates its own isolate instead of fatal-aborting the shared process. - Part 25 regression test: per-session process.versions flows through post-restore. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
NathanFlurry
force-pushed
the
perf-regression
branch
from
June 25, 2026 08:04
7264885 to
558f871
Compare
railway-app
Bot
temporarily deployed
to
secure-exec / secure-exec-pr-124
June 25, 2026 08:04
Destroyed
railway-app
Bot
temporarily deployed
to
rivet-frontend / secure-exec-pr-124
June 25, 2026 08:04
Destroyed
NathanFlurry
added a commit
that referenced
this pull request
Jun 25, 2026
…t ctors + clippy) (#127) #124 (perf: agent-SDK heap snapshot) landed while its CI was red, leaving main failing on fmt, clippy, and 13 missing-field/unused test-compile errors. Repair: - rustfmt 1.96.0: reformat embedded_runtime.rs, snapshot.rs, sidecar/execution.rs, javascript_v8.rs (chain/closure collapse the pinned toolchain expects). - userland_code: add the WarmSnapshot/Execute snapshot field (#124) to 11 ipc_binary BinaryFrame test constructors + 2 embedded_runtime_session frames (empty = bridge-only, the documented default); update the hand-built WarmSnapshot trailing-byte test for the new 2-field wire layout. - clippy -D warnings: checked_div in queue_tracker, is_multiple_of in sidecar/execution, while-let in the js backpressure test, and allow(dead_code) on the host_dir test's source-included module (SessionModuleReader is used elsewhere in the crate). Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts agentOS create-session latency by snapshotting the agent SDK so it is evaluated once per sidecar and reused across sessions, instead of re-evaluated on every `createSession`.
Results (Pi agent, llmock)
What's here
Notes