Skip to content

perf(sidecar): raw-binary readFileSync transfer (skip base64) #81

Description

@NathanFlurry

Background

Guest binary fs.readFileSync(path) (no encoding) goes
_fs.readFileBinary_fsReadFileBinary → mapped to fs.readFileSync with the
binary flag (crates/execution/src/v8_runtime.rs:183) → host returns the bytes as
a base64-encoded JSON object (javascript_sync_rpc_bytes_value:
{ "__agentOsType": "bytes", "base64": "..." }), and the guest decodes with
Buffer.from(base64, "base64") (crates/execution/assets/v8-bridge.source.js).

So every binary file read pays base64 encode (host) + decode (guest) plus ~33%
payload inflation. The base64 round-trip is wasted work.

Proposal

Transfer file bytes as raw binary instead of base64. The bridge response
protocol already has a raw-binary path (status = 2 / Uint8Array, see
crates/v8-runtime/src/host_call.rs). Route readFileSync's binary result through
that path and have the guest do Buffer.from(uint8array) directly.

Why it wasn't done in PR #77

The sync-RPC return type is a serde_json::Value, which can't carry raw bytes —
hence the base64 representation. Switching readFileSync to the status=2
raw-binary response is a cross-cutting protocol change that affects every
binary file read, so it was deferred rather than rushed (read was already 6.2×
faster from the pump fix in #77). Needs careful implementation + tests for:
binary readFile (sync + promises), writeFileBinary symmetry, and the
{__agentOsType:"bytes"} consumers elsewhere.

Expected impact

Removes base64 encode/decode + ~33% payload inflation per binary read; helps
read-heavy guests (bundlers, asset pipelines, large node_modules).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions