Skip to content

Dedupe extracted embedded native modules in compiled binaries - #29587

Merged
Jarred-Sumner merged 1 commit into
mainfrom
farm/17bdd035/dedupe-embedded-native-extracts
Aug 18, 2026
Merged

Jarred-Sumner merged 1 commit into
mainfrom
farm/17bdd035/dedupe-embedded-native-extracts

Conversation

@robobun

@robobun robobun commented Apr 22, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #29585.

Problem

A bun build --compile binary that loads an embedded native library (bun:ffi dlopen or process.dlopen on a .node) extracted a fresh copy to /tmp on every call and never cleaned it up. Each Worker re-extracted on its first dlopen, so Worker-heavy servers leaked fast (the reporter measured ~14 GB/hour). The cause: resolve_embedded_file_to_buf named each tmpfile with nanoTimestamp() plus a per-VM counter, so no two calls ever produced the same path.

Fix

Extraction is now stateless and idempotent. The on-disk file is the cache.

  • The name is {tmpdir}/.bun-{euid}-{wyhash(contents):x}.{ext}. Same content gives the same path, so all dlopens, Workers, and restarts of the binary share one file. The euid keeps users on a shared /tmp apart.
  • Reuse check: lstatat the canonical name and reuse it if it is a regular file, owned by our euid, with the expected size. lstatat (not fstatat) makes a planted symlink fail the ISREG check instead of being followed.
  • Extraction writes a unique scratch file, then rename(2)s it onto the canonical name. Racing Workers converge on one valid file. If the rename fails (a foreign owner squats the name on sticky /tmp), that call uses its scratch file.
  • The post-dlopen deletion in BunProcess.cpp is removed on both platforms: deleting the shared file would defeat the dedup, and the Windows delete-on-reboot required admin rights and only ever marked the file in active use. The orphaned Bun__unlink export is deleted with its last caller.

Verification

case /tmp copies
before: 10 dlopens + 5 Workers, 2 runs 20
after 1
  • test/regression/issue/29585.test.ts: one file after 10 dlopens, after a second process run, after deleting the file (re-extracts), and across 5 concurrent Workers. Fails pre-fix with Expected: 1 / Received: 10.
  • test/napi/napi.test.ts (--compile): the .node / process.dlopen path keeps a stable extracted count across two runs, on all platforms (verified on Windows Server 2019 x64).

no test proof · iteration 55 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/napi/napi.test.ts test/regression/issue/29585.test.ts

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compiled binary: Workers leak extracted native .so/.dylib files in /tmp (never cleaned up)

4 participants