Skip to content

Single Executable Application segfaults on musl builds but works on official node:alpine on arm64 #200

New issue
New issue
Closed
#233
Closed
Single Executable Application segfaults on musl builds but works on official node:alpine on arm64#200
#233
Labels
alpine-muslitems relating to the Alpine/musl builds

Description

@vanyauhalin
vanyauhalin
opened on Feb 7, 2026

Description

SEA built using musl builds result in segmentation faults when executed, while the same process works correctly on the official node:alpine image.

Reproduction

#!/bin/sh
# sea-test-official.sh

set -eu

echo "console.log('Hello from SEA')" > hello.js
echo '{"main":"hello.js","output":"sea-prep.blob"}' > sea-config.json
node --experimental-sea-config sea-config.json
cp "$(which node)" hello
npx --yes postject hello NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
chmod +x hello
./hello || echo "Error code: $?"
$ docker run -it --rm node:25.6.0-alpine sh -c "$(cat sea-test-official.sh)"
Wrote single executable preparation blob to sea-prep.blob
Start injection of NODE_SEA_BLOB in hello...
warning: Can't find string offset for section name '.note'
💉 Injection done!
npm notice
npm notice New minor version of npm available! 11.8.0 -> 11.9.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.9.0
npm notice To update run: npm install -g [email protected]
npm notice
Hello from SEA
(node:40) ExperimentalWarning: Single executable application is an experimental feature and might change at any time
(Use `hello --trace-warnings ...` to show where the warning was created)
#!/bin/sh
# sea-test-unofficial.sh

set -eu

apk add --no-cache curl xz libgcc libstdc++
curl -fsSL https://unofficial-builds.nodejs.org/download/release/v25.6.0/node-v25.6.0-linux-arm64-musl.tar.xz | tar -xJ
export PATH="$PWD/node-v25.6.0-linux-arm64-musl/bin:$PATH"

echo "console.log('Hello from SEA')" > hello.js
echo '{"main":"hello.js","output":"sea-prep.blob"}' > sea-config.json
node --experimental-sea-config sea-config.json
cp "$(which node)" hello
npx --yes postject hello NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
chmod +x hello
./hello || echo "Error code: $?"
$ docker run -it --rm alpine sh -c "$(cat sea-test-unofficial.sh)"
( 1/14) Installing brotli-libs (1.2.0-r0)
( 2/14) Installing c-ares (1.34.6-r0)
( 3/14) Installing libunistring (1.4.1-r0)
( 4/14) Installing libidn2 (2.3.8-r0)
( 5/14) Installing nghttp2-libs (1.68.0-r0)
( 6/14) Installing nghttp3 (1.13.1-r0)
( 7/14) Installing libpsl (0.21.5-r3)
( 8/14) Installing zstd-libs (1.5.7-r2)
( 9/14) Installing libcurl (8.17.0-r1)
(10/14) Installing curl (8.17.0-r1)
(11/14) Installing libgcc (15.2.0-r2)
(12/14) Installing libstdc++ (15.2.0-r2)
(13/14) Installing xz-libs (5.8.2-r0)
(14/14) Installing xz (5.8.2-r0)
Executing busybox-1.37.0-r30.trigger
OK: 17.0 MiB in 30 packages
Wrote single executable preparation blob to sea-prep.blob
Start injection of NODE_SEA_BLOB in hello...
💉 Injection done!
npm notice
npm notice New minor version of npm available! 11.8.0 -> 11.9.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.9.0
npm notice To update run: npm install -g [email protected]
npm notice
Segmentation fault (core dumped)
Error code: 139

Activity

  1. rvagg commented on Feb 9, 2026

    @rvagg
    rvagg
    on Feb 9, 2026
    Member

    Someone's going to need to investigate this further, perhaps that's you? You could also look through git history here to figure out who's interested in arm64 musl. It looks like a cross-compile artifact to me. The Alpine Docker containers for arm64 build natively inside the container (unlike the x64 ones which pull from here). We cross-compile here because we're running on x64 hardware. So there might be a cross compilation flag that's missing or some other change that's needed to the pipeline here to make it work properly. It's going to either need some deeper expertise here or some patience to tinker.

  2. sxa commented on Feb 9, 2026

    @sxa
    sxa
    on Feb 9, 2026
    Member

    Other than for x64 the Alpine docker images that are published are built in a completely different way from the builds on unofficial builds server and, in fact, are not even built on node.js infrastructure (It's not cross-compiled for a start so as rvagg suggests above that could be the difference) I'll be honest I'm 100% in favour of not publishing those unofficial container images at all, but as @rvagg says it will need a volunteer to look at this if there is any desire to make this experimental feature on an experimental platform work reliably :-)

    FYI @RafaelGSS @MikeMcC399 (Not because I expect them to volunteer, but because I've been discussing the Alpine docker images with them both recently and this is the sort of inconsistency we can see from the current process :-)

  3. sxa
    added
    alpine-muslitems relating to the Alpine/musl builds
    on Feb 20, 2026
  4. sxa
    changed the title [-]Single Executable Application segfaults on musl builds but works on official node:alpine[/-] [+]Single Executable Application segfaults on musl builds but works on official node:alpine on arm64[/+] on Apr 21, 2026

  5. sxa commented on Apr 21, 2026

    @sxa
    sxa
    on Apr 21, 2026
    Member

    Edited title to clarify architecture as this works ok on x64

  6. awbx commented on May 20, 2026

    @awbx
    awbx
    on May 20, 2026
    Contributor

    [Edited 2026-05-20 — original "section relocation" theory was wrong; replaced with corrected root-cause analysis below.]

    Reproducing this on Apple Silicon (native arm64) with Docker Desktop on both v25.6.0 and v26.1.0 of the unofficial-builds linux-arm64-musl tarball. The unmodified binary boots fine; only the postject-modified binary crashes.

    The 5 SIGILLs aren't the bug. Under strace -e signal=all you can see OpenSSL's ARM crypto-extension probes firing (_armv8_sm3_probe, _armv8_sha256_probe, etc.) and being caught by OpenSSL's installed SIGILL handler. That's normal CPU-feature detection. The actual death is a NULL-deref memcpy after OpenSSL init completes:

    #0  memcpy ()
#1  node::BlobDeserializer<...>::ReadArithmetic<unsigned int>()
#2  node::sea::FindSingleExecutableResource()
#3  node::sea::FixupArgsForSEA(int, char**)
#4  node::Start(int, char**)

    postject_find_resource("NODE_SEA_BLOB", ...) returns NULL → BlobDeserializer::ReadArithmetic does memcpy(dst, NULL, sizeof(uint32_t)) → SEGV.

    Why postject_find_resource() returns NULL — the unofficial arm64-musl binary has no PT_NOTE program header at all:

    # Official node:25.6.0-alpine arm64:
$ readelf -lW node | grep '^\s*NOTE'
  NOTE  0x000270 0x270 0x270 0x24 0x24 R  0x4
$ readelf -SW node | grep '\.note'
  [ 1] .note.gnu.build-id NOTE  ...

# Unofficial v26.1.0-linux-arm64-musl:
$ readelf -lW node | grep '^\s*NOTE'
(no output)
$ readelf -SW node | grep '\.note'
(no output)

    The unofficial build strips .note* sections and emits no PT_NOTE program header. When postject (via LIEF) adds a .note.NODE_SEA_BLOB section, there's no PT_NOTE segment to extend, so the note is unreachable to dl_iterate_phdr at runtime.

    Fix locations, cleanest first:

    1. unofficial-builds toolchain — stop stripping .note* from arm64-musl builds. The official node:alpine keeps .note.gnu.build-id; the unofficial cross-compile shouldn't be more aggressive. Likely a -s or strip --strip-all invocation that's specific to this arch/libc combination.
    2. postject — fail loudly when injecting into a binary with no PT_NOTE program header instead of silently producing a non-functional output.
    3. node BlobDeserializer — CHECK_NOT_NULL the resource pointer before memcpy and emit a clean error like "SEA fuse is set but no NODE_SEA_BLOB found".

    Reproduces on v25.6.0 and v26.1.0 of the unofficial-builds arm64-musl tarball. Doesn't reproduce with the official node:alpine Docker image (which has .note.gnu.build-id + PT_NOTE). cc @rvagg @sxa

  7. MikeMcC399 commented on May 20, 2026

    @MikeMcC399
    MikeMcC399
    on May 20, 2026
    Member

    Still crashes on v26.1.0 (latest published arm64-musl on unofficial-builds; v26.2.0 returns 404 there).

    On https://unofficial-builds.nodejs.org/logs/202605201309-v26.2.0/ you can see that arm64-musl.log shows the arm64 build hasn't finished building yet.

  8. awbx commented on May 20, 2026

    @awbx
    awbx
    on May 20, 2026
    Contributor

    [Edited 2026-05-20 — this comment claimed the bug was the same as nodejs/node#61483; that turned out to be wrong. They co-occur in postject runs but have different root causes. See the corrected analysis in my earlier comment — the actual root cause for this issue is that unofficial-builds arm64-musl strips all .note* sections and emits no PT_NOTE program header, so postject has nowhere to place a runtime-discoverable note.]

  9. awbx commented on May 20, 2026

    @awbx
    awbx
    on May 20, 2026
    Contributor

    Located the precise fix — and walking back my earlier guess about a strip flag.

    The unofficial arm64-musl tarball is the only unofficial build missing notes. The other cross-compile recipes are all fine:

    Recipe Cross-compiler source Has .note.gnu.build-id?
    arm64-musl Self-built richfelker/musl-cross-make (commit 3635262) ❌ No
    loong64 Ubuntu's g++-14-loongarch64-linux-gnu package ✅ Yes
    riscv64 / riscv64-pointer-compression Ubuntu's g++-14-riscv64-linux-gnu package ✅ Yes
    musl (x64-musl) Alpine's native g++ ✅ Yes

    Not a strip step. Distro-packaged cross-compilers (Ubuntu's gcc-X-arch-linux-gnu, Alpine's native gcc) ship with specs files that inject -Wl,--build-id into every link. The musl-cross-make-built toolchain has no such specs, and binutils' ld doesn't emit .note.gnu.build-id by default. So the binary just never gets the note rather than getting one and being stripped.

    A one-line fix in recipes/arm64-musl/run.sh:

     export CC="ccache /opt/aarch64-linux-musl-cross/bin/aarch64-linux-musl-gcc"
 export CXX="ccache /opt/aarch64-linux-musl-cross/bin/aarch64-linux-musl-g++ -static-libstdc++"
+export LDFLAGS="-Wl,--build-id=sha1"

  10. awbx commented on May 20, 2026

    @awbx
    awbx
    on May 20, 2026
    Contributor

    PR for the fix: #233

  11. rvagg
    added a commit that references this issue on Jul 11, 2026

    fix: emit build-id in arm64-musl cross-compiled binaries (#233)

    de249d6
  12. rvagg
    closed this as completedin #233on Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    alpine-muslitems relating to the Alpine/musl builds

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions