Skip to content

Conversation

@Ecordonnier
Copy link
Contributor

@Ecordonnier Ecordonnier commented Sep 9, 2025

In the context of musl, users are typically expecting statically-linked tools,
as this is one of musl's strong suits and one of the biggest reasons to use it
instead of glibc (that is not static-link-friendly for reasons).

  • remove the build-flag which was causing musl libc to be linked dynamically
  • re-add feat_os_unix_musl which was removed in 21d5cef
  • exclude stdbuf from feat_os_unix_musl, since libstdbuf.so can't be built with a statically linked musl libc
  • add test_musl_no_dynamic_deps so that this does not regress in the future

Fixes #8572

@github-actions
Copy link

github-actions bot commented Sep 9, 2025

GNU testsuite comparison:

Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)

@github-actions
Copy link

github-actions bot commented Sep 9, 2025

GNU testsuite comparison:

Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)

In the context of musl, users are typically expecting statically-linked tools,
as this is one of musl's strong suits and one of the biggest reasons to use it
instead of glibc (that is not static-link-friendly for reasons).

- remove the build-flag which was causing musl libc to be linked dynamically
- re-add feat_os_unix_musl which was removed in uutils@21d5cef
- exclude stdbuf from feat_os_unix_musl, since libstdbuf.so can't be built with a statically linked musl libc
- add test_musl_no_dynamic_deps so that this does not regress in the future

Fixes uutils#8572

Signed-off-by: Etienne Cordonnier <[email protected]>
@github-actions
Copy link

github-actions bot commented Sep 9, 2025

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)

@sylvestre sylvestre merged commit 5c33a30 into uutils:main Sep 9, 2025
95 checks passed
@Ecordonnier Ecordonnier deleted the eco/musl-libc-fix branch September 9, 2025 11:32
@omnivagant
Copy link

hello from a musl/linux based distribution dynamically linking rust packages, wouldn't it be better if it was called feat_os_unix_musl_static?

not that I think this change will affect us, since we can continue to use feat_os_unix

https://hackmd.io/@rust-compiler-team/ByozE7L4u

@Ecordonnier
Copy link
Contributor Author

hello from a musl/linux based distribution dynamically linking rust packages, wouldn't it be better if it was called feat_os_unix_musl_static?

not that I think this change will affect us, since we can continue to use feat_os_unix

https://hackmd.io/@rust-compiler-team/ByozE7L4u

If I understood correctly, the naming "feat_os_unix_musl_static" makes sense. So you're saying that Alpine Linux is dynamically linking rust packages to musl libc and that you want a working stdbuf?

Then we would need a way to toggle target-feature=-crt-static via a build flag. Short-term you can continue to use feat_os_unix and enable target-feature=-crt-static yourself e.g. via a patch.

@omnivagant
Copy link

Originally it was mostly a comment against the idea that "musl build" means "static build".

From what I can see, we don't have a problem with stdbuf even after this change.

I had forgotten, though, that tests have been disabled for some time (due to file descriptor issues on our package builders, something we don't see in CI, will have to check if this is still an issue), so of course we have to skip the here introduced test_musl_no_dynamic_deps test (there are also a, between architectures varying, number of other failing tests https://gitlab.alpinelinux.org/alpine/aports/-/jobs/2005822).

@Ecordonnier
Copy link
Contributor Author

Originally it was mostly a comment against the idea that "musl build" means "static build".

From what I can see, we don't have a problem with stdbuf even after this change.

I had forgotten, though, that tests have been disabled for some time (due to file descriptor issues on our package builders, something we don't see in CI, will have to check if this is still an issue), so of course we have to skip the here introduced test_musl_no_dynamic_deps test (there are also a, between architectures varying, number of other failing tests https://gitlab.alpinelinux.org/alpine/aports/-/jobs/2005822).

You should be getting a build error for stdbuf after this change, unless you're setting the rust flag to link musl dynamically yourself.

@omnivagant
Copy link

it doesn't look like it, from the build log I linked to above, what am I missing?

@Ecordonnier
Copy link
Contributor Author

@omnivagant I guess something in your build environment provides the crt-static rust flag I mentioned.

This is what I get with uutils coreutils 0.2.2 on Ubuntu 24.04:

ecordonnier@lj8k2dq3:~/dev/coreutils$ cargo --version
cargo 1.89.0 (c24e10642 2025-06-23)
ecordonnier@lj8k2dq3:~/dev/coreutils$ cargo build --target x86_64-unknown-linux-musl --features feat_os_unix
error: cannot produce cdylib for `uu_stdbuf_libstdbuf v0.2.2 (/home/ecordonnier/dev/coreutils/src/uu/stdbuf/src/libstdbuf)` as the target `x86_64-unknown-linux-musl` does not support these crate types

Also please note that as far as I can see, the file aports/community/uutils/APKBUILD does not install stdbuf, even in your branch uutils-0.2.0-r0

@omnivagant
Copy link

But we are not Ubuntu and, as I said, we patch Rust. The HackMD link show one of our efforts to resolve the issue with -musl targets in Rust.

I don't know how you would infer from the APKBUILD if we install stdbuf or not, as we iterate over what the uutils binary has built in (and I see now that a --list option was added, very nice). Here you can see that we do package stdbuf (it's a symlink to the uutils binary) - https://pkgs.alpinelinux.org/contents?file=stdbuf&path=&name=uutils-coreutils&branch=edge
This is version still at 0.1.0 of uutils-coreutils, as the upgrade is work in progress.

In the uutils-0.2.0-r0 branch I have added upgrade to 0.2.2, it is from the build log of that I linked to above.

Here is a more recent build log, where I allow tests to fail with || true - https://gitlab.alpinelinux.org/alpine/aports/-/jobs/2007024

At the bottom we would see content changes to the packages of the aport but, besides size differences, there are none. stdbuf is still packaged and its tests seem to pass,

test test_stdbuf::invalid_input ... ok
test test_stdbuf::test_permission ... ok
test test_split::test_split_num_prefixed_chunks_by_lines ... ok
test test_stdbuf::test_no_such ... ok
test test_stdbuf::test_stdbuf_line_buffering_stdin_fails ... ok
test test_stdbuf::test_stdbuf_no_buffer_option_fails ... ok
test test_stdbuf::test_stdbuf_line_buffered_stdout ... ok
test test_stdbuf::test_stdbuf_trailing_var_arg ... ok
test test_stty::all_and_print_setting ... ok
test test_stdbuf::test_stdbuf_unbuffered_stdout ... ok
test test_stty::all_and_setting ... ok
test test_stdbuf::test_stdbuf_invalid_mode_fails ... ok

The only stdbuf related errors I get is when running tests with the release or release-small profiles,

The lib target `stdbuf` in package `uu_stdbuf_libstdbuf v0.2.2 (/builds/alpine/aports/community/uutils/src/coreutils-0.2.2/src/uu/stdbuf/src/libstdbuf)` has the same output filename as the
 lib target `stdbuf` in package `uu_stdbuf_libstdbuf v0.2.2 (/builds/alpine/aports/community/uutils/src/coreutils-0.2.2/src/uu/stdbuf/src/libstdbuf)`.
Colliding filename is: /builds/alpine/aports/community/uutils/src/coreutils-0.2.2/target/release/deps/libstdbuf.so
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `stdbuf` in package `uu_stdbuf_libstdbuf v0.2.2 (/builds/alpine/aports/community/uutils/src/coreutils-0.2.2/src/uu/stdbuf/src/libstdbuf)` has the same output filename as the
 lib target `stdbuf` in package `uu_stdbuf_libstdbuf v0.2.2 (/builds/alpine/aports/community/uutils/src/coreutils-0.2.2/src/uu/stdbuf/src/libstdbuf)`.
Colliding filename is: /builds/alpine/aports/community/uutils/src/coreutils-0.2.2/target/release/deps/libstdbuf.so.dwp
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0.2.0 prebuilt musl binaries are not statically linked

3 participants