chore: Build for x86_64-unknown-linux-musl with all features and optimized binary size#689
chore: Build for x86_64-unknown-linux-musl with all features and optimized binary size#689binarylogic merged 5 commits intomasterfrom unknown repository
Conversation
Signed-off-by: Alexander Rodin <[email protected]>
Signed-off-by: Alexander Rodin <[email protected]>
Signed-off-by: Alexander Rodin <[email protected]>
Signed-off-by: Alexander Rodin <[email protected]>
Signed-off-by: Alexander Rodin <[email protected]>
|
@a-rodin this is fantastic! Thank you for doing this. I spent a considerable amount of time trying to do exactly this and was unsuccessful. In fact, I've been reaching out to our network to try and find someone to help with this, so I'm really happy to see this done. 😄 I'll have @LucioFranco review and then we can merge and get it fully integrated into our releases. |
LucioFranco
left a comment
LucioFranco
left a comment
There was a problem hiding this comment.
This is fantastic! I pulled this down locally and ran the build, everything looks good. I'm very happy this was even possible! I have a few questions but nothing blocking.
I say we 
@binarylogic I think we will have to build the new image and push it to docker hub before we can merge this?
| RUN rustup target add x86_64-unknown-linux-musl No newline at end of file | ||
| # Nightly Rust is used because at the moment only nightly uses recent enough | ||
| # LLVM version that supports libc++ with musl. It can be changed to a stable | ||
| # version after Rust 1.38 release. |
There was a problem hiding this comment.
Leaving a note here: we should open an issue for this.
cc @binarylogic
There was a problem hiding this comment.
@LucioFranco I was just trying to compile a musl Vector on a Linux x86_64 machine with cargo build --target x86_64-unknown-linux-musl --release --verbose using Rust stable 1.36.0 and hit an openssl error. Is this problem related to this issue?
Any consideration of using Rust native rustls? See https://jbp.io/2019/07/01/rustls-vs-openssl-performance.html. This works well for rusoto https://github.com/rusoto/rusoto/blob/master/rusoto/core/README.md#usage-with-rustls
There was a problem hiding this comment.
@moderation The OpenSSL error shouldn't be related to differences between nightly and stable Rust. Probably pkg-config is installed on your system, but returns empty CFLAGS and LIBS for openssl package. In that case setting environment variables OPENSSL_STATIC=1 and OPENSSL_INCLUDE_DIR/OPENSSL_LIB_DIR might help. See rust-openssl/rust-openssl#604 and https://github.com/timberio/vector/blob/master/distribution/docker/alpine/Dockerfile#L10-L13 for an example.
There was a problem hiding this comment.
@a-rodin thanks for the tip. I'm now able to get through to the linking stage with env OPENSSL_STATIC=1 OPENSSL_INCLUDE_DIR="/usr/include" OPENSSL_LIB_DIR="/usr/lib/x86_64-linux-gnu" cargo build --target x86_64-unknown-linux-musl --release --verbose. I get error: linking with cc failed: exit code: 1 following by a large stack trace / error dump. Closer. Maybe need to wait for newer Rust releases.
There was a problem hiding this comment.
@moderation I've seen linker crashing when building native rdkafka dependency without rdkafka/cmake_build feature, it is unrelated to Rust at all and caused by linking unused dynamic library for librdkafka. You can try cargo build --target x86_64-unknown-linux-musl --release --verbose --features="default rdkafka/cmake_build", in that case the library that makes C++ linker crash would just not be created at all.
| # Rust crate openssl-sys supports only released versions of LibreSSL. | ||
| # In order to be able to use the latest stable release with musl, | ||
| # we need to backport unreleased changes from this PR: | ||
| # https://github.com/libressl-portable/portable/pull/529/files. |
There was a problem hiding this comment.
what are the implications of this? Is there any follow up that we might need to do here?
There was a problem hiding this comment.
program_invocation_short_name is a global variable that contains short name of the program. It is not a part of standard C library, but a GNU extension. It is provided by musl though. The original code in LibreSSL checks on Linux whether C library is glibc and generates a compile-time error if it is not, even if program_invocation_short_name is available. With the patch the error is not thrown and program_invocation_short_name from musl is used.
Because the issue is fixed in upstream by the mentioned PR, it should be possible to remove this patching after the next release of LibreSSL.
|
Everything looks great here. I'm going to merge and follow up with documentation changes. |
ghost
mentioned this pull request
4 participants
This PR enables all features, including those with native dependencies, for
x86_64-unknown-linux-musltarget. The resulting binary is fully statically linked, LTO-optimized, and stripped.The build toolchain is based on Clang and
libc++from LLVM project, in addition tomusl.libc++is used byrdkafkaandleveldb. Although it should be possible to link GNUlibstdc++statically, it makes the resulting binaries licensed under LGPL, which might be not desirable as Vector's license is Apache 2. In contrast,libc++is MIT/UIUC licensed.Clang is used not only because it is easy to pair with
libc++, but also because it enables cross-language linker plugin LTO. Linker plugin LTO is different from LTO codegen option in Rust. Both types of LTO can be used together to reduce binary size and improve runtime performance.Here are the measurements of the binary size with different LTO combinations for
x86_64-unknown-linux-musltarget:lto=thinltolto=thinlto=thinlto=thinltoltoltoFrom the results listed above it can be seen that Rust codegen LTO type
ltoand linker plugin LTO typeltodeliver the best size of compressed stripped binary. It theoretically should also have the best performance, as LTO reduces functions call overhead by inlining small functions.For comparison, a similar table for
x86_64-unknown-linux-gnu:lto=thinltoAs a result, Rust codegen LTO type
ltoand linker plugin LTO typeltoare used forx86_64-unknown-linux-musl. Rust LTO type is not set inCargo.toml, but is added at the build time because LTO significantly increases build times, and in my opinion it is better to start from enabling it only for certain build types in CI.Stripping of Vector binary is added for all target platforms because it can't hurt, but only reduce resulting binaries size.
Ref #661, #630, #654, #669