Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Since Rust 1.83.0, rust-src component leads to bloated runtime dependencies #199

Closed
veehaitch opened this issue Dec 4, 2024 · 3 comments
Closed

Since Rust 1.83.0, rust-src component leads to bloated runtime dependencies #199

veehaitch opened this issue Dec 4, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@veehaitch
Copy link

The commit that introduces Rust 1.83.0 (b259ef7) appears to bloat the closure of packages built with the rust toolchain that have the rust-src component.

rust-toolchain.toml:

[toolchain]
channel = "stable"
components = [ "rust-src" ]
targets = [ "x86_64-unknown-linux-gnu" ]

default.nix (adapted the example from the nixpkgs manual):

with import <nixpkgs>
{
  overlays = [
    (import (fetchTarball "https://github.com/oxalica/rust-overlay/archive/34971069ec33755b2adf2481851f66d8ec9a6bfa.tar.gz"))
  ];
};
let
  rustToolchain = rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
  rustPlatform = makeRustPlatform {
    cargo = rustToolchain;
    rustc = rustToolchain;
  };
in

rustPlatform.buildRustPackage rec {
  pname = "ripgrep";
  version = "12.1.1";

  src = fetchFromGitHub {
    owner = "BurntSushi";
    repo = "ripgrep";
    rev = version;
    hash = "sha256-+s5RBC3XSgb8omTbUNLywZnP6jSxZBKSS1BmXOjRF8M=";
  };

  cargoHash = "sha256-l1vL2ZdtDRxSGvP0X/l3nMw8+6WF67KPutJEzUROjg8=";

  doCheck = false;

  meta = {
    description = "Fast line-oriented regex search tool, similar to ag and ack";
    homepage = "https://github.com/BurntSushi/ripgrep";
    license = with lib.licenses; [ mit unlicense ];
    maintainers = with lib.maintainers; [];
  };
}
# nix-build
# nix path-info --recursive --size --closure-size --human-readable ./result
/nix/store/4hcdxyjf9yiq7qf3i4548drb6sjmwa1v-xgcc-13.3.0-libgcc                                   155.9 KiB       155.9 KiB
/nix/store/w19cxz37j5nrkg8w80y91bga89310jgi-libunistring-1.2                                       1.8 MiB         1.8 MiB
/nix/store/jwsdpq2yxw43ixalh93z726czz7bay2j-libidn2-2.3.7                                        352.6 KiB         2.1 MiB
/nix/store/sl141d1g77wvhr050ah87lcyz2czdxa3-glibc-2.40-36                                         28.8 MiB        31.1 MiB
/nix/store/5frxzv8fj2zaj04lbv3z848p8zvwpz3g-gmp-6.3.0                                            706.6 KiB        31.8 MiB
/nix/store/bklfkdh58y2hr24796hln00l2ly68fvg-mpfr-4.2.1                                           775.7 KiB        32.5 MiB
/nix/store/d72018202lbgj7ckmd020d56apny73wg-libmpc-1.3.1                                         277.4 KiB        32.8 MiB
/nix/store/haci0d018njygwdga95bsbx8ww2iswy7-zlib-1.3.1                                           125.2 KiB        31.2 MiB
/nix/store/kskaix9pdvh22pkgnaqd3k9yd7r1xh1z-isl-0.20                                               2.5 MiB        34.3 MiB
/nix/store/1qx77645wk0rd00h49rvjh9vkgs8vwvy-linux-headers-6.10                                     6.5 MiB         6.5 MiB
/nix/store/vpsla1ivhavzd4fmi95yzmgb4g9rd072-glibc-2.40-36-bin                                      2.7 MiB        33.7 MiB
/nix/store/n15bxkd7id2gvlwpihm4kp675aqkrwmj-glibc-2.40-36-dev                                      2.3 MiB        42.6 MiB
/nix/store/lw294qfdnf29lhxqwzy9byrmx975rvn9-gcc-13.3.0-libgcc                                    155.9 KiB       155.9 KiB
/nix/store/y3kdn61k93rq2jx1lj2x72lnsk0l92qh-gcc-13.3.0-lib                                         8.7 MiB        39.9 MiB
/nix/store/0vsyw5bhwmisszyfd1a0sdnwvnf4qa5a-gcc-13.3.0                                           220.7 MiB       276.5 MiB
/nix/store/56nav6x43mv8b5bipazq4d9ycr1ngvav-expand-response-params                                16.4 KiB        31.1 MiB
/nix/store/717iy55ncqs0wmhdkwc5fg2vci5wbmq8-bash-5.2p32                                            1.6 MiB        32.6 MiB
/nix/store/j6z0q427f6jm3d9mmrb4pgz72i93yp2f-attr-2.5.2                                            90.6 KiB        31.2 MiB
/nix/store/4ac0simsl36mz7ksv64ag7nmpc86s1mh-acl-2.3.2                                            125.2 KiB        31.3 MiB
/nix/store/w3jiv6xd520k1d1rvhxn7mr0pqzindjs-gmp-with-cxx-6.3.0                                   749.0 KiB        40.6 MiB
/nix/store/ph44jcx3ddmlwh394mh1wb7f1qigxqb1-coreutils-9.5                                          1.4 MiB        42.3 MiB
/nix/store/88rlbc5ka2m5zir98ajkgpqkb1i82wcs-rustc-1.83.0-x86_64-unknown-linux-gnu                318.5 MiB       362.5 MiB
/nix/store/0w0sbmymx7xify583f78djpx0c47xhqv-rustfmt-preview-1.83.0-x86_64-unknown-linux-gnu       10.9 MiB       373.5 MiB
/nix/store/zxk207030dbrz6ikiq61sz2c08gpljxq-pcre2-10.44                                            1.8 MiB        32.9 MiB
/nix/store/20s3724qq3g0j14v6zc0s7z16k7i3j39-libselinux-3.7                                         1.2 MiB        34.1 MiB
/nix/store/hc7rrrmb1wilsldsii4fbmvcl68lijsl-util-linux-minimal-2.39.4-lib                          1.8 MiB        32.9 MiB
/nix/store/pmf31rbzvv618qirwmzppjb63wsrl4zi-libffi-3.4.6                                          71.9 KiB        31.2 MiB
/nix/store/fxnchbq09sq279ssqdsgzigsyvp6hz06-glib-2.80.4                                           14.7 MiB        50.8 MiB
/nix/store/ncznv9xrbi34k4zlnd25dp7kyaqjscn7-libgpg-error-1.50                                    832.7 KiB        31.9 MiB
/nix/store/mxqw137xwdd4fgnna0z4a7j8cx969br8-libgcrypt-1.10.3-lib                                   1.4 MiB        33.3 MiB
/nix/store/spvci786gald6nsf7z5a095fsc1dc3xx-libsecret-0.21.4                                     672.8 KiB        53.6 MiB
/nix/store/9x0gj79aslszsgn8qcgwxv6ni27szl00-cargo-1.83.0-x86_64-unknown-linux-gnu                 33.4 MiB        87.2 MiB
/nix/store/c1n8g5129aycgj4cxxgry8460ki7glfj-binutils-2.43.1-lib                                    2.8 MiB        34.0 MiB
/nix/store/jp66i2lg1p2s8lkwsqh33c3913rgdq8j-clippy-preview-1.83.0-x86_64-unknown-linux-gnu        13.8 MiB       376.3 MiB
/nix/store/nzmp6x4694fn4al0dxq4g1q4fr4yyg7l-rust-std-1.83.0-x86_64-unknown-linux-gnu             146.9 MiB       147.1 MiB
/nix/store/v9znmjrv9f18072n1y1g5caf9irzs6v2-rust-docs-1.83.0-x86_64-unknown-linux-gnu            659.1 MiB       659.1 MiB
/nix/store/vcvhwiilizhijk7ywyn58p9l005n9sbn-binutils-2.43.1                                       30.1 MiB        72.9 MiB
/nix/store/l7n97992gd5piaw8phkxzsz176gfk1yj-binutils-wrapper-2.43.1                               50.0 KiB        88.3 MiB
/nix/store/lvnwdmnjm7nvaq0a3vhvvn46iy4ql7gr-gnugrep-3.11                                         926.6 KiB        33.8 MiB
/nix/store/vh9fsdhgxcnab2qk7vdp2palkkn6j3cp-gcc-wrapper-13.3.0                                    61.8 KiB       316.1 MiB
/nix/store/vpg1chc5fncwvd5fh6j9nywx0bcdpf04-rust-src-1.83.0-x86_64-unknown-linux-gnu              32.1 MiB        64.8 MiB
/nix/store/frx5kr6cmy43qij0kcif59z47pdhkik3-rust-default-1.83.0                                  180.5 MiB         1.7 GiB
/nix/store/hchmc1f3czwxxn1q8nk9ya9h50cnyn1p-ripgrep-12.1.1                                         4.9 MiB         1.7 GiB

When using b259ef7:

# nix-build
# nix path-info --recursive --size --closure-size --human-readable ./result
/nix/store/4hcdxyjf9yiq7qf3i4548drb6sjmwa1v-xgcc-13.3.0-libgcc   155.9 KiB       155.9 KiB
/nix/store/w19cxz37j5nrkg8w80y91bga89310jgi-libunistring-1.2       1.8 MiB         1.8 MiB
/nix/store/jwsdpq2yxw43ixalh93z726czz7bay2j-libidn2-2.3.7        352.6 KiB         2.1 MiB
/nix/store/sl141d1g77wvhr050ah87lcyz2czdxa3-glibc-2.40-36         28.8 MiB        31.1 MiB
/nix/store/lw294qfdnf29lhxqwzy9byrmx975rvn9-gcc-13.3.0-libgcc    155.9 KiB       155.9 KiB
/nix/store/y3kdn61k93rq2jx1lj2x72lnsk0l92qh-gcc-13.3.0-lib         8.7 MiB        39.9 MiB
/nix/store/41zm31n8rnd9wb207fpz430dwydhqg3s-ripgrep-12.1.1         4.9 MiB        44.8 MiB
@oxalica oxalica added the bug Something isn't working label Dec 4, 2024
@oxalica
Copy link
Owner

oxalica commented Dec 4, 2024

Cause: rust-lang/rust#129687
Since 1.83, rustc will discover and embed local std path in the binary if possible, which results in a runtime dependency. Before that, a virtual path /rust/<hash> is used.

You can workaround it for now via --remap-path-prefix like this. I'm not sure if this can be solely fixed in rust-overlay. We may need to add the remap flag in rustPlatform.buildRustPackage's implementation. The embedded path seems to be$(rustc --print sysroot)/lib/rustlib/src/rust/library.

@oxalica oxalica mentioned this issue Dec 4, 2024
Merged
@oxalica
Copy link
Owner

oxalica commented Dec 4, 2024

According to the upstream comment: rust-lang/rust#129687 (comment)

There are 2 solutions:

  1. Use a minimal toolchain for build system, thus eliminating rust-src component from build closure at all. I'd 👍 on this, because dev toolchains and release toolchains are expected to be different. You also want to shrink build dependencies from these extensions.
    Nixpkgs' rust toolchain is also minimized and does not carry rust-src by default, so it is free from this issue.
  2. If you want to use a full-blown dev toolchains with rust-src for the build anyway, add a remap flag in preConfigure: 
    rustSrcBase="$(rustc --print=sysroot)/lib/rustlib/src/rust"
export RUSTFLAGS="--remap-path-prefix=$rustSrcBase=/"

I don't think it's actionable in rust-overlay part. So I'm closing for now.

@oxalica oxalica closed this as not planned Won't fix, can't repro, duplicate, stale Dec 4, 2024
@veehaitch
Copy link
Author

Thank you for addressing this so quickly! 🙏🏼

mfenniak added a commit to testtrim/testtrim that referenced this issue Mar 27, 2025
@mfenniak
fix: prevent cargo/rustc from being a runtime dependency of app (#453)
e1b381a 
Dev environment was being pulled in as a runtime dependency of the app and container due to the "rust-src" extension. (More notes: oxalica/rust-overlay#199)

Reviewed-on: https://codeberg.org/testtrim/testtrim/pulls/453
Co-authored-by: Mathieu Fenniak <[email protected]>
Co-committed-by: Mathieu Fenniak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@oxalica @veehaitch