Skip to content

separateDebugInfo: use CARGO_BUILD_RUSTFLAGS#261727

Closed
szlend wants to merge 1 commit intoNixOS:masterfrom
szlend:rust-separate-debug-info-fix
Closed

separateDebugInfo: use CARGO_BUILD_RUSTFLAGS#261727
szlend wants to merge 1 commit intoNixOS:masterfrom
szlend:rust-separate-debug-info-fix

Conversation

@szlend
Copy link
Member

@szlend szlend commented Oct 17, 2023

Description of changes

RUSTFLAGS overrides any flags set in $src/.cargo/config.toml. Instead use CARGO_BUILD_RUSTFLAGS which merges rustflags.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@szlend szlend requested a review from Ericson2314 as a code owner October 17, 2023 21:52
@szlend szlend requested review from Ericson2314 and alyssais and removed request for Ericson2314 October 17, 2023 21:52
@szlend
Copy link
Member Author

szlend commented Oct 17, 2023

Tbh I was really surprised that separateDebugInfo mangled my rustflags. I feel like adding debug flags should be a separate option and it should be up to me to set up any debug flags I want. What if I want separate debug info, but only for line tables?

Ideally if I wanted nix to set up debug flags for me I would write something like:

enableDebugInfo = true;
separateDebugInfo = true;

Though this is a breaking change so maybe we could explicitly disable debug flags instead:

enableDebugInfo = false;
separateDebugInfo = true;

Either way though we shouldn't override RUSTFLAGS.

@alyssais
Copy link
Member

Instead use CARGO_BUILD_RUSTFLAGS which merges rustflags.

Is this documented somewhere? To me the documentation implies they're aliases for each other.

@szlend
Copy link
Member Author

szlend commented Oct 17, 2023

Is this documented somewhere? To me the documentation implies they're aliases for each other.

It's not documented as such at all, but that's how it works. So it's unclear whether that's intentional or not.

@szlend
Copy link
Member Author

szlend commented Oct 17, 2023

There are four mutually exclusive sources of extra flags. They are checked in order, with the first one being used:

  • CARGO_ENCODED_RUSTFLAGS environment variable.
  • RUSTFLAGS environment variable.
  • All matching target..rustflags and target..rustflags config entries joined together.
  • build.rustflags config value.

Actually, I guess this sort of implies that CARGO_BUILD_RUSTFLAGS is not one of the mutually exclusive ways to specify rustflags.

@ofborg ofborg bot added 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Oct 18, 2023
@returntoreality
Copy link
Contributor

This will work on cargo based rust projects (which are by far the majority), but will (AFAICT) not work for meson which can also compile rust (e.g. mesa's rusticl).

@alyssais
Copy link
Member

alyssais commented Oct 18, 2023

This comment implies it won't even work for Cargo.

Have you confirmed that (proper, with source locations etc.) debug info is actually generated with this change, and that CARGO_BUILD_RUSTFLAGS isn't just ignored? I can't check myself until I have some free build capacity, probably later today or tomorrow.


What we really need here, for this and some other Rust flags problems in Nixpkgs, is a compiler wrapper for Rust, just like we have for C. All we can do until we have that is try to paper over the cracks. So far, nobody has stepped up to do that.

(cc @yu-re-ka)

@szlend
Copy link
Member Author

szlend commented Oct 18, 2023

Yes, I've been using this env var for a long time in a number of projects, typically to enable --cfg flags without which these rust projects would not compile under Nix. It's definitely not ignored and it merges fine with existing rustflags.

A rustc wrapper would also work, but we wouldn't want to override user defined RUSTC_WRAPPERs either.

@alyssais
Copy link
Member

Yes, I've been using this env var for a long time in a number of projects, typically to enable --cfg flags without which these rust projects would not compile under Nix. It's definitely not ignored and it merges fine with existing rustflags.

Okay. Is the Cargo comment wrong, then, or am I misreading it?

A rustc wrapper would also work, but we wouldn't want to override user defined RUSTC_WRAPPERs either.

RUSTC_WRAPPER wouldn't solve our problem anyway, since it's Cargo-specific. We'd do what we do for the C compiler, which is to just make the rustc executable our wrapped rustc.

@szlend
Copy link
Member Author

szlend commented Oct 18, 2023

I can't find where this is implemented but there's a unit test covering this exact scenario: https://github.com/rust-lang/cargo/blob/a275529de24d893a9a6090f1d521822589e22a14/tests/testsuite/cargo_config/mod.rs#L294-L324

Edit: Sorry, one test above. Typing this from my phone.

@szlend
Copy link
Member Author

szlend commented Oct 18, 2023

This will work on cargo based rust projects (which are by far the majority), but will (AFAICT) not work for meson which can also compile rust (e.g. mesa's rusticl).

Tbh it seems weird to me that this one flag is supposed to just enable debug symbols for all kinds of build systems. Imo this really should be up to individual builders like buildRustPackage to implement.

I think all this flag should really do is run strip on the outputs and move debug symbols into its own file.

@alyssais
Copy link
Member

alyssais commented Oct 18, 2023

Imo this really should be up to individual builders like buildRustPackage to implement.

Then we'd have to separately implement the same thing for Cargo, Meson, the kernel, who knows what else?

The solution, again, is a wrapper, which would only need to be implemented once per compiler, not once per build system.

@szlend
Copy link
Member Author

szlend commented Oct 18, 2023

Then we'd have to separately implement the same thing for Cargo, Meson, the kernel, who knows what else?

A wrapper or a builder setting these, whatever makes the most sense and avoids setting these globally. Otherwise we'll eventually be setting flags for dozens of other build systems that are irrelevant to what you're actually building. I think the same goes for CC flags which could probably go to the cc wrapper as well.

@alyssais alyssais mentioned this pull request Nov 16, 2023
13 tasks
alyssais added a commit to alyssais/nixpkgs that referenced this pull request Nov 17, 2023
Setting RUSTFLAGS causes Cargo to ignore other ways of configuring
flags, including the target-specific RUSTFLAGS options.  This broke
pkgsCross.musl64.crosvm, and was surprising to users.

Fixes: NixOS#261727
@szlend szlend closed this Nov 24, 2023
ghost pushed a commit that referenced this pull request Nov 30, 2023
Setting RUSTFLAGS causes Cargo to ignore other ways of configuring
flags, including the target-specific RUSTFLAGS options.  This broke
pkgsCross.musl64.crosvm, and was surprising to users.

Fixes: #261727
@LunNova
Copy link
Member

LunNova commented Feb 27, 2024

Would it make more sense to set CARGO_PROFILE_RELEASE_DEBUG=full here instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants