binutils: 2.31.1 -> 2.33.1 by guibou · Pull Request #78204 · NixOS/nixpkgs

guibou · 2020-01-21T19:21:28Z

Motivation for this change

Bump of binutils to a more recent version. It closes #78197 . Please review with care because I have NO idea what I'm doing. This will trigger a mass rebuild and mostly everything in nixpkgs depend on this.

I'm opening this PR as a proof of motivation to work on this issue, but really, feel free to tell me that I totally missed something and that binutils is stuck at 2.31.1 on nixpkgs for good reasons.

Things done

I bumped the package and ensured patchs applies. This is a mass rebuild, so I did not even tried to wait for the end of the build on my laptop. At least, patches are correctly applied (after fixup) and the build is running.

- I've removed the stack of patch linked to https://sourceware.org/bugzilla/show_bug.cgi?id=23428 . The associated issue says it is closed and targeted for 2.32. - I've ugraded the "no_plugin" patch. The logic changed in https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=41f37a6fb71f2a3de388108f5cdfca9cbe6e9d51 and I tried to keep the same logic by disabling everything. It closes NixOS#78197

tomberek · 2020-02-02T04:40:05Z

I get a few failures with coreutils:

tests/misc/mknod.sh
tests/cp/parent-perm-race.sh
tests/mkdir/perm.sh

seems to be the same ones as grahamcofborg gets (x86_64-linux, NixOS)

lovesegfault · 2020-04-23T20:45:24Z

I get some test failures with this. Here's the full build log

lovesegfault · 2020-04-23T20:50:44Z

Also: binutils-2.34 has since been released.

lovesegfault · 2020-04-23T21:34:32Z

Here's the patch to apply to this PR to get 2.34:

diff --git a/pkgs/development/tools/misc/binutils/default.nix b/pkgs/development/tools/misc/binutils/default.nix
index e1ff92d00f4..ebbddbb2190 100644
--- a/pkgs/development/tools/misc/binutils/default.nix
+++ b/pkgs/development/tools/misc/binutils/default.nix
@@ -8,15 +8,13 @@
 , bison ? null
 , flex
 , texinfo
+, perl
 }:
 
 let
   reuseLibs = enableShared && withAllTargets;
 
-  # Remove gold-symbol-visibility patch when updating, the proper fix
-  # is now upstream.
-  # https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=330b90b5ffbbc20c5de6ae6c7f60c40fab2e7a4f;hp=99181ccac0fc7d82e7dabb05dc7466e91f1645d3
-  version = "2.33.1";
+  version = "2.34";
   basename = "binutils";
   # The targetPrefix prepended to binary names to allow multiple binuntils on the
   # PATH to both be usable.
@@ -31,7 +29,7 @@ let
   # HACK to ensure that we preserve source from bootstrap binutils to not rebuild LLVM
   normal-src = stdenv.__bootPackages.binutils-unwrapped.src or (fetchurl {
     url = "mirror://gnu/binutils/${basename}-${version}.tar.bz2";
-    sha256 = "1cmd0riv37bqy9mwbg6n3523qgr8b3bbm5kwj19sjrasl4yq9d0c";
+    sha256 = "1rin1f5c7wm4n3piky6xilcrpf2s0n3dd5vqq8irrxkcic3i1w49";
   });
 in
 
@@ -64,10 +62,6 @@ stdenv.mkDerivation {
     # cross-compiling.
     ./always-search-rpath.patch
 
-  ] ++ lib.optionals (!stdenv.targetPlatform.isVc4)
-  [
-    # https://sourceware.org/bugzilla/show_bug.cgi?id=22868
-    ./gold-symbol-visibility.patch
   ] ++ lib.optional stdenv.targetPlatform.isiOS ./support-ios.patch;
 
   outputs = [ "out" "info" "man" ];
@@ -75,9 +69,11 @@ stdenv.mkDerivation {
   depsBuildBuild = [ buildPackages.stdenv.cc ];
   nativeBuildInputs = [
     bison
+    perl
+    texinfo
   ] ++ (lib.optionals stdenv.targetPlatform.isiOS [
     autoreconfHook
-  ]) ++ lib.optionals stdenv.targetPlatform.isVc4 [ texinfo flex ];
+  ]) ++ lib.optionals stdenv.targetPlatform.isVc4 [ flex ];
   buildInputs = [ zlib gettext ];
 
   inherit noSysDirs;

lovesegfault · 2020-04-23T21:44:05Z

2.34 fails in the same way (build log)

Here are the salient points:

coreutils> FAIL: tests/misc/mknod
coreutils> ======================
coreutils> mknod: cannot set permissions of 'f1': Function not implemented
coreutils> mkfifo: cannot set permissions of 'f2': Function not implemented
coreutils> mkdir: cannot change permissions of 'f3': Function not implemented
coreutils> FAIL tests/misc/mknod.sh (exit status: 1)

coreutils> FAIL: tests/cp/parent-perm-race
coreutils> ===============================
coreutils> cp: preserving permissions for 'd/ownership': Function not implemented
coreutils> FAIL tests/cp/parent-perm-race.sh (exit status: 1)

coreutils> FAIL: tests/mkdir/perm
coreutils> ======================
coreutils> mkdir: cannot change permissions of 'parent/sub': Function not implemented
coreutils> parent/sub: expected d-wx--x-w-, got d-wx----w-
coreutils> mkdir: cannot change permissions of 'parent/sub': Function not implemented
coreutils> parent/sub: expected d-wx--x-w-, got d-wx----w-
coreutils> FAIL tests/mkdir/perm.sh (exit status: 1)

lovesegfault · 2020-04-23T21:46:36Z

Checking whether coreutils 8.32 fixes the issue.

lovesegfault · 2020-04-23T21:49:51Z

coreutils 8.32 fails in the exact same way. Unclear to me why this is happening.

lovesegfault · 2020-04-23T21:57:26Z

Could this have something to do with how the fs where these tests run is mounted? cc. @andi @flokli.

flokli · 2020-04-24T11:25:54Z

@lovesegfault We disable various syscalls that might produce files which we can't serialize into the nix-store, see https://github.com/NixOS/nix/blob/master/src/libstore/build.cc#L3001-L3028.

In the case of coreutils' tests/cp/parent-perm-race, it tries to chmod g+s a directory, which is disallowed by the seccomp configured inside the first part of the linked code.

For now, I propose disabling these problematic tests, if they're all because of these restrictions. I have to admit, I haven't looked into the other failing tests :-).

Back to "how can we actually enable these tests": Nix runs builds as unprivileged users, and sources are chown'ed to the build user. So we might be able to at least lift the setuid/setgid binary restrictions, if we check the outputs after the build to not contain these bits.

The reasoning behind is that build processes should be fine to create setuid/setgid binaries at will, as long as they are not persisted in the outputs. Assuming the testsuite puts these artifacts in a temporary directory, or cleans up afterwards, there shouldn't be setuid/setgid binaries in the output.

I assume we already walk over all the files once after the build, so checking for no leftover setuid/setgid bits and bailing out if we see something shouldn't be that costly.

Edit: Contains related information, but is not what we'd actually want here: NixOS/nix#2522

flokli · 2020-04-24T20:41:07Z

closing in favor of #85951.

ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch. labels Jan 21, 2020

ofborg bot requested a review from Ericson2314 January 21, 2020 19:40

FRidh added this to the 20.09 milestone Feb 16, 2020

lovesegfault mentioned this pull request Apr 24, 2020

binutils: 2.31.1 -> 2.34 #85951

Merged

10 tasks

flokli closed this Apr 24, 2020

lovesegfault mentioned this pull request May 5, 2020

binutils: 2.31.1 -> 2.34 #86954

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

binutils: 2.31.1 -> 2.33.1#78204

binutils: 2.31.1 -> 2.33.1#78204
guibou wants to merge 1 commit intoNixOS:stagingfrom
guibou:binutils_bump

guibou commented Jan 21, 2020

Uh oh!

tomberek commented Feb 2, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020 •

edited

Loading

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

flokli commented Apr 24, 2020 •

edited

Loading

Uh oh!

flokli commented Apr 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

guibou commented Jan 21, 2020

Motivation for this change

Things done

Uh oh!

tomberek commented Feb 2, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

lovesegfault commented Apr 23, 2020

Uh oh!

flokli commented Apr 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flokli commented Apr 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lovesegfault commented Apr 23, 2020 •

edited

Loading

flokli commented Apr 24, 2020 •

edited

Loading