Quick summary

• nix blows up number of -L flags passed to GCC
• GCC passes all -L flags to its subprogram cc1 via an environment variable
• If that's longer than 128 KB, then everything crashes with E2BIG

Details

So once again, I'm building some Haskell with lots of library dependencies.

Trying to upgrade this build to 18.03, the nix-build of the Haskell package newly fails with

Setup: Missing dependencies on foreign libraries:
* Missing C libraries: glog

This error message is wrong (cabal doesn't correctly propagate the argument list too long error shown below; I filed a bug about it being misleading at haskell/cabal#5355); the dependency exists.

The problem is that because in some eventual gcc invocation the amount of arguments passed to GCC turns out to be very long.

strace confirms:

strace -fye execve -s 100000000 -v runhaskell Setup.hs build 2>&1 | grep E2BIG

execve("cc1", [arguments here], [env vars here, "COLLECT_GCC_OPTIONS='-fno-stack-protector -L... 150KB of -L flags here'"] = -1 E2BIG (Argument list too long)

E2BIG (Argument list too long), in this case because COLLECT_GCC_OPTIONS is longer than 128 KB (32 * 4 KB pages, see here, and a repro script I made here).

What is the COLLECT_GCC_OPTIONS environment variable? It is an environment variable set by gcc before calling out to cc1, over which it communicates flags to cc1. Most (if not all?) flags given to gcc will make it into this variable. So it can grow very big (easily larger than the 128 KB limit, especially on nix).

Note that even flags given in a "response file" via gcc @myresponsefile.rsp (which was designed to pass GCC flags via a file instead of command line args to circumvent command line arg limits) will be put into COLLECT_GCC_OPTIONS by gcc itself to communicate them to cc1 (I have just confirmed that with a small example on my Ubuntu 16.04). So using @myresponsefile.rsp is not a workaround. (Yes, this seems to defeat the purpose of response files, but I suspect those were originally made to circumvent a much smaller limit of command line argument on Windows, where the limit is well below the 128 KB limit for environment variable lengths on Linux).

Aside: nix inflates the number of -L flags by the fact that each -L option to gcc is present multiple times, but those duplicates make only for factor 4x or so; even if they were deduplicated, I'd already be at half of MAX_ARG_STRLEN with my medium-size Haskell project; so if I added a couple more dependencies to my Haskell project (all recursive nix Haskell dependencies make it as -L options into the gcc command line), I'd quickly exceed that limit again even without duplication.

Problems to fix

1. Deduplicating the -L flags passed to GCC will help the issue by a small constant factor and make a couple more projects compile, but won't help with projects with many dependencies.
2. The fundamental issue seems to be a GCC problem (its way of passing arbitrarily sized information via environment variables to a direct child program doesn't work), so technically it's not nix's department. But we need to do something about it, because otherwise we can't build large Haskell projects (on nix or otherwise).

Steps to reproduce

• Build a Haskell project with lots of dependencies on nixpkgs 18.03.
  • update: and lots of native C dependencies, and lots of executable sections in the cabal file, see comment below

Environment

• on top of nixpkgs commit a0b977b; tested on both NixOS and Ubuntu 16.04