[haskell-updates] ghcWithPackages: fix ghclibdir computation with variantSuffix by sternenseemann · Pull Request #166548 · NixOS/nixpkgs

sternenseemann · 2022-03-31T10:40:08Z

This is an incorrectness pointed out in #153319 which we already have
a proper solution for.

Description of changes

Things done

This is an incorrectness pointed out in NixOS#153319 which we already have a proper solution for.

This is the correctest and clearest way to do it I can think of at the moment that doesn't need us to add anything. "${ghcCommand}-${ghc.version}" also works, but is clunkier and harder to replicate for downstream users.

This is done for consistency with generic-builder.nix and because it's easier for downstream users to replicate which will inevitably use our code as inspiration.

sternenseemann · 2022-03-31T11:14:18Z

sternenseemann · 2022-03-31T11:16:27Z

Set rebuild is caused by ghc-binary-8.10.7 being corrected to ghc-8.10.7:

- /nix/store/wbij2lvvciix06xndfhawqbhbm82y42d-hello-1.0.0.2.drv:{out}
+ /nix/store/88d4ca757hx451qr5yqs1iy179rrjab8-hello-1.0.0.2.drv:{out}
• The input derivation named `ghc-9.0.2` differs
  - /nix/store/l4r0kvvc0g496rx0gqzqlzlqg50jl8ji-ghc-9.0.2.drv:{out}
  + /nix/store/mfcjmby0f6dwhf28an4vk12ddck03j71-ghc-9.0.2.drv:{out}
  • The input derivation named `alex-3.2.7.1` differs
    - /nix/store/igwcj2lp04464lpxhspsxz0r6lnhfymz-alex-3.2.7.1.drv:{out}
    + /nix/store/ji99l4m2l1hh7k55f2sccpxm5052hskj-alex-3.2.7.1.drv:{out}
    • The input derivation named `happy-1.20.0` differs
      - /nix/store/a73wg49pg0hjbdfsagd8bjcd7rnf5har-happy-1.20.0.drv:{out}
      + /nix/store/zq2lgp22vrx4j3dkn4gjvnvzgw30wcn6-happy-1.20.0.drv:{out}
      • The environments do not match:
          setupCompilerEnvironmentPhase=''
          NIX_BUILD_CORES=$(( NIX_BUILD_CORES < 16 ? NIX_BUILD_CORES : 16 ))
          runHook preSetupCompilerEnvironment
          
          echo "Build with /nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7."
          
          
          setupPackageConfDir="$TMPDIR/setup-package.conf.d"
          mkdir -p $setupPackageConfDir
          packageConfDir="$TMPDIR/package.conf.d"
          mkdir -p $packageConfDir
          
          setupCompileFlags="-package-db=$setupPackageConfDir -j$NIX_BUILD_CORES +RTS -A64M -RTS -threaded -rtsopts"
          configureFlags="--verbose --prefix=$out --libdir=\$prefix/lib/\$compiler --libsubdir=\$abi/\$libname ←--datadir=$data/share/ghc-binary-8.10.7←→--datadir=$data/share/ghc-8.10.7→  --with-gcc=$CC --package-db=$packageConfDir   --ghc-options=-j$NIX_BUILD_CORES +RTS -A64M -RTS  --disable-split-objs --disable-library-profiling  --disable-profiling --disable-shared --disable-coverage --enable-static --disable-executable-dynamic --disable-tests --disable-benchmarks --enable-library-vanilla --disable-library-for-ghci --ghc-option=-split-sections $configureFlags"
          for p in "${pkgsBuildBuild[@]}" "${pkgsBuildHost[@]}" "${pkgsBuildTarget[@]}"; do
            # If this dependency has a package database, then copy the contents of it,
          # unless it is one of our GHCs. These can appear in our dependencies when
          # we are doing native builds, and they have package databases in them, but
          # we do not want to copy them over.
          #
          # We don't need to, since those packages will be provided by the GHC when
          # we compile with it, and doing so can result in having multiple copies of
          # e.g. Cabal in the database with the same name and version, which is
          # ambiguous.
          if [ -d ←"$p/lib/ghc-binary-8.10.7/package.conf.d"←→"$p/lib/ghc-8.10.7/package.conf.d"→ ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ]; then
            cp -f ←"$p/lib/ghc-binary-8.10.7/package.conf.d/"*.conf←→"$p/lib/ghc-8.10.7/package.conf.d/"*.conf→ $setupPackageConfDir/
            continue
          fi
          
          done
          ghc-pkg --package-db="$setupPackageConfDir" recache
          for p in "${pkgsHostHost[@]}" "${pkgsHostTarget[@]}"; do
            # If this dependency has a package database, then copy the contents of it,
          # unless it is one of our GHCs. These can appear in our dependencies when
          # we are doing native builds, and they have package databases in them, but
          # we do not want to copy them over.
          #
          # We don't need to, since those packages will be provided by the GHC when
          # we compile with it, and doing so can result in having multiple copies of
          # e.g. Cabal in the database with the same name and version, which is
          # ambiguous.
          if [ -d ←"$p/lib/ghc-binary-8.10.7/package.conf.d"←→"$p/lib/ghc-8.10.7/package.conf.d"→ ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ]; then
            cp -f ←"$p/lib/ghc-binary-8.10.7/package.conf.d/"*.conf←→"$p/lib/ghc-8.10.7/package.conf.d/"*.conf→ $packageConfDir/
            continue
          fi
          
            if [ -d "$p/include" ]; then
              configureFlags+=" --extra-include-dirs=$p/include"
            fi
            if [ -d "$p/lib" ]; then
              configureFlags+=" --extra-lib-dirs=$p/lib"
            fi
          if [[ -d "$p/Library/Frameworks" ]]; then
            configureFlags+=" --extra-framework-dirs=$p/Library/Frameworks"
          fi
          done
          ghc-pkg --package-db="$packageConfDir" recache
          
          runHook postSetupCompilerEnvironment
      ''
    • Skipping environment comparison
  • The input derivation named `happy-1.20.0` differs
    • These two derivations have already been compared
  • The input derivation named `hscolour-1.24.4` differs
    - /nix/store/8sbyxf5xln4cxbpkk8shk1a06gm4af1n-hscolour-1.24.4.drv:{out}
    + /nix/store/7m449d61zjh3jz7i9s1cgky2d541dik5-hscolour-1.24.4.drv:{out}
    • The input derivation named `hscolour-1.24.4` differs
      - /nix/store/hqw5jk50jxiadf09w8s0y6y0jrxxl3k0-hscolour-1.24.4.drv:{out}
      + /nix/store/m4jc8ax7w6siz03wa7sxvfrwxm1ipzxy-hscolour-1.24.4.drv:{out}
      • The environments do not match:
          setupCompilerEnvironmentPhase=''
          NIX_BUILD_CORES=$(( NIX_BUILD_CORES < 16 ? NIX_BUILD_CORES : 16 ))
          runHook preSetupCompilerEnvironment
          
          echo "Build with /nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7."
          
          
          setupPackageConfDir="$TMPDIR/setup-package.conf.d"
          mkdir -p $setupPackageConfDir
          packageConfDir="$TMPDIR/package.conf.d"
          mkdir -p $packageConfDir
          
          setupCompileFlags="-package-db=$setupPackageConfDir -j$NIX_BUILD_CORES +RTS -A64M -RTS -threaded -rtsopts"
          configureFlags="--verbose --prefix=$out --libdir=\$prefix/lib/\$compiler --libsubdir=\$abi/\$libname ←--datadir=$data/share/ghc-binary-8.10.7←→--datadir=$data/share/ghc-8.10.7→  --with-gcc=$CC --package-db=$packageConfDir   --ghc-options=-j$NIX_BUILD_CORES +RTS -A64M -RTS  --disable-split-objs --disable-library-profiling  --disable-profiling --disable-shared --disable-coverage --enable-static --disable-executable-dynamic --disable-tests --disable-benchmarks --enable-library-vanilla --disable-library-for-ghci --ghc-option=-split-sections $configureFlags"
          for p in "${pkgsBuildBuild[@]}" "${pkgsBuildHost[@]}" "${pkgsBuildTarget[@]}"; do
            # If this dependency has a package database, then copy the contents of it,
          # unless it is one of our GHCs. These can appear in our dependencies when
          # we are doing native builds, and they have package databases in them, but
          # we do not want to copy them over.
          #
          # We don't need to, since those packages will be provided by the GHC when
          # we compile with it, and doing so can result in having multiple copies of
          # e.g. Cabal in the database with the same name and version, which is
          # ambiguous.
          if [ -d ←"$p/lib/ghc-binary-8.10.7/package.conf.d"←→"$p/lib/ghc-8.10.7/package.conf.d"→ ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ]; then
            cp -f ←"$p/lib/ghc-binary-8.10.7/package.conf.d/"*.conf←→"$p/lib/ghc-8.10.7/package.conf.d/"*.conf→ $setupPackageConfDir/
            continue
          fi
          
          done
          ghc-pkg --package-db="$setupPackageConfDir" recache
          for p in "${pkgsHostHost[@]}" "${pkgsHostTarget[@]}"; do
            # If this dependency has a package database, then copy the contents of it,
          # unless it is one of our GHCs. These can appear in our dependencies when
          # we are doing native builds, and they have package databases in them, but
          # we do not want to copy them over.
          #
          # We don't need to, since those packages will be provided by the GHC when
          # we compile with it, and doing so can result in having multiple copies of
          # e.g. Cabal in the database with the same name and version, which is
          # ambiguous.
          if [ -d ←"$p/lib/ghc-binary-8.10.7/package.conf.d"←→"$p/lib/ghc-8.10.7/package.conf.d"→ ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ] && [ "$p" != "/nix/store/467916mp56bgz4v7lgbs226d7w5dmrmi-ghc-binary-8.10.7" ]; then
            cp -f ←"$p/lib/ghc-binary-8.10.7/package.conf.d/"*.conf←→"$p/lib/ghc-8.10.7/package.conf.d/"*.conf→ $packageConfDir/
            continue
          fi
          
            if [ -d "$p/include" ]; then
              configureFlags+=" --extra-include-dirs=$p/include"
            fi
            if [ -d "$p/lib" ]; then
              configureFlags+=" --extra-lib-dirs=$p/lib"
            fi
          if [[ -d "$p/Library/Frameworks" ]]; then
            configureFlags+=" --extra-framework-dirs=$p/Library/Frameworks"
          fi
          done
          ghc-pkg --package-db="$packageConfDir" recache
          
          runHook postSetupCompilerEnvironment
      ''
    • Skipping environment comparison
  • Skipping environment comparison
• Skipping environment comparison

sternenseemann · 2022-03-31T11:19:12Z

cc @guibou This is hopefully the complete version of what you've attempted before, testing should confirm that though.

sternenseemann · 2022-04-01T14:25:14Z

Seems like the datadirs of packages are a bit iffy, but in the same way as before, would like to look into this as well though.

Edit: We should do it separately, since it would change behavior, it has the potential to break things. The other things we are changing were just broken before!

ghcWithPackages: use packageCfgDir over ghc.name where appropriate

383fbfa

This is an incorrectness pointed out in NixOS#153319 which we already have a proper solution for.

github-actions bot added the 6.topic: haskell General-purpose, statically typed, purely functional programming language label Mar 31, 2022

sternenseemann added 3 commits March 31, 2022 12:47

ghcWithPackages: use haskellCompilerName for ghclibdir

456faf7

This is done for consistency with generic-builder.nix and because it's easier for downstream users to replicate which will inevitably use our code as inspiration.

treewide: replace uses of ghc.name to find packages' datadir

6016ed5

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Mar 31, 2022

sternenseemann marked this pull request as ready for review March 31, 2022 11:16

sternenseemann requested review from cdepillabout, expipiplus1 and maralorn as code owners March 31, 2022 11:16

sternenseemann linked an issue Mar 31, 2022 that may be closed by this pull request

ghcWithPackages environment doesn't build with integer-simple #153319

Closed

roberth mentioned this pull request Mar 31, 2022

haskellPackages: update stackage and hackage #166460

Merged

sternenseemann changed the title ~~[haskell-updates] ghcWithPackages: use packageCfgDir over ghc.name where appropriate~~ [haskell-updates] ghcWithPackages: fix ghclibdir computation with variantSuffix Apr 3, 2022

sternenseemann merged commit 103d171 into NixOS:haskell-updates Apr 3, 2022

sternenseemann deleted the fix-ghclibdir-computation branch April 3, 2022 12:22

sternenseemann mentioned this pull request Apr 7, 2022

Fully static Haskell executables - overview issue #43795

Open

60 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[haskell-updates] ghcWithPackages: fix ghclibdir computation with variantSuffix#166548

[haskell-updates] ghcWithPackages: fix ghclibdir computation with variantSuffix#166548
sternenseemann merged 4 commits intoNixOS:haskell-updatesfrom
sternenseemann:fix-ghclibdir-computation

sternenseemann commented Mar 31, 2022

Uh oh!

sternenseemann commented Mar 31, 2022 •

edited

Loading

Uh oh!

sternenseemann commented Mar 31, 2022

Uh oh!

sternenseemann commented Mar 31, 2022

Uh oh!

sternenseemann commented Apr 1, 2022 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

sternenseemann commented Mar 31, 2022

Description of changes

Things done

Uh oh!

sternenseemann commented Mar 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sternenseemann commented Mar 31, 2022

Uh oh!

sternenseemann commented Mar 31, 2022

Uh oh!

sternenseemann commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sternenseemann commented Mar 31, 2022 •

edited

Loading

sternenseemann commented Apr 1, 2022 •

edited

Loading