Skip to content

Comments

Revive multiple outputs for Haskell packages.#32112

Merged
peti merged 2 commits intoNixOS:masterfrom
nc6:hask-split-out
Dec 4, 2017
Merged

Revive multiple outputs for Haskell packages.#32112
peti merged 2 commits intoNixOS:masterfrom
nc6:hask-split-out

Conversation

@nc6
Copy link
Contributor

@nc6 nc6 commented Nov 27, 2017

Incorporates fix to ghcWithPackages from domenkozar.

@nc6 nc6 requested a review from peti as a code owner November 27, 2017 16:12
@GrahamcOfBorg GrahamcOfBorg added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels Nov 27, 2017
@vcunat
Copy link
Member

vcunat commented Nov 27, 2017

@peti: through your https://github.com/peti/nixpkgs haskell-updates -> https://hydra.nixos.org/jobset/nixpkgs/haskell-updates or you want it some other way?

@vcunat vcunat added the 6.topic: haskell General-purpose, statically typed, purely functional programming language label Nov 27, 2017
@peti
Copy link
Member

peti commented Nov 27, 2017

@domenkozar
Copy link
Member

41 failing looks pretty good :D

@domenkozar
Copy link
Member

@peti I went through all of failures and they are not related to this PR. What else is missing to merge this?

@peti
Copy link
Member

peti commented Dec 3, 2017

It would be great if someone could test whether functions like ghcWithPackages and ghcWithHoogle actually work properly in that branch.

@nc6
Copy link
Contributor Author

nc6 commented Dec 3, 2017

@peti

[nc@varda:~/nixpkgs.hask-split-out]$  nix-shell -p "haskellPackages.ghcWithPackages (ps: [ps.turtle])" -I .

[nix-shell:~/nixpkgs.hask-split-out]$ ghci 
GHCi, version 8.0.2: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/nc/.ghc/ghci.conf
λ> import Turtle
(0.08 secs, 0 bytes)

@peti
Copy link
Member

peti commented Dec 3, 2017 via email

@nc6
Copy link
Contributor Author

nc6 commented Dec 3, 2017

[nc@varda:~/nixpkgs.hask-split-out]$  nix-shell -p "haskellPackages.ghcWithHoogle (ps: [ps.turtle])" -I .

[nix-shell:~/nixpkgs.hask-split-out]$ hoogle "MonadIO io => Shell a -> FoldM IO a r -> io r"
Turtle.Shell foldIO :: MonadIO io => Shell a -> FoldM IO a r -> io r
Turtle.Bytes shellStrictWithErr :: MonadIO io => Text -> Shell ByteString -> io (ExitCode, ByteString, ByteString)
Turtle.Bytes procStrict :: MonadIO io => Text -> [Text] -> Shell ByteString -> io (ExitCode, ByteString)
Turtle.Bytes procStrictWithErr :: MonadIO io => Text -> [Text] -> Shell ByteString -> io (ExitCode, ByteString, ByteString)
CoreMonad liftIO2 :: MonadIO m => (a -> b -> IO c) -> a -> b -> m c
MonadUtils liftIO2 :: MonadIO m => (a -> b -> IO c) -> a -> b -> m c
CoreMonad liftIO3 :: MonadIO m => (a -> b -> c -> IO d) -> a -> b -> c -> m d
MonadUtils liftIO3 :: MonadIO m => (a -> b -> c -> IO d) -> a -> b -> c -> m d
Data.StateVar ($~) :: (HasUpdate t a b, MonadIO m) => t -> (a -> b) -> m ()
Data.StateVar ($~!) :: (HasUpdate t a b, MonadIO m) => t -> (a -> b) -> m ()
-- plus more results not shown, pass --count=20 to see more

@nc6
Copy link
Contributor Author

nc6 commented Dec 3, 2017

@peti See above!

@peti peti merged commit 7993e87 into NixOS:master Dec 4, 2017
@orivej
Copy link
Contributor

orivej commented Dec 4, 2017

Could not this mass rebuild have gone via staging? I had to revert it in my local checkout just to deploy hydra without building ghc myself.

@vcunat
Copy link
Member

vcunat commented Dec 4, 2017

I think it had its Hydra job, but staging got merged into master in the meantime...

@orivej
Copy link
Contributor

orivej commented Dec 4, 2017

Oh, well, it happens…

@vcunat
Copy link
Member

vcunat commented Dec 4, 2017

Problem: output cycle in happy on Darwin: https://hydra.nixos.org/build/65352540 (killing ~1.3k builds)

@vcunat
Copy link
Member

vcunat commented Dec 4, 2017

@LnL7
Copy link
Member

LnL7 commented Dec 5, 2017

The cycles between out and bin are caused by #25537

@peti
Copy link
Member

peti commented Dec 5, 2017

ghcWithPackages appears to be broken again: #32332. I cannot even reproduce the example reported as success in #32112 (comment).

@domenkozar
Copy link
Member

@peti I will fix this today.

@peti
Copy link
Member

peti commented Dec 5, 2017

I thought it was fixed already? I'm a little disappointed, to be honest. This is the second time this PR has completely broken the Haskell infrastructure and it seems pretty obvious that no-one has ever used this code before. This does not inspire confidence.

@domenkozar
Copy link
Member

@peti I have fixed your borken example as per output in #32082

It's unfair to expect things are working when there is no indication what is intended to work.

I'll take another look and add tests.

@peti
Copy link
Member

peti commented Dec 5, 2017

It's unfair to expect things are working when there is no indication what is intended to work.

I beg your pardon? The expectation that ghcWithPackages can successfully create a working development environment was clearly communicated in #32112 (comment), was it not?

@domenkozar
Copy link
Member

domenkozar commented Dec 5, 2017

@peti did you read the comment next to yours that shows that?

@peti
Copy link
Member

peti commented Dec 5, 2017

I am sorry, but I don't follow. Which comment are you referring to? And what exactly does that comment show?

@nc6
Copy link
Contributor Author

nc6 commented Dec 5, 2017

@peti This was tested before the PR was opened. I can replicate your example quite happily on my branch:

[nc@varda:~/nixpkgs.hask-split-out]$ nix-shell -p "haskellPackages.ghcWithPackages (ps: [ps.conduit])" -I .

[nix-shell:~/nixpkgs.hask-split-out]$ ghci
GHCi, version 8.0.2: http://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/nc/.ghc/ghci.conf
λ> import Data.Conduit
(0.02 secs, 0 bytes)

That it's broken on master is annoying, but it's not due to this being untested.

@peti
Copy link
Member

peti commented Dec 5, 2017

@nc6, are you sure that you're actually building the branch you intend to build? I.e. does the -I . flag really achieve what it's intended to? It looks to me like you mean to use -I nixpkgs=$PWD, but obviously I cannot know for sure. All I can say is that this very example fails when I run it on my machine,

@globin
Copy link
Member

globin commented Dec 5, 2017

Can confirm this works on master:

$ git show           
commit 9f90700b564f35caa4ca28aa8d56cfffd4d1ae6f (HEAD -> up, upstream/master)
Merge: fd7953d95f4 62e6f2eb1dc
Author: Jörg Thalheim <[email protected]>
Date:   Tue Dec 5 10:47:55 2017 +0000

    Merge pull request #32336 from Mic92/vim-codefmt
    
    vim-codefmt: init at 2017-03-04

$ NIX_PATH=nixpkgs=. nix-shell -p "haskellPackages.ghcWithPackages (ps: [ps.conduit])"             

[nix-shell:~/dev/nixpkgs-upstream]$ ghci
GHCi, version 8.0.2: http://www.haskell.org/ghc/  :? for help
Prelude> import Data.Conduit
Prelude Data.Conduit> 

@peti
Copy link
Member

peti commented Dec 5, 2017

@globin, can you please try a revision prior to my revert (8d8061e), i.e. e5e3282?

@vcunat
Copy link
Member

vcunat commented Dec 5, 2017

Right, -I . won't work as expected, at least unless you add a symlink nixpkgs -> .

@globin
Copy link
Member

globin commented Dec 5, 2017

Ah sry didn't notice your revert, fails with:

There are problems in package conduit-1.2.12.1:
  dependency "exceptions-0.8.3-ADEDRATtUrMK1JW4zMQ9U" doesn't exist
  dependency "lifted-base-0.2.3.11-Ei0JSPdt7KOCD2HIA5jFqV" doesn't exist
  dependency "mmorph-1.0.9-5bASEvmIkr5CpdAYXwPsn0" doesn't exist
  dependency "monad-control-1.0.2.2-1Ci8dYJ8Tb7E3HTcTqOb8l" doesn't exist
  dependency "mtl-2.2.1-BLKBelFsPB3BoFeSWSOYj6" doesn't exist
  dependency "primitive-0.6.2.0-4578caNkWQ54Gt1mxLF2Yh" doesn't exist
  dependency "resourcet-1.1.9-L2BUQrxWZMwDiwhcuyPuNi" doesn't exist
  dependency "transformers-base-0.4.4-GKSs0LKukAL8ymq2naFmrH" doesn't exist
  dependency "transformers-compat-0.5.1.4-IuFogs8HAVUJBWVNMhtssu" doesn't exist

The following packages are broken, either because they have a problem
listed above, or because they depend on a broken package.
conduit-1.2.12.1
builder for '/nix/store/vc9453svk0na3w237x6zvbsia0qph9j0-ghc-8.0.2-with-packages.drv' failed with exit code 1

@nc6
Copy link
Contributor Author

nc6 commented Dec 5, 2017

Oh, darn. Yeah, if I try with -I nixpkgs=$PWD, I see the same :-(

@domenkozar
Copy link
Member

@nc6 I'll give this another try

@domenkozar
Copy link
Member

I have found the issue, symlinkJoin doesn't handle multiple outputs at all. This is a bit deeper rabbit hole than I thought, will give it a few more days.

@domenkozar
Copy link
Member

Fixed! Only darwin left: #32082 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: haskell General-purpose, statically typed, purely functional programming language 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants