Skip to content

WIP: nixos: mkEnableOption on steroids for discoverable man configuration.nix#14860

Closed
oxij wants to merge 4 commits intoNixOS:masterfrom
oxij:nixos-related-packages
Closed

WIP: nixos: mkEnableOption on steroids for discoverable man configuration.nix#14860
oxij wants to merge 4 commits intoNixOS:masterfrom
oxij:nixos-related-packages

Conversation

@oxij
Copy link
Member

@oxij oxij commented Apr 21, 2016

This is a WIP on #9306. I wanted to implement this for a while now, and now it even kinda works, and generates pretty awesome man page if I may say so myself.

How to test this:

  • Merge with your local branch.

  • Do

    nix-build ./nixpkgs/nixos/release.nix -A manpages
    

    (you might need to set your NIX_PATH)

  • Run man result/share/man/man5/configuration.nix.5.

  • Lo and behold.

  • Search, e.g. /tmux.

  • Lo and behold again.

screenshot

Current problems:

  • Needs a ton of refactoring in nixos/modules, I rewrote a bunch of pieces just for testing purposes, but grep shows that there's a lot of work there. Hopefully (but not likely), I will work that out till the end of this week.

  • Ideally, this code needs to access metas of all the packages, but broken derivations, derivations with unfree licenses, and derivations with assertions evaluate to exceptions. That's even more crazy: they fail differently:

    • Broken and unfree packages fail on p.meta access (in stdenv.mkDerivation).
    • Packages with assertions fail on callPackage, that is, access to p, that is, even before mkDerivation. Which makes sense, but that's too strict for both pretty and complete solution I had in mind.

    For now I just commented unfree (because I didn't want to mark them as "broken") and assert-failing packages out.

    The former case can probably be fixed by some kind of stdenv hack that pushes meta out as early as possible.
    The latter needs nix support for catching exceptions (and assertions) or a new style of derivation definitions a-la "nixos modules with assertions".
    If this got to be implemented somehow I would totally start generating and then searching a separate man page with descriptions of all the available packages instead of nix-env -q.

  • As it turns out, quite a lot of nixos modules are nonfunctional simply because they refer to nonexistent packages. Rewriting them using this mechanisms highlights all of those previously invisible problems (which is good, but annoying).

  • Names of the new functions are ugly, better ideas are very much welcome.

Questionable design decisions:

  • The very first patch in the sequence. I think it's awesome, but your OCD towards lexicographical ordering might vary.
  • packages in mkOption. At first I did a separate mkOption-variant for this, but then all the functions using it got really ugly. So I decided that emulating TeX bibliography is okay (the idea is that the last paragraph in the description says "what are those packages listed here for").
Things done
  • Tested using sandboxing (nix-build --option build-use-chroot true or nix.useChroot on NixOS)
  • Built on platform(s)
    • NixOS
    • OS X
    • Linux
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

oxij added 3 commits April 20, 2016 22:34
…unctions

This allows one to specify "related packages" in NixOS that get rendered into
the configuration.nix(5) man page. The interface philosophy is pretty much
stolen from TeX bibliography.
@mention-bot
Copy link

By analyzing the blame information on this pull request, we identified @edolstra, @maggesi and @shlevy to be potential reviewers

@oxij oxij force-pushed the nixos-related-packages branch from 97b309c to 78b784b Compare April 21, 2016 07:56
@edolstra
Copy link
Member

This PR doesn't really describe what it does. What does the generated manpage look like?

👎 on function names like mkWheneverToPkgOption, I have no idea what that does. In general, I don't like adding more abstraction/obfuscation to the NixOS module system - when I see mkOption I know what it does; functions like mkEnableOption' just make it harder to understand a module.

@oxij
Copy link
Member Author

oxij commented Apr 21, 2016

This PR doesn't really describe what it does. What does the generated manpage look like?

It (mostly) does #9306, but I guess it's easier to show, than to explain. Added a screenshot to the top.

👎 on function names like mkWheneverToPkgOption, I have no idea what that does. In general, I don't like adding more abstraction/obfuscation to the NixOS module system - when I see mkOption I know what it does; functions like mkEnableOption' just make it harder to understand a module.

Yep, names are subject to change. mkWheneverToPkgOption would probably
be gone too. This is just an experiment for me and a preview for you
that should hopefully show me whether there's a chance to merge this
upstream (or should I just relax and implement it on top of my own
branch full of awesome rejected things).

NixOS already has mkEnableOption, I just implemented a better one
(currently by copy-paste, but I plan to drop/deprecate current
mkEnableOption after I refactor everything). Currently
mkEnableOption is actually enforced in new PRs.

So, you can safely assume that the current set of patches won't add any
new mkOption-like functions (or one, in case of mkEnableOption
deprecation).

Moreover, I actually plan to add another mkOption variant for
whatever.package'-like options. E.g. considerfonts.fonts` option for
a moment. Wouldn't it be awesome to list most of (or even all) the
available fonts in the man page? My current implementation (out of this
branch yet) does this by:

font.fonts = mkPackageListOption {
  name = "List of primary font paths.";
  default = literalPackages pkgs [ "pkgs.some" "pkgs.font" ];
  example = literalPackages pkgs [ "pkgs.dejavu_fonts" ];
  packages = literalPackages pkgs [ "all" "other" "fonts" ];
};

which reduces to

font.fonts = mkOption {
  name = "List of primary font paths.";
  default = [ pkgs.some pkgs.font ];
  defaultText = ''[ "pkgs.some" "pkgs.font" ]'';
  example = literalExample ''[ "pkgs.dejavu_fonts" ]'';
  packages = literalPackages pkgs [ "pkgs.some" "pkgs.font" "pkgs.dejavu_fonts" "all" "other" "fonts" ];
};

Moreover, if one to group all the fonts in top-level/all-packages.nix
into a common attrset ''"all" "other" "fonts"'' part could become
completely automatic.

Can we agree that this is cool? (I would then also try to drop
`defaultText' thing altogether, at least from the nixos modules, because
it's only ever used in this type of use case.)

Ask away if something is still unclear.

@oxij
Copy link
Member Author

oxij commented Apr 21, 2016

/cc @nbp @bjornfor from #9306

@joachifm
Copy link
Contributor

joachifm commented Apr 21, 2016

I've just read through the patch (and your comments), and I honestly don't get what it's doing. Is there a 5-word summary for us mere mortals? A better mkEnableOption sounds great.

@oxij
Copy link
Member Author

oxij commented Apr 21, 2016

You write this
https://github.com/NixOS/nixpkgs/pull/14860/files#diff-d82ed2c982beffa445203678d076e1edR16
and get this
screenshot

@oxij
Copy link
Member Author

oxij commented Apr 21, 2016

After thinking some more about this, I think I can push all of this into
types. How does

whatever.enable = mkOption {
  type = types.service;
  default = true;
  description = "whatever service";
  package = whatever.package.default;
};

whatever.package = mkOption {
  type = types.package;
  default = literalPackage pkgs "pkgs.whatever";
  example = literalPackage pkgs "pkgs.whatever-alternative";
  packages = literalPackages pkgs [ "other" "options" ];
  description = "whatever service package";
};

look? We can probably even specify default defaults [sic!] in types if
desired. Then mkOption alone would be sufficient and we could throw
away mkEnableOption and all the others.

@edolstra?

@nbp
Copy link
Member

nbp commented Apr 21, 2016

I agree with @edolstra , adding additional mk*Option is a bad idea, and replacing mkEnableOption by the simple mkOption with a proper type sounds like a better approach to me.

In #14860 (comment) I will suggest that you rename package to descriptionPackage. On the other hand the option declaration should not depend on options definitions. Thus descriptionPackage should default to one version of the package, and not one set by whatever option.

I do not understand what packages is supposed to be used for.

@oxij
Copy link
Member Author

oxij commented Apr 21, 2016

Okay then, type hackery it is.

"> I do not understand what packages is supposed to be used for."

Multiple possible options and/or types.listOf types.path. See e.g.
fonts.fonts example above, same should, imho, be done for xorg modules
and other similar stuff.

(bah, github mail reply cites broke)

@ericsagnes
Copy link
Contributor

In relation to the font example, you might be interested by how the input methods plugins option description is generated. (It use the attribute set trick you described)

Also the input method packages option use an extended package type that check a meta attribute of the package so an unrelated package cannot be used. (the main motivation was that different input methods have similar plugins names so it is easy to mistake a ibus plugin with a fcitx one)

In the input method case, it would be nice and more flexible to generate the option description by automatically selecting packages with the right meta attribute, but as far as I know there is no cheap way to do that at the moment.

@nbp
Copy link
Member

nbp commented Apr 23, 2016

@oxij

I do not understand what packages is supposed to be used for.

Multiple possible options and/or types.listOf types.path. See e.g. fonts.fonts example above, same should, imho, be done for xorg modules and other similar stuff.

I still do not understand what this is supposed to mean. Is that a way to limit the set of packages that you can use as argument, like an enum?

@groxxda
Copy link
Contributor

groxxda commented May 2, 2016

I really like the idea of listing the supported alternatives for module options 👍 personally I'd prefer nixos-option to list them and not having to look them up in a man page but that would probably be easy to add.

I'm just not sure if the options should be hard-coded into the module. Maybe the package should be able to link itself to a module (-option) (something like meta.provides = [ services.xyz.package ])?

@olejorgenb
Copy link
Contributor

olejorgenb commented Aug 15, 2016

The latter needs nix support for catching exceptions (and assertions)

builtins.tryEval catches (some) errors. broken errors for instance is caught. It's not really documented, but this article have some information.

@jagajaga
Copy link
Member

Ping all.

@oxij
Copy link
Member Author

oxij commented Nov 1, 2016

Arseniy Seroka [email protected] writes:

Ping all.

I remember about this, but my hand are currently pretty full with other
stuff. If someone wants to implement this as described above (with new
types) I will be happy to comment on those patches.

@fpletz
Copy link
Member

fpletz commented Aug 15, 2017

Feel free to reopen if there is still interest in this.

@fpletz fpletz closed this Aug 15, 2017
@oxij oxij deleted the nixos-related-packages branch August 29, 2017 11:21
@oxij oxij restored the nixos-related-packages branch December 7, 2017 17:17
@oxij
Copy link
Member Author

oxij commented Dec 7, 2017

Hm. Okay. How do I reopen this? There's no such button.

@oxij
Copy link
Member Author

oxij commented Dec 7, 2017

Well, whatever. Please see #32424.

@oxij
Copy link
Member Author

oxij commented Dec 22, 2017 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.