Add Nixpkgs entrypoint lib, `<flake>.lib.nixpkgs.configure`, `(import <nixpkgs>).configure` by roberth · Pull Request #402298 · NixOS/nixpkgs

roberth · 2025-04-27T13:57:33Z

A clean entrypoint into Nixpkgs not too burdened by 20 years of history.
Add a place for functions that don't depend on any configuration.
This is a simpler alternative to nixpkgs: Allow passing {host,build}Platform directly + Nixpkgs fn docs #324614
Related Add nixpkgsFlake.lib.lib #398332

Things done

Add a 👍 reaction to pull requests you find important.

A clean entrypoint into Nixpkgs not too burdened by 20 years of history.

wolfgangwalther · 2025-04-27T15:19:04Z

I've been thinking a bit about the nixpkgs entrypoint / how to call "nixpkgs the function" in the last couple of days. I'm coming from #380342 and how the top-level package sets like pkgsCross, pkgsStatic & co. are just really bad in some ways. In #400351 (comment), I argued that there are two aspects to those package sets:

hiding knowledge of different flags / combinations behind a single keyword
a much better CLI interface, because it's based on attr paths

Those aspects are orthogonal to this PR here, which is precisely about the other case - calling nixpkgs from code with a flexible interface. But I came to some of the same conclusions and was thinking about how to implement an entirely new entrypoint, that doesn't need to stay backwards-compatible. With a new interface, which would do the "composition" (e.g. cross + static etc.) before even calling nixpkgs, we could avoid #380342 and friends entirely.

I think it makes sense to think about both things at the same time and implement something that works well for both. For example, if we find a better way to say "I want a fully static package sets" instead of pkgsStatic, then this is something that people might want to use via code as well, so that they don't need to replicate the knowledge that is in the definition of pkgsStatic themselves. Possibly even more so for pkgsCross / the system/examples.nix file.

The __functor (TIL) approach nicely merges the traditional and the new interfaces and avoids calling nixpkgs twice. The way, that I came up with, to achieve the same via CLI is to add a new top-level folder. In my local experiments, I called this folder overrides, because it would contain overrides for the nixpkgs-function arguments. I think this folder could equally well be called configure, though - which I quite like.

The idea would then be to turn:

nix-build -A pkgsStatic.hello into nix-build configure -A static.pkgs.hello or
nix build '.#pkgsStatic.hello' into nix build './configure#static.pkgs.hello.

I'm not too happy with the latter, and also experimented with a fake derivation in the flake's packages output, which would allow it more like nix build '.#configure.static.pkgs.hello', but that would essentially take away the name configure from the package set for the old interface.

Those attribute chains could then be arbitrarily nested, so we could for example do cross.x86_64-freebsd.static.pkgs.hello. .pkgs ends the configuration part and actually calls the nixpkgs function. By making the configure function proposed in this PR an attrset with __functor itself, we could also expose those attributes in the same for usage via code. Thus, we could do something like <flake>.lib.nixpkgs.configure.cross.x86_64-freebsd.static.pkgs etc. (Note: I am surely not bound to the term "cross"...)

I'd really like to hear your thoughts on this.

wolfgangwalther · 2025-04-27T15:26:11Z

flake.nix

+          However, not everything that is specific to Nixpkgs is _relative_ to the _configured_ package set.
+
+          This library is specifically for un-configured things such as the `configure` function, which is used to produce the configured package set.
+          Other functions and values that relate to packages without referring to instantiated packages can be added here too.


Suggested change

Other functions and values that relate to packages without referring to instantiated packages can be added here too.

Other functions and values that relate to packages without referring to instantiated packages can be added here, too.

Is this the right place for the comment, though? The place to add more functions is not here, but in pkgs/abs/default.nix, right? i.e. the functions should be part of lib.nixpkgs.<new_function>, right?

Seems like the whole big comment would be better off at the top of abs/default.nix?

Not really but it works with :doc in the repl.
Something about adding functions is more the topic of an internal # comment.

wolfgangwalther · 2025-04-27T15:30:28Z

pkgs/abs/default.nix

The meaning of the /abs/ folder is not immediately clear to me. I'd argue that this should not be inside pkgs/ at all, because pkgs implies the result of calling this function.

Given the CLI-entrypoint-idea in my previous comment, this could theoretically move to configure/ - although that would not work well for other functions than configure in the future.

The meaning of the /abs/ folder is not immediately clear to me.

Remnant of a working title, "absolute", as opposed to relative to configuration parameters.

I'd argue that this should not be inside pkgs/ at all, because pkgs implies the result of calling this function.

In my mind, pkgs was always the Nixpkgs part of the nixpkgs repo, coincidentally matching the pkgs identifier, but you're right. It's confusing this way.
How about nixpkgs/? Then we have a decent correspondence between this repo's components (including potentially as libraries) and their paths. Then only significant exception is that pkgs/ isn't inside of nixpkgs/ for historical reasons and convenience.

How about nixpkgs/? Then we have a decent correspondence between this repo's components (including potentially as libraries) and their paths. Then only significant exception is that pkgs/ isn't inside of nixpkgs/ for historical reasons and convenience.

If we put this into nixpkgs/, then the CLI interface would become something like nix-build nixpkgs -A static.pkgs. Cool.

The nice thing is, that we could also add a "fake derivation" to packages for the flake, which would allow something like nix build '.#nixpkgs.static.pkgs - because the top-level package name "nixpkgs" is really not needed.

Then only significant exception is that pkgs/ isn't inside of nixpkgs/ for historical reasons and convenience.

Maybe that's not too bad anyway - because in a way pkgs are used for both "nixpkgs" and "NixOS". Somebody running NixOS doesn't really need to know about the difference? Those packages are available via { pkgs, ... }, so they live in pkgs/ - cool.

(Not sure whether I convinced myself with that, though ... :D)

wolfgangwalther · 2025-04-27T15:34:52Z

pkgs/abs/default.nix

+      elaborateSomewhat = platform: if isString platform then { system = platform; } else platform;
+
+      hostPlatform = elaborateSomewhat parameters.hostPlatform;
+      buildPlatform = elaborateSomewhat parameters.buildPlatform;


This is what lib.systems.systemToAttrs does, we can call it here.

Suggested change

elaborateSomewhat = platform: if isString platform then { system = platform; } else platform;

hostPlatform = elaborateSomewhat parameters.hostPlatform;

buildPlatform = elaborateSomewhat parameters.buildPlatform;

hostPlatform = lib.systems.systemToAttrs parameters.hostPlatform;

buildPlatform = lib.systems.systemToAttrs parameters.buildPlatform;

I vaguely remembered, but I didn't take the time to look up the new proper solution yet ❤️

wolfgangwalther · 2025-04-27T15:37:43Z

pkgs/abs/default.nix

+        localSystem = elaborateSomewhat (if isCross then parameters.buildPlatform else parameters.hostPlatform);
+        ${if isCross then "crossSystem" else null} = elaborateSomewhat parameters.hostPlatform;


We already elaborated-somewhat above, so this could just be:

Suggested change

localSystem = elaborateSomewhat (if isCross then parameters.buildPlatform else parameters.hostPlatform);

${if isCross then "crossSystem" else null} = elaborateSomewhat parameters.hostPlatform;

localSystem = if isCross then buildPlatform else hostPlatform;

${if isCross then "crossSystem" else null} = hostPlatform;

wolfgangwalther · 2025-04-27T15:40:18Z

pkgs/abs/default.nix

+      hostPlatform = elaborateSomewhat parameters.hostPlatform;
+      buildPlatform = elaborateSomewhat parameters.buildPlatform;
+
+      isCross = parameters?buildPlatform && ! lib.systems.equals hostPlatform buildPlatform;


To properly call equals, we'd need fully elaborated systems, I think. That's because there could be redundant data passed to one system, that will elaborate to the same platform, but is not exactly the same attrset.

For example:

{ system = "x86_64-linux"; } { config = "x86_64-unknown-linux-gnu"; }

Those two should be equal, but equals as-is would return false.

However, do we even need to do all the isCross logic? The old interface needed it, because system would default to set localSystem. This defaults to set hostPlatform, though.

Could we reduce isCross to isCross = parameters ? buildPlatform? I'm not sure what the implications for the NixOS nixpkgs modules would be, though.

Note, that top-level/default.nix does "equals dance" as well (but with elaborated systems), so this might not make any practical difference, if we leave it out here.

wolfgangwalther · 2025-04-27T16:10:12Z

pkgs/abs/default.nix

+    - `stdenv`: The standard build environment
+    - `testers`: Functions that produce derivations that fail to build if something is wrong.
+  */
+  # TODO: review copied `crossOverlays` doc: isn't it supposed to be "host packages only"? We don't even have "target" in this context.


That's probably right, yes.

I think we should:

Call them hostOverlays.

Pass them to overlays if isCross is false.

Pass them to crossOverlays if isCross is true.

That would match the new hostPlatform-centric interface, right?

That seems exactly right!

roberth · 2025-04-28T06:29:13Z

@wolfgangwalther thank you for your excellent review.

I quite like your idea to provide an attribute-based interface for the Nixpkgs entrypoint. I think it qualifies as a "fluent interface", which is generally not something we should do in the Nix language, but this is an exception.

Nix does not have classes that are compiled a way, nor even vtables that efficiently store sets of "methods", so this pattern must not be used in places where performance matters, such as lib functions or other frequently evaluated things like package builders.
It's easy to do too much evaluation work after each step of the fluent builder, especially if it's not a dedicated builder. For example, adding a fluent builder to a package attrset (type = "derivation";) would be very costly.
They're not even as nice syntactically as they are in dot-method-parentheses languages like Rust or Java. Basically it adds a soft requirement that all possible values can be enumerated into sets of attribute names.

In this case, the we have overriding concerns for each of these, so I think we should use the pattern, while adding a note in the implementation and manual that this style of interface should only be used for CLI-oriented wrapper interfaces (or "facades").

So I think we could have this in both a new file, as well as in the flake, and that's even if flakes will have a better configurability feature at some point.

roberth added 2 commits April 27, 2025 15:53

Add <flake>.lib.nixpkgs.configure

3930c9f

A clean entrypoint into Nixpkgs not too burdened by 20 years of history.

/default.nix: Add entrypoint library

0e1ef54

github-actions bot added the 6.topic: flakes The experimental Nix feature label Apr 27, 2025

roberth requested review from arianvp and infinisil April 27, 2025 13:58

nix-owners bot requested a review from Ericson2314 April 27, 2025 13:59

roberth mentioned this pull request Apr 27, 2025

Add nixpkgsFlake.lib.lib #398332

Draft

13 tasks

github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Apr 27, 2025

roberth mentioned this pull request Apr 27, 2025

systemd: Add functions.escapeSystemdExecArg[s] #401680

Open

13 tasks

wolfgangwalther self-requested a review April 27, 2025 14:43

wolfgangwalther reviewed Apr 27, 2025

View reviewed changes

wolfgangwalther mentioned this pull request Apr 29, 2025

pkgs/top-level/stage.nix: move most nixpkgs sets to variants #400351

Merged

13 tasks

wolfgangwalther mentioned this pull request Jun 22, 2025

modules.nixosTest.setPackage: init #299542

Draft

13 tasks

nixpkgs-ci bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Oct 25, 2025

	Other functions and values that relate to packages without referring to instantiated packages can be added here too.
	Other functions and values that relate to packages without referring to instantiated packages can be added here, too.

		localSystem = elaborateSomewhat (if isCross then parameters.buildPlatform else parameters.hostPlatform);
		${if isCross then "crossSystem" else null} = elaborateSomewhat parameters.hostPlatform;

Uh oh!

Conversation

roberth commented Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

wolfgangwalther commented Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

roberth commented Apr 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

roberth commented Apr 27, 2025 •

edited

Loading

wolfgangwalther commented Apr 27, 2025 •

edited

Loading