Skip to content

splice.nix: make pkgs splicedPackages when required#362499

Closed
Artturin wants to merge 2 commits intoNixOS:masterfrom
Artturin:pkgsisspliced1
Closed

splice.nix: make pkgs splicedPackages when required#362499
Artturin wants to merge 2 commits intoNixOS:masterfrom
Artturin:pkgsisspliced1

Conversation

@Artturin
Copy link
Member

@Artturin Artturin commented Dec 6, 2024

Take 2 of #349316

This will make pkgs used in callPackage, and pkgsCross.X.pkgs have
packages with __spliced.

https://www.github.com/NixOS/nixpkgs/blob/3029741718f4c765fbc5ebf76bea3d6c8ff15fe5/pkgs/development/interpreters/python/passthrufun.nix#L37

https://www.github.com/NixOS/nixpkgs/blob/d2bd9a39dec88eddd5c192abee69939e67f43d12/pkgs/top-level/python-packages.nix#L10720

nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
error:
       … while evaluating the attribute 'aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced'
         at /home/artturin/nixgits/my-nixpkgs/.worktree/1/pkgs/development/python-modules/protobuf/4.nix:119:13:
          118|   passthru = {
          119|     inherit protobuf;
             |             ^
          120|   };

       error: attribute '__spliced' missing
       at «string»:1:1:
            1| pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
             | ^

to

nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
{
  buildBuild = «derivation /nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildHost = «derivation /nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildTarget = «repeated»;
  hostHost = «derivation /nix/store/mszvybzs4zxh43awyrjnybsfcb265n9r-protobuf-aarch64-unknown-linux-gnu-25.4.drv»;
  hostTarget = «repeated»;
}
```<!--
^ Please summarise the changes you have done and explain why they are necessary here ^

For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->

## Things done

<!-- Please check what applies. Note that these are not hard requirements but merely serve as information for reviewers. -->

- Built on platform(s)
  - [ ] x86_64-linux
  - [ ] aarch64-linux
  - [ ] x86_64-darwin
  - [ ] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
  - [ ] `sandbox = relaxed`
  - [ ] `sandbox = true`
- [ ] Tested, as applicable:
  - [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
  - and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
  - or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
  - made sure NixOS tests are [linked](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#linking-nixos-module-tests-to-a-package) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [25.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2505.section.md) (or backporting [24.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2411.section.md) and [25.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2505.section.md) Release notes)
  - [ ] (Package updates) Added a release notes entry if the change is major or breaking
  - [ ] (Module updates) Added a release notes entry if the change is significant
  - [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [ ] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).

<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!

List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->

---

Add a :+1: [reaction] to [pull requests you find important].

[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc

This will make `pkgs` used in `callPackage`, and `pkgsCross.X.pkgs` have
packages with `__spliced`.

https://www.github.com/NixOS/nixpkgs/blob/3029741718f4c765fbc5ebf76bea3d6c8ff15fe5/pkgs/development/interpreters/python/passthrufun.nix#L37

https://www.github.com/NixOS/nixpkgs/blob/d2bd9a39dec88eddd5c192abee69939e67f43d12/pkgs/top-level/python-packages.nix#L10720

```
nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
error:
       … while evaluating the attribute 'aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced'
         at /home/artturin/nixgits/my-nixpkgs/.worktree/1/pkgs/development/python-modules/protobuf/4.nix:119:13:
          118|   passthru = {
          119|     inherit protobuf;
             |             ^
          120|   };

       error: attribute '__spliced' missing
       at «string»:1:1:
            1| pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
             | ^
```

to

```
nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
{
  buildBuild = «derivation /nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildHost = «derivation /nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildTarget = «repeated»;
  hostHost = «derivation /nix/store/mszvybzs4zxh43awyrjnybsfcb265n9r-protobuf-aarch64-unknown-linux-gnu-25.4.drv»;
  hostTarget = «repeated»;
}
```
Now that `pkgs` is `__splicedPackages` on cross we can remove these
variables I set.

Assignments like `patchutils = pkgs.patchutils_0_3_3;` in
`all-packages.nix` cannot be removed because the `pkgs`
in `patchutils = pkgs.patchutils_0_3_3;` in `all-packages.nix` is coming from
this `pkgs:` https://www.github.com/NixOS/nixpkgs/blob/b6e486730fc875ec79a3dea0f1f46eaf9f6ffa9d/pkgs/top-level/all-packages.nix#L9
instead of the `pkgs` in `with pkgs;`
@github-actions github-actions bot added the 6.topic: erlang General-purpose, concurrent, functional high-level programming language label Dec 6, 2024
@Aleksanaa
Copy link
Member

Can we add a __splicedPackages = pkgs alias (with warning?)

@Kamillaova
Copy link
Contributor

Can we add a __splicedPackages = pkgs alias (with warning?)

AFAIK currently __splicedPackages == pkgs for non-cross targets

@Ericson2314
Copy link
Member

How is intentionally referring to the unspliced current stage to avoid infinite recursions done with this?

@Artturin
Copy link
Member Author

Artturin commented Dec 6, 2024

How is intentionally referring to the unspliced current stage to avoid infinite recursions done with this?

pkgsHostTarget

Weird that the ofborg eval in #349316 didn't catch the inf rec but the actions eval did.

@Artturin
Copy link
Member Author

Artturin commented Dec 6, 2024

Ahh, pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced in the OP now fails with inf rec so it should be a recent regression

@DavHau
Copy link
Member

DavHau commented Jan 10, 2025

Selectively patching splicing into things feels like a never ending workaround. Can't we make mkDerivation itself output splices for each package it generates? Then it would be everywhere by default.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 15, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 10, 2025
@Artturin Artturin closed this Jul 23, 2025
@Artturin Artturin deleted the pkgsisspliced1 branch July 23, 2025 15:12
@Artturin Artturin restored the pkgsisspliced1 branch July 23, 2025 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: erlang General-purpose, concurrent, functional high-level programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants