Skip to content

tests: run more of them in ci and hydra#428706

Draft
jopejoe1 wants to merge 3 commits intoNixOS:masterfrom
jopejoe1:run-more-tests
Draft

tests: run more of them in ci and hydra#428706
jopejoe1 wants to merge 3 commits intoNixOS:masterfrom
jopejoe1:run-more-tests

Conversation

@jopejoe1
Copy link
Member

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

Copy link
Contributor

@wolfgangwalther wolfgangwalther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eval fails with no helpful error message, at least I can't see any. Might be more helpful when run locally, but not sure.

@jopejoe1
Copy link
Member Author

jopejoe1 commented Aug 8, 2025

The failure is due to the CI running out of memory.

@wolfgangwalther
Copy link
Contributor

Oh, that's interesting. Is this failure random or introduced because eval for this PR now takes much more memory?

In either case, we'd have to adjust the chunkSize for CI.

Related: #431459.

@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Aug 8, 2025

(CI was somehow not triggered correctly on the last push (?))

Edit: Still not running the PR workflow, I have no idea why...

Edit2: The new PR run is pending, because the old one, that ran out of memory has somehow not been cancelled correctly.

@wolfgangwalther
Copy link
Contributor

The old PR run seems to be stuck here: https://github.com/NixOS/nixpkgs/actions/runs/16829073601/job/47672217906

I tried cancelling it, but that didn't change a thing. I guess we'll have to wait until this is killed after... 6 hours (?).

(This reminds me of putting more reasonable time-limits in place for all the CI jobs...)

@wolfgangwalther
Copy link
Contributor

So, I guess the big question is whether we really want to add all these calls to pkgsCross and friends to Eval. This will likely increase Eval time significantly, if it already increased memory usage for it that way.

An alternative would be to explicitly not recurse into these attrsets, but add comments explaining why.

(I do think the change to an attrset instead of a list is a good one in any case, we should keep that)

@wolfgangwalther
Copy link
Contributor

With #432286 merged, a rebase of this should not cause OOM anymore.

pkgs.spike
];

sanity = lib.recurseIntoAttrs (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can actually do this recursion, because:

  • This takes too many resources to Eval, and
  • some of these use "variants" (pkgsLLVM, etc.), which are disabled for CI. The long-term goal is to not allow any variants used within nixpkgs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of these use "variants" (pkgsLLVM, etc.), which are disabled for CI. The long-term goal is to not allow any variants used within nixpkgs.

If that is the case, I'm not sure that we should have those tests in their current state at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was introduced in #243321.

Using them via ofborg still potentially makes sense, right? I'm not sure whether that's possible without recurseIntoAttrs, though. Now I wonder how recurseForDerivations behaves exactly - what happens when I do nix-build -A tests.cross.sanity and the sanity attrset does not have recurseForDerivations set? Would it still go through that attrset, because I specified it explicitly or would this already require the recurse attribute?

@wolfgangwalther
Copy link
Contributor

Seems like it even OOMs with the swap enabled. This is not going to work with all the pkgsCross stuff.

@jopejoe1 jopejoe1 marked this pull request as draft August 17, 2025 21:25
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 15, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants