Skip to content

Comments

flake: update nixpkgs: 22.11 -> 23.05#8569

Merged
edolstra merged 8 commits intoNixOS:masterfrom
vcunat:p/flake-update
Sep 20, 2023
Merged

flake: update nixpkgs: 22.11 -> 23.05#8569
edolstra merged 8 commits intoNixOS:masterfrom
vcunat:p/flake-update

Conversation

@vcunat
Copy link
Member

@vcunat vcunat commented Jun 22, 2023

Motivation

Pinned nixpkgs commit over half a year old.

Context

#1254 (comment)

Checklist for maintainers

Maintainers: tick if completed or explain if not relevant

  • agreed on idea
  • agreed on implementation strategy
  • tests, as appropriate
    • functional tests - tests/**.sh
    • unit tests - src/*/tests
    • integration tests - tests/nixos/*
  • documentation in the manual
  • documentation in the internal API docs
  • code and comments are self-explanatory
  • commit message explains why the change was made
  • new feature or incompatible change: updated release notes

Priorities

Add 👍 to pull requests you find important.

@vcunat vcunat changed the title flake: update inputs, nixpkgs: 22.11 -> 23.05 flake: update nixpkgs: 22.11 -> 23.05 Jun 22, 2023
@vcunat
Copy link
Member Author

vcunat commented Jun 22, 2023 •
edited
Loading

nix flake check

  • nixpkgs-lib-tests: nix changed error message which breaks a too sensitive test

@vcunat vcunat mentioned this pull request Jun 22, 2023
Merged
1 task
@vcunat vcunat marked this pull request as ready for review June 23, 2023 08:12
@vcunat vcunat requested a review from edolstra as a code owner June 23, 2023 08:12
@vcunat
Copy link
Member Author

vcunat commented Jun 23, 2023

After the nixpkgs fix, nix build and nix flake check passed for me on x86_64 NixOS.

@roberth roberth mentioned this pull request Jun 26, 2023
Merged
12 tasks
@Ericson2314
Copy link
Member

Ericson2314 commented Jun 27, 2023

What's the status on this? Is the stuff we need in Nixpkgs already?

@nixos-discourse
Copy link

nixos-discourse commented Jun 27, 2023

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/is-nixbsd-a-posibility/29612/17

@domenkozar
Copy link
Member

domenkozar commented Jun 27, 2023

Once CI passes, we should just merge. There's nothing blocking it.

@domenkozar
Copy link
Member

domenkozar commented Jun 27, 2023

I'm not sure why macos is failing on: lib-path-tests> error: getting status of '/nix/store/.links': Operation not permitted

@vcunat
Copy link
Member Author

vcunat commented Jun 27, 2023

Yes, it's in there and I had updated the lockfile and the commands above passed for me. I thought CI also passed but I'm not sure.

@domenkozar
Copy link
Member

domenkozar commented Jun 27, 2023

It seems that linux passes, but macos fails. I don't see the history, but did the same error happen before on linux?

@vcunat
Copy link
Member Author

vcunat commented Jun 27, 2023 •
edited
Loading

All tests including this one seem to pass on nix master, but I have no idea why this PR could make a difference here.

@domenkozar
Copy link
Member

domenkozar commented Jun 27, 2023

I can't reproduce it on aarch64-darwin and I don't have access to an intel machine.

Maybe you can pass -L to nix invocation so that we get the full log?

@vcunat
Copy link
Member Author

vcunat commented Jun 27, 2023 •
edited
Loading

I have access to a native x86_64-darwin, if you tell me what to run/inspect.

@domenkozar
Copy link
Member

domenkozar commented Jun 27, 2023

nix flake check should reproduce it.

@vcunat
Copy link
Member Author

vcunat commented Jun 27, 2023

Uh, I don't know :-/

$ nix flake check -L
trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
trace: warning: The option `services.openssh.permitRootLogin' defined in `/nix/store/gmgk5lx7bsckhwpsba1mni4qnrqhpykk-source/tests/nixos/nix-copy.nix' has been renamed to `services.openssh.settings.PermitRootLogin'.
trace: warning: Module argument `nodes.http_dns.config` is deprecated. Use `nodes.http_dns` instead.
trace: warning: Module argument `nodes.github.config` is deprecated. Use `nodes.github` instead.
trace: warning: Module argument `nodes.sourcehut.config` is deprecated. Use `nodes.sourcehut` instead.
checking Hydra job 'hydraJobs.tests.setuid.i686-linux'Segmentation fault: 11

(repeatedly)

@roberth roberth mentioned this pull request Jun 27, 2023
Closed
@vcunat
Copy link
Member Author

vcunat commented Jun 27, 2023

I tried to run the same with newer nix than was on that machine and it all passed.

@Ericson2314 Ericson2314 mentioned this pull request Jun 29, 2023
Closed
@roberth
Copy link
Member

roberth commented Jun 29, 2023

I'm not sure why macos is failing on: lib-path-tests> error: getting status of '/nix/store/.links': Operation not permitted

Judging from lib-path-tests> this is from nix running inside the sandbox.
lib/tests/release.nix does configure a different store location, but lib/path/tests/default.nix does not.

Probably the hard link optimization wasn't triggered in an older version of nix. As that runs async, it could be quite non-deterministic.

It would be great to have a slightly more functional default nix store when nix runs in the sandbox. We'll need that for effective use of RFC 92 dynamic derivations anyway. For a test like this, we run nix just for instantiation, which is basically the same requirement that we have for 92, and we've established that a very restrictive recursive nix socket that basically only allows addToStore is unlikely to cause a structural problem.

@roberth roberth mentioned this pull request Jun 29, 2023
Open
@vcunat
Copy link
Member Author

vcunat commented Jun 29, 2023

I haven't mentioned that the machine I tried is configured close to defaults, i.e. without sandboxing.

@edolstra
Copy link
Member

edolstra commented Jun 30, 2023 •
edited
Loading

I set up a jobset for this PR: https://hydra.nixos.org/eval/1797165

nixpkgsLibTests works fine on macOS on Hydra, so this seems to be a GHA-specific issue. I would suggest removing nixpkgsLibTests from checks (so it will still run on Hydra). Or maybe only running it on x86_64-linux on GHA.

There are two other problems:

@edolstra edolstra requested a review from thufschmitt as a code owner July 3, 2023 10:21
@github-actions github-actions bot added the tests label Jul 3, 2023
@edolstra
Copy link
Member

edolstra commented Jul 3, 2023

The crashing make check in the coverage job is caused by some interaction between gcov and rapidcheck. Disabling the rapidcheck tests makes the coverage job succeed.

So we may need to decide whether we care more about coverage analysis or property testing...

@roberth
Copy link
Member

roberth commented Jul 3, 2023

Yikes.

These lines in the log seem the describe the cause:

[----------] 2 tests from StorePathTest_RapidCheck
[ RUN      ] StorePathTest_RapidCheck.prop_regex_accept
corrupted size vs. prev_size in fastbins
[       OK ] StorePathTest_RapidCheck.prop_regex_accept (8 ms)
[ RUN      ] StorePathTest_RapidCheck.prop_round_rip
[       OK ] StorePathTest_RapidCheck.prop_round_rip (8 ms)
[----------] 2 tests from StorePathTest_RapidCheck (17 ms total)

corrupted size vs. prev_size in fastbins seems indicative of a memory corruption. I'll try valgrind.

@Ericson2314
Copy link
Member

Ericson2314 commented Aug 25, 2023

We have Hydra results! https://hydra.nixos.org/eval/1798739?compare=1798723&full=0#tabs-now-fail

  • aarch64-darwin is newly broken because AWS SDK failed to build
  • aarch64-linux is fixed
  • coverage is newly broken
  • tests.containers is newly broken

roberth added 2 commits August 30, 2023 19:34
@roberth @Ericson2314
nixpkgsLibTests: Only test our Nix
3384f70 
Interface has changed upstream.
It *should* be fine to test 23.05's other Nix versions as those
*should* succeed, but that's not the case and it's obfuscating
our terrible CI setup's log.
@roberth @Ericson2314
Fix nix-copy test
be3362e
@Ericson2314 Ericson2314 mentioned this pull request Aug 31, 2023
Merged
8 tasks
@edolstra edolstra added this to the nix-2.18 milestone Sep 1, 2023
@vcunat @roberth @edolstra
flake: update nixpkgs: 22.11 -> 23.05
539cc5e 
The lowdown input can't be updated; `nix build` would fail to find it.

Co-authored-by: Robert Hensing <[email protected]>
@roberth
Copy link
Member

roberth commented Sep 1, 2023

Meeting notes 2023-09-01

  • Need to fix master by either fixing+merging the NixOS 23.05 update or reverting the PR that broke aarch64-linux
    • @edolstra will look into issue in auto uid experimental feature + the coverage issue
  • "vanila" (without our overrides) AWS SDK is buildable on release-23.05 with tests.
    • let's try building again with the tests, hopefully the AWS tests pass this time.

@nixos-discourse
Copy link

nixos-discourse commented Sep 1, 2023

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-09-01-nix-team-meeting-minutes-84/32466/1

@edolstra
Copy link
Member

edolstra commented Sep 1, 2023

@roberth I updated this PR to use the latest 23.05, but it's still failing in the aws-sdk-cpp build: https://hydra.nixos.org/build/233688522

@Ericson2314
Copy link
Member

Ericson2314 commented Sep 2, 2023 •
edited
Loading

https://hydra.nixos.org/build/233688522 and https://hydra.nixos.org/build/232704350 look like the same derivation? I think we need to manually restart the failing builds in Hydra, since the 23.05 bump did not include a mass rebuild.

@vcunat
Copy link
Member Author

vcunat commented Sep 2, 2023

That failing build has been retried several times already; you can see some attempts e.g. on https://hydra.nixos.org/build/232704299#tabs-buildsteps

(You point to two different builds of nix but both depend on the same aws-sdk-cpp.)

@roberth
Copy link
Member

roberth commented Sep 3, 2023

I'm having network difficulties with my aarch64-darwin node, so I can't troubleshoot that build right now.

@edolstra
Copy link
Member

edolstra commented Sep 19, 2023

I've disabled the systemd-nspawn test. That looks like a regression/change in systemd in NixOS 23.05 and I have no idea how to fix it.

@Ericson2314
Copy link
Member

Ericson2314 commented Sep 19, 2023

Thanks @edolstra for taking a look at this.

@edolstra
Use "touch -h"
b6b2a0a 
https://hydra.nixos.org/build/235888160

This is needed because Nixpkgs now contains dangling symlinks
(pkgs/test/nixpkgs-check-by-name/tests/symlink-invalid/pkgs/by-name/fo/foo/foo.nix).
@edolstra edolstra marked this pull request as ready for review September 19, 2023 15:59
cole-h
cole-h reviewed Sep 19, 2023
View reviewed changes
flake.nix
Comment on lines +4 to +5
#inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05-small";
inputs.nixpkgs.url = "github:edolstra/nixpkgs/fix-aws-sdk-cpp";
Copy link
Member

@cole-h cole-h Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://nixpk.gs/pr-tracker.html?pr=256102 once this hits small, we can go back.

@edolstra
Copy link
Member

edolstra commented Sep 20, 2023

Merging this since the jobset is now green: https://hydra.nixos.org/eval/1799383

@edolstra edolstra merged commit 6ce358b into NixOS:master Sep 20, 2023
@vcunat vcunat deleted the p/flake-update branch September 20, 2023 09:34
cole-h
cole-h reviewed Sep 20, 2023
View reviewed changes
tests/nixos/containers/containers.nix
# Run systemd-nspawn in a Nix build.
host.succeed("nix build -v --auto-allocate-uids --sandbox -L --offline --impure --file ${./systemd-nspawn.nix} --argstr nixpkgs ${nixpkgs}")
host.succeed("[[ $(cat ./result/msg) = 'Hello World' ]]")
#host.succeed("nix build -v --auto-allocate-uids --sandbox -L --offline --impure --file ${./systemd-nspawn.nix} --argstr nixpkgs ${nixpkgs}")
Copy link
Member

@cole-h cole-h Sep 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I attempted to re-enable this test in #9008.

@Gerg-L Gerg-L mentioned this pull request Sep 22, 2023
Merged
PkmX pushed a commit to PkmX/nixpkgs.lib that referenced this pull request Aug 23, 2024
@roberth
lib/path/tests: Fix test setup on darwin
6c3c783 
These statements are taken from the `lib/test/release.nix` tests,
which previously also worked on darwin.

Unblocks NixOS/nix#8569 when backported
@mergify mergify bot mentioned this pull request Mar 28, 2025
Closed
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@edolstra edolstra Awaiting requested review from edolstra edolstra is a code owner

@thufschmitt thufschmitt Awaiting requested review from thufschmitt

1 more reviewer

@cole-h cole-h cole-h left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@roberth roberth

Labels

tests with-tests Issues related to testing. PRs with tests have some priority

Projects

Nix team
Archived in project

Milestone

nix-2.18

Development

Successfully merging this pull request may close these issues.

8 participants

@vcunat @Ericson2314 @nixos-discourse @domenkozar @roberth @edolstra @Misterio77 @cole-h