Skip to content

mk-python-derivation: don't expose check args when doCheck = false#327264

Merged
natsukium merged 1 commit intoNixOS:stagingfrom
natsukium:python/rebuild-less-check-args
Sep 15, 2024
Merged

mk-python-derivation: don't expose check args when doCheck = false#327264
natsukium merged 1 commit intoNixOS:stagingfrom
natsukium:python/rebuild-less-check-args

Conversation

@natsukium
Copy link
Member

Description of changes

This change prevents rebuilds if we change check flags when doCheck = false. It's useful, for example, when tests are separated in passthru.tests.

Perhaps this concern will be addressed.
#272177 (comment)

Note: This PR should target staging, but I'm targeting master to reduce rebuilds during the review.

before

get caches from cache.nixos.org (no rebuild)

nix-build -E 'with import <nixpkgs> { }; python312Packages.sudachipy' -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/4530ffc439786922b453390f23db14923abd8a08.tar.gz

rebuild sudachipy even though pytest is not run

nix-build -E 'with import <nixpkgs> { }; python312Packages.sudachipy.overrideAttrs (_: { pytestFlagsArray = [ "-vv" ]; })' -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/4530ffc439786922b453390f23db14923abd8a08.tar.gz

after

rebuild sudachipy with some dependencies

nix-build -E 'with import <nixpkgs> { }; python312Packages.sudachipy' -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d15b1126bfa4fb929ebf97a927d3ae1bc34f27ab.tar.gz

No rebuild even if overridden to add pytestFlagsArray.

nix-build -E 'with import <nixpkgs> { }; python312Packages.sudachipy.overrideAttrs (_: { pytestFlagsArray = [ "-vv" ]; })' -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d15b1126bfa4fb929ebf97a927d3ae1bc34f27ab.tar.gz

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jul 15, 2024
@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Jul 15, 2024
This change prevents rebuilds if we change check flags when
doCheck = false. It's useful, for example, when tests are separated in
`passthru.tests`.
@natsukium natsukium force-pushed the python/rebuild-less-check-args branch from d15b112 to 2e2f188 Compare July 27, 2024 02:27
@natsukium
Copy link
Member Author

I didn't see any regression in my hydra job, so I'm going to rebase and merge this weekend.

@natsukium natsukium changed the base branch from master to staging September 14, 2024 02:32
@natsukium natsukium marked this pull request as ready for review September 14, 2024 02:33
@natsukium natsukium requested a review from pbsds September 14, 2024 02:33
Copy link
Member

@pbsds pbsds left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM and should achieve the intended effect. Good idea!

Comment on lines +349 to +355
# `escapeShellArgs` should be used as well as `disabledTestPaths`,
# but some packages rely on existing raw strings.
disabledTests = attrs.disabledTests;
} // optionalAttrs (attrs ? pytestFlagsArray) {
pytestFlagsArray = attrs.pytestFlagsArray;
} // optionalAttrs (attrs ? unittestFlagsArray) {
unittestFlagsArray = attrs.unittestFlagsArray;
Copy link
Member

@pbsds pbsds Sep 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flags arrays should also in the future switch over to escapeShellArgs (or structured attrs), maybe a comment for those as well would be nice. I myself have done workarounds like [ "-m" "'not flaky'" ]. Not a blocker

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is clear that similar actions are needed for other attributes.

@natsukium natsukium merged commit fff13ab into NixOS:staging Sep 15, 2024
@natsukium natsukium deleted the python/rebuild-less-check-args branch September 15, 2024 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants