Skip to content

treewide: modernize Python package pytest flags and move away from pytestFlagsArray#424782

Merged
mweinelt merged 6 commits intoNixOS:stagingfrom
ShamrockLee:pytestFlags-misc
Jul 15, 2025
Merged

treewide: modernize Python package pytest flags and move away from pytestFlagsArray#424782
mweinelt merged 6 commits intoNixOS:stagingfrom
ShamrockLee:pytestFlags-misc

Conversation

@ShamrockLee
Copy link
Contributor

@ShamrockLee ShamrockLee commented Jul 13, 2025

Use (enabled|disabled)(TestPaths|TestMarks|Tests) and pytestFlags instead of the __structuredAttrs-incompatible pytestFlagsArray.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other contributing documentation in corresponding paths.

Add a 👍 reaction to pull requests you find important.

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 13, 2025
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 13, 2025
@ShamrockLee ShamrockLee force-pushed the pytestFlags-misc branch 3 times, most recently from 74c2c9f to 267de9f Compare July 13, 2025 09:50
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: Enlightenment DE The Enlightenment Desktop Environment 6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab labels Jul 13, 2025
@ShamrockLee ShamrockLee force-pushed the pytestFlags-misc branch 2 times, most recently from c86d497 to 9a2baca Compare July 13, 2025 10:47
@ShamrockLee ShamrockLee marked this pull request as ready for review July 13, 2025 11:16
@ShamrockLee ShamrockLee mentioned this pull request Jul 13, 2025
Merged
12 tasks
wolfgangwalther
wolfgangwalther reviewed Jul 13, 2025
View reviewed changes
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 13, 2025
@ShamrockLee
python3Packages.duckdb: select test paths with enabledTestPaths
ca61fd3
@ShamrockLee
python3Packages.ipydatawidgets: deselect tests with disabledTestPaths…
405b36a 
… and __structuredAttrs = true
@ShamrockLee
python3Packages.scikit-learn: disable test mark expression with __str…
0c1b02b 
…ucturedAttrs = true
@ShamrockLee
home-assistant.tests.components: refactor with enabledTestPaths and d…
08572b9 
…isabledTestPaths instead of pytestFlagsArray
@ShamrockLee
unblob: use disabledTests
7bca6ea 
disabledTests no longer Bash-expand as array elements
and thus no longer swallow parameters between square brackets.
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 14, 2025
wolfgangwalther
wolfgangwalther approved these changes Jul 14, 2025
View reviewed changes
Copy link
Contributor

@wolfgangwalther wolfgangwalther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for taking on this huge effort to migrate pytestFlagsArray.

Diff LGTM.

@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jul 14, 2025
@ShamrockLee
treewide: modernize Python package pytest flags
6248978 
Use `(enabled|disabled)(TestPaths|TestMarks|Tests)` and `pytestFlags`
instead of the `__structuredAttrs`-incompatible `pytestFlagsArray`.
@ShamrockLee
Copy link
Contributor Author

ShamrockLee commented Jul 14, 2025

Oops! I forgot to push the new python3Packages.scikit-image change.

Pushed.

@mweinelt mweinelt merged commit f4614c2 into NixOS:staging Jul 15, 2025
23 of 27 checks passed
@ShamrockLee ShamrockLee mentioned this pull request Jul 16, 2025
Open
4 tasks
@philiptaron
Copy link
Contributor

philiptaron commented Jul 17, 2025

git bisect says that this broke python3Packages.imageio.

@ShamrockLee
Copy link
Contributor Author

ShamrockLee commented Jul 18, 2025

git bisect says that this broke python3Packages.imageio.

Could you share the error message? I guess there might be some typos.

@bcdarwin bcdarwin mentioned this pull request Jul 18, 2025
Merged
13 tasks
@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Jul 18, 2025

git bisect says that this broke python3Packages.imageio.

Could you share the error message? I guess there might be some typos.

It fails with a lot of these:

E               RuntimeError: Cache directory is corrupt. Delete `.test_images` at project root.

Full log: https://logpaste.com/EtJszpL3

@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Jul 18, 2025

This seems to be in a if use_internet block:

        if use_internet:
            with working_directory(checkout_dir):
                result = os.system("git pull")

            if result != 0:
                request.config.cache.set("imageio_test_binaries", None)
>               raise RuntimeError(
                    "Cache directory is corrupt. Delete `.test_images` at project root."
                )
E               RuntimeError: Cache directory is corrupt. Delete `.test_images` at project root.

@wolfgangwalther
Copy link
Contributor

wolfgangwalther commented Jul 18, 2025

It's called like this:

pytest flags: -m pytest -m not\ \(needs_internet\)

Before it was called like this:

pytest flags: -m pytest -m not\ needs_internet

I think the parentheses are breaking this.

@wolfgangwalther wolfgangwalther mentioned this pull request Jul 19, 2025
Closed
8 tasks
@Shawn8901
Copy link
Contributor

Shawn8901 commented Jul 23, 2025

As the imageio package is still broken (due to obvious bad impl by upstream on the code block), what are the plans for that, Is already someone on that topic?
i stumbled over it while test building staging-next for my machines.

@philiptaron
Copy link
Contributor

philiptaron commented Jul 23, 2025 •
edited
Loading

I'm actually doing it now. Couple more hours, I think.

@philiptaron philiptaron mentioned this pull request Jul 23, 2025
Merged
13 tasks
@qbisi qbisi mentioned this pull request Jul 28, 2025
Merged
13 tasks
qbisi added a commit to qbisi/nixpkgs that referenced this pull request Jul 28, 2025
@qbisi
python312Packages.pymatgen: fix disabledTestPaths
d52048e 
fix regression introduced in NixOS#424782
@ShamrockLee ShamrockLee mentioned this pull request Jul 29, 2025
Merged
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mweinelt mweinelt mweinelt approved these changes

@wolfgangwalther wolfgangwalther wolfgangwalther approved these changes

@hsjobeki hsjobeki Awaiting requested review from hsjobeki

@GetPsyched GetPsyched Awaiting requested review from GetPsyched

@fricklerhandwerk fricklerhandwerk Awaiting requested review from fricklerhandwerk

Assignees

No one assigned

Labels

6.topic: Enlightenment DE The Enlightenment Desktop Environment 6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab 6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ShamrockLee @philiptaron @wolfgangwalther @Shawn8901 @mweinelt