[python-updates] 2023/12/02 major updates (python312Packages, pythonRuntimeDepsCheckHook) by mweinelt · Pull Request #271586 · NixOS/nixpkgs

mweinelt · 2023-12-02T03:27:33Z

This is a bulk update of python packages, that includes major version bumps.

TODO

pythonRuntimDepsCheckHook (see python3.pkgs.pythonRuntimeDepsCheckHook: init #270457)
Recurse into python312Packages, stop recursing into python310Packages
Prune SETUPTOOLS_SCM_PRETEND_VERSION
Review and merge staging changes on the project board
Re-review changes and consider them for merge (ETA: 2023-12-02, 12:00+ UTC)
Run bulk updater (ETA: 2023-12-02, 21:00+ UTC)
Start evaluation

Once the evaluation has finished, jobs will start building and breakage can be reviewed over at https://hydra.nixos.org/jobset/nixpkgs/python-updates.

Additionally, a review report will appear at https://malob.github.io/nix-review-tools-reports.

Known Issues:

meson-python on python312 depends on meson: 1.2.3 -> 1.3.0 #268583
numpy 1.26.2 still uses distutils in its buildsystem, needs research which commits we need to backport
pytest-asyncio 0.22.0 breaks bleak, aioesphomeapi and a few other packages - reverted!

Latest eval report

https://shells.darmstadt.ccc.de/~hexa/93.html

SuperSandro2000 · 2023-12-02T16:10:33Z

pkgs/development/interpreters/python/hooks/python-runtime-deps-check-hook.py

+    metadata = get_metadata(args.wheel)
+    tests = [test_requirement(requirement) for requirement in metadata.requires_dist]
+
+    if not all(tests):


We shoud list some of the possible options for people here like buildNpmPackage

Please take some time to make your review comments more legible.

I think you are asking for more actionable feedback to the user. I can do that, but buildNpmPackage offers these 3 hints, because the way to resolve things are much more unclear.

The only thing I would recommend is pythonRelaxDepsHook, if version constraints are too tight.

You got exactly what I wanted to say, no need to write romans. :)

We could print something like

Please try to use pythonRelaxDepsHook to release the version constraints if the library allows it and tests continue to pass. If the hook does not work for some reason, please try to fall back to manually patching in postPatch.

You got exactly what I wanted to say, no need to write romans. :)

It took me more than a minute to grok though, and that happens more often than you think.

hmmmm, I try to remember that in the future but that's just my normal writing style. I am not leaving things out on purpose.

pkgs/development/python-modules/setuptools-scm/setup-hook.sh

pkgs/development/python-modules/psycopg/default.nix

mweinelt · 2023-12-04T20:42:06Z

Report for the latest eval on my private hydra, if someone wants to dig in.

https://shells.darmstadt.ccc.de/~hexa/55.html

pkgs/development/python-modules/dbus/default.nix

trofi · 2023-12-25T11:08:44Z

Bisect says 536b7d1 python311Packages.pytest: clean up deps, expose testing extra broke libei in staging-next:

$ nix build --no-link -f. -L libei
...
libei> Program python3 (attr, jinja2) found: NO modules: jinja2
libei> proto/meson.build:17:15: ERROR: python3 is missing modules: attr
libei> A full log can be found at /build/source/build/meson-logs/meson-log.txt

mweinelt · 2023-12-25T11:43:27Z

On mobile right now, but a fix was referenced in the staging-next PR.

imincik · 2024-01-15T17:07:31Z

It looks like sip upgrade to 6.8.0 broke both qgis and qgis-ltr (see: #281100).

mattfawcett · 2024-01-16T20:34:44Z

pythonPackages.pyosmium no longer builds with these upgrades due to it trying to patch something already patched in the latest version. Created #281389 to fix it.

imincik · 2024-01-18T10:18:32Z

@mweinelt , I found this commit merged as a part of this PR updating python3Packages.rasterio to version 4 which doesn't exist. Package with this change doesn't even build ! It looks very suspicious to me !

BTW, I am one of the maintainers of this package and I haven't seen any notification about the change.

jonringer · 2024-01-18T16:32:18Z

@mweinelt , I found this commit merged as a part of this PR updating python3Packages.rasterio to version 4 which doesn't exist. Package with this change doesn't even build ! It looks very suspicious to me !

This should have only been included if they didn't mark that tag as a pre-release. Looks like it was already reverted but the forced update of all packages will sometimes get these errors propagating to us.

jonringer · 2024-01-18T16:34:21Z

although, looked at that branch... and it's from 9 years ago, not sure why it did get included.... the release metadata should be getting fetched from github.

mweinelt · 2024-01-18T16:45:37Z

We arrive at the following using https://api.github.com/repos/rasterio/rasterio/releases and because we determine the highest version in _determine_latest_version.

print(_get_latest_version_github("python311Packages.rasterio", "rasterio", "", "1.3.8", "major"))
('4', '07j1fba1jm55qby0ljxmm983n229qgarjdm9gwyfzdq4cag0bvb0', 'release-test-')

CertainLach · 2024-01-30T11:38:05Z

xformers upgrade is broken here:
5dde5f3

Version 0.03 exists, but this is a typo in upstream repository:

And in fact, xformers version is downgraded here
0.0.22.post7 => 0.03

Version 0.0.22.post7 is much newer:

This was added in NixOS#271586, but never used due to the spelling mistake of a double s.

github-actions bot added 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab labels Dec 2, 2023

mweinelt mentioned this pull request Dec 2, 2023

python3.pkgs.pythonRuntimeDepsCheckHook: init #270457

Closed

13 tasks

SuperSandro2000 approved these changes Dec 2, 2023

View reviewed changes

SuperSandro2000 reviewed Dec 2, 2023

View reviewed changes

pkgs/development/python-modules/psycopg/default.nix Show resolved Hide resolved

mweinelt force-pushed the python-updates branch 2 times, most recently from e5af2a4 to fa00e71 Compare December 2, 2023 23:32

mweinelt mentioned this pull request Dec 3, 2023

python311Packages.zeroconf: 0.119.0 -> 0.120.0 #265608

Closed

13 tasks

mweinelt force-pushed the python-updates branch from fa00e71 to ca33ecd Compare December 3, 2023 02:54

natsukium mentioned this pull request Dec 3, 2023

python311Packages.botocore: 1.31.57 -> 1.32.7 #270402

Merged

13 tasks

mweinelt force-pushed the python-updates branch 3 times, most recently from ecb5694 to b89d6c9 Compare December 3, 2023 14:32

This was referenced Dec 3, 2023

meson: 1.2.3 -> 1.3.0 #268583

Merged

python310Packages.immutabledict: 3.0.0 -> 4.0.0 #271890

Closed

mweinelt force-pushed the python-updates branch 3 times, most recently from c859349 to 0151fcd Compare December 4, 2023 17:29

mweinelt force-pushed the python-updates branch from 5bbba79 to 2f42d7c Compare December 4, 2023 21:06

mweinelt commented Dec 4, 2023

View reviewed changes

pkgs/development/python-modules/dbus/default.nix Outdated Show resolved Hide resolved

mweinelt force-pushed the python-updates branch from 558efe4 to 72d817c Compare December 5, 2023 00:36

ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 5, 2023

This was referenced Dec 26, 2023

python311Packages.connexion: 2.14.2 -> 3.0.1 #262544

Closed

Restore python dependency validation #250865

Closed

natsukium mentioned this pull request Dec 29, 2023

python311Packages.setuptools-scm-git-archive: 1.4 -> 1.4.1 #277392

Closed

13 tasks

fabaff mentioned this pull request Dec 30, 2023

Build failure: aiogram #277097

Closed

yu-re-ka mentioned this pull request Jan 6, 2024

[staging-next] Build failure: krita #279054

Closed

chuangzhu mentioned this pull request Jan 12, 2024

komikku: 1.33.0 -> 1.34.2 #280506

Closed

13 tasks

gador mentioned this pull request Jan 12, 2024

pgadmin: 8.1 -> 8.2, python3Packages.webauthn 1.11.1 -> 2.0.0, python3Packages.flask-security-too: fix pydantic issue #280556

Merged

13 tasks

natsukium mentioned this pull request Jan 14, 2024

python3Packages.cmdstanpy: relax dependency, drop legacy builder #280777

Merged

13 tasks

cbourjau mentioned this pull request Jan 14, 2024

onnx: Fix build #281011

Merged

13 tasks

gepbird mentioned this pull request Jan 15, 2024

python3Packages.pygame: 2.2.0 -> 2.4.0 #237941

Closed

12 tasks

imincik mentioned this pull request Jan 15, 2024

Build failure: qgis build is failing during sip-build #281100

Closed

sephii mentioned this pull request Jan 16, 2024

python3Packages.wagtail: relax django-taggit dependency #275356

Merged

13 tasks

quyse mentioned this pull request Jan 17, 2024

Build failure: trueseeing #281164

Closed

imincik mentioned this pull request Jan 18, 2024

Revert "python3Packages.rasterio: 1.3.9 -> 4" #281747

Merged

13 tasks

sengaya mentioned this pull request Jan 19, 2024

ansible-lint: 6.21.1 -> 6.22.1 #274473

Closed

13 tasks

CertainLach mentioned this pull request Feb 1, 2024

python3Packages.xformers: 0.03 -> 0.0.23.post1, add cuda support #285495

Merged

13 tasks

arianvp mentioned this pull request Feb 8, 2024

Hatch check phase fails on darwin #287216

Closed

GetPsyched mentioned this pull request Feb 8, 2024

Update request: mlxtend 0.22.0 → 0.23.0 #266238

Closed

1 task

wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this pull request Jun 12, 2024

python12Packages.pytest-regressions: remove unused disabledTestPathss

f6aa38f

This was added in NixOS#271586, but never used due to the spelling mistake of a double s.

wolfgangwalther mentioned this pull request Jun 12, 2024

python12Packages.pytest-regressions: remove unused disabledTestPathss #319351

Merged

2 tasks

wolfgangwalther added a commit to wolfgangwalther/nixpkgs that referenced this pull request Jun 12, 2024

python12Packages.pytest-regressions: remove unused disabledTestPathss

61eaf7b

This was added in NixOS#271586, but never used due to the spelling mistake of a double s.

Uh oh!

Comments

Conversation

mweinelt commented Dec 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO

Known Issues:

Latest eval report

Uh oh!

SuperSandro2000 Dec 2, 2023

Choose a reason for hiding this comment

Uh oh!

mweinelt Dec 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Dec 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mweinelt Dec 2, 2023

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Dec 2, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mweinelt commented Dec 4, 2023

Uh oh!

Uh oh!

trofi commented Dec 25, 2023

Uh oh!

mweinelt commented Dec 25, 2023

Uh oh!

imincik commented Jan 15, 2024

Uh oh!

mattfawcett commented Jan 16, 2024

Uh oh!

imincik commented Jan 18, 2024

Uh oh!

jonringer commented Jan 18, 2024

Uh oh!

jonringer commented Jan 18, 2024

Uh oh!

mweinelt commented Jan 18, 2024

Uh oh!

CertainLach commented Jan 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

16 participants

mweinelt commented Dec 2, 2023 •

edited

Loading

mweinelt Dec 2, 2023 •

edited

Loading

SuperSandro2000 Dec 2, 2023 •

edited

Loading