Skip to content

Python Updates 2022-09-15#191333

Merged
mweinelt merged 401 commits intostaging-nextfrom
python-updates
Sep 19, 2022
Merged

Python Updates 2022-09-15#191333
mweinelt merged 401 commits intostaging-nextfrom
python-updates

Conversation

@mweinelt
Copy link
Member

@mweinelt mweinelt commented Sep 15, 2022 •
edited
Loading

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Sep 15, 2022
@mweinelt mweinelt changed the title Python updates 2022-09-15 Python Updates 2022-09-15 Sep 15, 2022
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Sep 15, 2022
@FRidh
Copy link
Member

FRidh commented Sep 15, 2022

Fixing scipy using #180243

@mweinelt
Copy link
Member Author

mweinelt commented Sep 15, 2022

Ah cool, you migrated the build to meson. 👍

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 15, 2022
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 15, 2022
@SuperSandro2000 SuperSandro2000 mentioned this pull request Sep 15, 2022
Merged
@ofborg ofborg bot added the 8.has: clean-up This PR removes packages or removes other cruft label Sep 16, 2022
@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 16, 2022
@SuperSandro2000 SuperSandro2000 mentioned this pull request Sep 16, 2022
Closed
13 tasks
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 16, 2022
@mweinelt mweinelt force-pushed the python-updates branch 4 times, most recently from e9a0cf5 to 013fdf4 Compare September 17, 2022 12:55
@SuperSandro2000 SuperSandro2000 force-pushed the python-updates branch 2 times, most recently from 10a8d08 to 53a9a6a Compare September 17, 2022 18:57
@ofborg ofborg bot added ofborg-internal-error Ofborg encountered an error 2.status: merge conflict This PR has merge conflicts with the target branch labels Sep 18, 2022
@mweinelt mweinelt merged commit f1dcc23 into staging-next Sep 19, 2022
@mweinelt
Copy link
Member Author

mweinelt commented Sep 19, 2022

Too tired to do more fixes for now.

@cole-h cole-h removed the ofborg-internal-error Ofborg encountered an error label Sep 19, 2022
@trofi
Copy link
Contributor

trofi commented Sep 20, 2022

Did not verify the bisection, but suspect that 246216e python3Packages.afdko: 3.9.0 -> 3.9.1 broken the build in staging and staging-next of python3Packages.afdko:

$ nix build --no-link -f. python310Packages.afdko -L
...
python3.10-afdko> applying patch /nix/store/00r3wz6naq0nmrg7jnf2bgsppkf1ia6f-use-dynamic-system-antlr4-runtime.patch
python3.10-afdko> patching file CMakeLists.txt
python3.10-afdko> Hunk #1 succeeded at 36 with fuzz 2.
python3.10-afdko> patching file c/makeotf/lib/cffread/CMakeLists.txt
python3.10-afdko> patching file c/makeotf/lib/hotconv/CMakeLists.txt
python3.10-afdko> Hunk #1 succeeded at 69 with fuzz 2.
python3.10-afdko> applying patch /nix/store/sickncxw0s730j6gfrnlsi5ndgysi6la-libxml2-cmake-find-package.patch
python3.10-afdko> patching file CMakeLists.txt
python3.10-afdko> Hunk #1 FAILED at 42.
python3.10-afdko> 1 out of 1 hunk FAILED -- saving rejects to file CMakeLists.txt.rej

@trofi trofi mentioned this pull request Sep 24, 2022
Closed
13 tasks
This was referenced Sep 29, 2022
Closed
Closed
Closed
@mweinelt mweinelt mentioned this pull request Sep 30, 2022
Closed
@linsui linsui mentioned this pull request Oct 2, 2022
Merged
13 tasks
@cpcloud
Copy link
Contributor

cpcloud commented Oct 3, 2022

I think f2a71e9 may have broken use of the top-level pkgs.poetry package, since its version of poetry is sourced from a vendored poetry2nix, which is still on version 1.1.14:

This seems to have made its way into nixos-unstable

…/nixpkgs (59d2991)
❯ nix build -f . poetry
error: builder for '/nix/store/0fa9kcmxq5bz1dmg8xpni1anrbjg5mpq-python3.10-six-1.16.0.drv' failed with exit code 2;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
       >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
       > ModuleNotFoundError: No module named 'setuptools'
       >
       >
       For full logs, run 'nix log /nix/store/0fa9kcmxq5bz1dmg8xpni1anrbjg5mpq-python3.10-six-1.16.0.drv'.
error: 1 dependencies of derivation '/nix/store/mfb3vwlnxpwvsmg1r96qp6i9n2ahwkzc-python3.10-html5lib-1.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/bvzl52l1maa7yw70cwql3kxnhzf6id57-python3.10-poetry-1.1.14.drv' failed to build

@mweinelt
Copy link
Member Author

mweinelt commented Oct 3, 2022 •
edited
Loading

poetry and python3Packages.poetry have no relationship. The former uses poetry2nix and that needs to be updated to pass setuptools where appropriate. This is due to 8895944, which prevents accidentally passing setuptools into the python builder.

You can try to use python3Packages.poetry for the time being.

@cpcloud
Copy link
Contributor

cpcloud commented Oct 3, 2022

poetry and python3Packages.poetry have no relationship.=

I am aware :)

Thanks for the clarification!

@smaret
Copy link
Member

smaret commented Oct 6, 2022

I've just realized that this update broke one of the package I maintain (which is fixed in #194642). Would it be possible to notify the maintainers of all the packages that are updated during these mass updates? This would give us a chance to fix things before they land on master.

If it's not possible for technical reasons to ping all maintainers, perhaps it would make sense to have a python-maintainer team on GH?

@mweinelt
Copy link
Member Author

mweinelt commented Oct 6, 2022

Would it be possible to notify the maintainers of all the packages that are updated during these mass updates?

Sadly not. The python package set has over 5000 packages and GitHub allows us to do 15 review requests.

If it's not possible for technical reasons to ping all maintainers, perhaps it would make sense to have a python-maintainer team on GH?

That's a good idea. But we'd need some tooling that aggregates maintainers in python3Packages, resolves their github username, and updates the team list regularly.

@mweinelt
Copy link
Member Author

mweinelt commented Oct 6, 2022

We have roughly 670 python package maintainers, this is going to be rough and needs some discussion.

@bdd bdd mentioned this pull request Oct 29, 2022
Merged
13 tasks
@tjni tjni mentioned this pull request Nov 7, 2022
Merged
13 tasks
This was referenced Jan 22, 2023
Closed
Closed
Closed
@JamieMagee JamieMagee mentioned this pull request Apr 20, 2023
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 6.topic: TeX Issues regarding texlive and TeX in general 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@mweinelt @FRidh @kirillrdy @samuela @trofi @cpcloud @smaret @vcunat @cole-h @SuperSandro2000