Repository navigation
Python 3.10 manylinux wheels #19630
Description
Hello.
Could you please release CPython 3.10 manylinux wheels on PyPI?
Python 3.10.0 release candidate 1 was released this week: https://www.python.org/downloads/release/python-3100rc1/
It should be quite safe to release wheels for release candidate. ABI for Python 3.10 should be frozen from now on, even for bugfixes. From release notes:
There will be no ABI changes from this point forward in the 3.10 series and the goal is that there will be as few code changes as possible.
Thank You.
Reacted by Miro Hrončok and 0ri0n
Activity
tylerjereddy commentedon Aug 10, 2021
Matching issue in SciPy is cross-linked above--I'll keep an eye on what Chuck/the NumPy team decides to do.
charris commentedon Aug 10, 2021
To build wheels for 3.10 currently requires that it be available in manylinux2014 (linux), and azure (windows and mac). It was about three weeks after 3.9 was released before we could build wheels for all the platforms.
charris commentedon Aug 10, 2021
Note that rc1 did break a couple of NumPy tests.
charris commentedon Aug 10, 2021
Checking manylinux2014, likely the first out the door, they currently support 3.10.0b2. I imagine that will be updated in the next few weeks, maybe when rc2 comes out, if it comes out.
charris commentedon Aug 10, 2021
Hmm, it does look like manylinux2014 does support rc1.
There was no PR for that, it must have just been pushed. I think we can release linux wheels for that.
charris commentedon Aug 10, 2021
I can build manylinux2014 Python3.10.0rc1 wheels, but they can't be tested until 3.10 is added to the testing docker images. @matthew-brett @mattip Who does that?
mattip commentedon Aug 10, 2021
Me. The repo is https://github.com/multi-build/docker-images, I will update it.
mattip commentedon Aug 10, 2021
I updated the docker images. The version available via
apt getafter adding the deadsnake apt repo isI hope that is recent enough? Note these are xenial images, but xenial is EOL. I guess I should update the whole thing to focal (20.04)
hrnciar commentedon Aug 10, 2021
This seems quite outdated, the latest released version is
Python 3.10.0rc1.mattip commentedon Aug 10, 2021
The 3.10rc1 is available in the focal images. But now a new problem: ubuntu has dropped 32-bit i686. How much longer will we ship wheels for that platform?
hroncok commentedon Aug 10, 2021
Honestly, I would not bother shipping 32-bit i686 wheels for recent Python versions. Ubuntu has dropped it, so did Fedora...
charris commentedon Aug 10, 2021
@mattip, I'm trying to recall why we use different images for building and testing, something to do with versions?
charris commentedon Aug 10, 2021
New error:
undefined symbol: _PyGen_SendEDIT: Needs a more recent cython .
charris commentedon Aug 10, 2021
@mattip The 64 bit wheels on Intel now work, but we need
multibuild/focal_arm64v8for ARM64.charris commentedon Aug 10, 2021
The Python 3.10 wheels for Intel 64 bit linux should be available from the nightly builds in an hour or so. You can install them like so:
19 remaining items
encukou commentedon Aug 19, 2021
What other time period would you suggest to use? Your arguments could just as well apply to two years after the freeze.
rgommers commentedon Aug 19, 2021
Eh no, they don't. Most important Python projects are using a 3-9 month release cycle, with the typical spacing for minor releases being 6 months.
@encukou I think you're missing the main point. I do not expect CPython to suddenly become ABI stable between minor releases, and a 2 month window is already a lot nicer than nothing. The problem is the expectation that it's okay to have tons of users upgrade to a new release and expect it to all work fine. What Fedora, Homebrew and the Windows App Store are doing is the main problem here, especially coupled with the expectation that it's a good idea to use all those Python install methods together with PyPI.
encukou commentedon Aug 19, 2021
Ah, so the issue is in upgrading? Yes, I guess all Python distributors should either preserve the old version of Python on upgrade, or not make the upgrade automatic. (edit: I filed a Fedora bug)
I thought we're talking more about new installs of Python – if we agree that new users should get 3.10 by default when 3.10 comes out, then we do need wheels for 3.10 soon.
Important projects are also tested with alphas/betas. (Not necessarily by the upstream maintainers.) For rc1 they can either "just" build new wheels, or (in hopefully rare cases) make a point release with a specific fix.
Not suddenly, but there is a limited API subset that is ABI stable, and there's a bunch of effort ongoing to make it more useful. I don't expect NumPy to use it any time soon – as it's be slower than the less stable API – but for some projects the performance/maintenance cost tradeoff should be worth it.
rgommers commentedon Aug 19, 2021
We are talking about both upgrades and new installs. New users should not get 3.10 by default on day 1. Unless they understand the tradeoffs and make a conscious choice that they want to be early adopters. But for something like the Windows App Store, that's certainly not the case. The default in Windows App Store should be 3.9.x for some time (1-3 months?) after 3.10 release date.
There just aren't enough new/improved things in Python 3.10 that that outweighs some libraries not being installable at all and beginners wasting a lot of time.
I'm sure you are well aware of how complex it is to build wheels in general, and that's why you're using quotes here. It takes a lot of work - we have to move the NumPy wheel build machinery from Azure DevOps to GitHub Actions it looks like for example. Projects more complex than NumPy may have other blockers: maintainer bandwidth, QA trajectories for any official binaries, size limit issues on PyPI that take weeks to get a response, etc.
I suggest to make an inventory after the release of what is missing. My prediction is that despite requests like this issue and people spending time on it, there will be a lot of libraries especially for scientific computing and ML/AI, that will not be available for 3.10 by the release date.
encukou commentedon Aug 19, 2021
A Windows question would be for @brettcannon, I believe?
Can/should we determine who new users are and separate them from old users who are better early adopters? (And should we rather use "early adopter" for users of things like Fedora Rawhide prereleases, which had 3.10 since the alphas?)
AFAICS, the current schedule is the RC period (2 months) for the ecosystem to prepare & test, and the final release is for all users. I don't think we can improve that for 3.10. Should we add another release date for App-Store-like releases for 3.11?
Yes, exactly.
But there are also projects for which this is easier (or zero-effort if they're pure-python), but they're blocked by the more heavy-weight foundational projects. As I see it, NumPy is in a small category of hard+urgent packages.
Definiely.
The 3.9 wheels issue was based on a user report; this one was filed proactively based on the "inventory" for 3.9.
If you have any additions to the "inventory", let me know :)
brettcannon commentedon Aug 23, 2021
Nope, a Windows question is a @zooba question.
zooba commentedon Aug 26, 2021
Each Python version in the Windows Store is a separate app (much as I dislike that model, unfortunately the CPython release model forces it). People who want 3.10 have to explicitly look it up and install it, otherwise they are stuck on 3.n forever.
Now, the default
python.exestub that ships with Windows gets updated on its own schedule. Right now, that's basically when I feel comfortable with ecosystem status (my current smoke test is "can Ipip install jupyter ...and render a chart in a notebook"). If the ecosystem is lagging, it won't be switched - certainly doesn't happen automatically on the first day. However, it does happen instantly when I decide to do it, because Microsoft controls the version it goes to on the server side.There will be more downloads on day 0 from python.org than through the store stub ;)
encukou commentedon Sep 2, 2021
Thanks! That's essentially the same packages we are looking at. It's nice to have a confirmation :)
rgommers commentedon Sep 2, 2021
That's great to hear, thanks for the explanation @zooba.
Beyond what's in https://bugzilla.redhat.com/show_bug.cgi?id=1989081, I'd suggest (packages with compiled code only; pure Python stuff like Dask and Xarray will work already):
hroncok commentedon Sep 2, 2021
Cython has a pure Python universal wheel available, so the manylinux etc. wheel is not blocking, just faster.
charris commentedon Oct 7, 2021
Because NumPy wheels are still azure based, the main blocker for supporting new Python versions is their availability on azure for OS X and windows.
rgommers commentedon Oct 7, 2021
Did anyone look for other projects who run 3.10 Windows/macOS on Azure? The Windows App Store is supposed to have 3.10 final by now.
Probably we should ensure we're migrated to GHA and cibuildwheel before the Python 3.11.0 release?
hugovk commentedon Oct 7, 2021
Azure Pipelines will have 3.10 next week when new images are released. Tracking issue:
GitHub Actions and Azure Pipelines use the same images, but GHA can also install 3.10 on demand, and AZP cannot. Last year, they suggested upvoting this feature request for it to do so.
(So GHA installs will be slightly faster next week when using pre-cached 3.10 instead of install on-demand.)
hugovk commentedon Oct 7, 2021
Having said that, does Azure Pipelines have
3.10-dev?Some CIs have
-devtracking nightly builds (Travis IIRC), others have it tracking most recent pre-release/release (GHA).In either case, would they be good enough for a release? Or at least to start work setting stuff up? Might find some bugs caused by the sudden introduction of a two-digit minor version number :)
zooba commentedon Oct 8, 2021
These should be RC or later, so if they're not good enough for a release, we'd love to hear why (over on python-dev or https://discuss.python.org, but I'm happy to take if the answer is posted here).
Changes during RC are usually less impactful than between .0 and .1 (and if they're not, everyone will be very thankful we made whatever fix was needed!), so it should be at least as safe as after release.