@spackbot maintainers

@AndrewGaspar @Sinan81 @arjun-raj-kuppala @BarrySmith @cchambreau @drbenmorgan @glennpj @jedbrown @lee218llnl @robqiao @skosukhin @trws @vvolkl @HadrienG2 @balay @boegel @dhuth @estewart08 @haampie @joseeroman @matthiasdiener @scemama @srekolam @vchuravy

Does this mean python2 won't install no-matter-what, or that we'll have to pass extra flags to support an installation?

I have a legacy application I help that refuses to move off of python2 due to various reasons. I've been building that application with spack as it has over 100 dependencies, but it would be a shame if I had to migrate away from spack.

@trws Correct, all changes to packages other than Python are simply a result of the changes to Python. These changes do propagate to other packages in the sense that if you try to install one of these packages that requires Python 2, Spack will prompt you with a warning that the version is deprecated and ask for confirmation before installing.

@rblake-llnl Once this PR is merged, you'll still be able to build with Python, but you'll either need to manually type "Y" to confirm that you want to install with a deprecated version or pass the --yes-to-all flag. After the 0.18.0 release, if we remove those versions of Python 2, Spack will complain that it doesn't know of any Python 2 versions. You should still be able to enter Python 2 as an external package and Spack will build with it, but that assumes that the other packages remain in Spack. I would be fine with un-deprecating any other packages you need if I'm allowed to deprecate and remove Python 2.

@spackbot run pipeline

I've started that pipeline for you!

I've started that pipeline for you!

I'm 100% fine with this for affected packages that I contributed to.

No objections from my side on deprecating 3.x versions of easybuild, but the packages it requires should be deprecated too in the same way (py-easybuild-framework, py-easybuild-easyblocks, py-easybuild-easyconfigs)?

🤔

As of today, Python 3.6 is also deprecated. I went through all packages again and deprecated a few more. Maintainers on these new packages who may want to see this PR: @RemiLacroix-IDRIS @ax3l @skosukhin

I'm fine with removing python 2 support. If necessary, I am happy to use latest Spack releases with python 2 support.

@adamjstewart: So deprecated means that Spack won't let you install those versions at all or will it be possible to force the install?

Regarding the VAPOR package that got me notified, I'm a bit annoyed as it is still being used on our cluster even though it currently requires Python 3.6. VAPOR is still maintained so hopefully they will release new versions that support newer Python versions eventually...

So deprecated means that Spack won't let you install those versions at all or will it be possible to force the install?

See the docs for details. Once a version is marked as deprecated, fetching the package will cause Spack to prompt the user to warn them that this version is deprecated and ask them if they're sure they want to install it. This prompt can be avoided by using spack install --deprecated if you want to automate this. Once the 0.18 release comes out, we'll then completely remove these versions from Spack. Deprecation is just a first step before removal.

Regarding the VAPOR package that got me notified, I'm a bit annoyed as it is still being used on our cluster even though it currently requires Python 3.6. VAPOR is still maintained so hopefully they will release new versions that support newer Python versions eventually...

Actually, according to NCAR/VAPOR#1822, it seems like they might already support Python 3. Can you investigate this?

Actually, according to NCAR/VAPOR#1822, it seems like they might already support Python 3. Can you investigate this?

They do support Python 3.6 but last time I checked VAPOR failed to compile with Python 3.7+. I will try again.

But anyway there is still a bit of time before VAPOR gets actually removed from Spack.

Is this going to affect bootstrapping from source on distros (like RHEL8) where the system python is 3.6? Not sure why that CI test was skipped here.

@alalazo would know better than me, but I'm guessing that it won't be an issue. Users should still be able to use externally installed Python 3.6 (or even Python 2) and build packages with it, although we may remove Python 2-only dependencies from packages someday (especially backports that are deprecated here).

@tgamblin The rhel8 test is generally skipped in PRs that modify only package recipes. The rationale is that changing a recipe shouldn't affect which Python interpreter can be used to run Spack.

If it makes things easier to review/merge I can remove all changes to packages other than Python. Just wanted people to get an idea of which packages would be affected by this deprecation and get a list of people to ping for review.

Sorry I am not a big noise here but this is quite a bad problem in my eyes. Python2 may be dead in some areas but unfortunately it is still very much in use in some areas. Just as much software depended on fortran 77 for many years there is plenty of software that hasn't been updated and won't be for some considerable time (mostly because there isn't money, this doesn't mean the software isn't useful. Grants aren't always easy to get especially if you just want to update to a new version of python that isn't sexy as far as funders are concerned...).

Also we have a duty as a scientific community to be able to reproduce our results. In the UK the timescales for this are at least 10 years (this is how long scientific projects are required to maintain their data and still be able to analyse it to prove reproducibility and accuracy).

Now I could understand saying that all versions except 2.7.18 are deprecated as almost all 2.x software will run on 2.7.18 and this reduces the testing space by a large area.

As a positive example ARIA a program used to calculate many NMR Structures in the PDB is still a 2.7 code base... Xplor also very much used in the NMR structural community has only just gone python3. I am a CoI on the ccpn project (though I speak in a personal position here) https://ccpn.ac.uk they have a new python3 software but still have python2 legacy software.

Also if spack were a fully mature piece of software this would be ok, but I am just having to move from 0.16 to 0.17 to get around bugs in new packages I am writing. what happens when the fix is in 0.19, I presume I just give up.

I hope I am not sounding disruptive just worried, I have spent a lot of time investing in learning how to use spack because of its flexibility and because it didn't kill my installs like home-brew by removing version will nilly. I can still compile programs with really ancient versions of gcc with spack if need be why should I not be able to work with old version of python?

For anyone who absolutely needs Python 2 or 3.5-3.6 support, I'm willing to keep it, but I would ask that you add yourself as a maintainer for the Python package so that you can help review PRs like #27798.

Offer still stands 😃

From my perspective, I understand your concerns, and if this was as simple as keeping the version('2.7.18', ...) line in the package, I would agree. However, a very large portion of our build recipe for Python simply deals with differences between different versions of Python. With PRs like #27798 and #28346, the changes in these PRs could easily break Python 2 installations. As a volunteer myself, I don't have the bandwidth to test EOL versions of Python. In #27798, we port our build system to use pip to support newer packages that don't come with a setup.py file. But pip is an entire package manager, and is really overkill for this. In the future, I would like to switch to using build and installer. However, the develop branch of installer requires Python 3.7+.

I think the biggest problem with Python is that much like Spack is still a work in progress, so is Python's build system. The "best practice" instructions for writing and building Python packages have changed so much in the last 2 years that it's been hard for me to keep up with. If we want to keep up with these changes, it will become increasingly difficult if we also need to support EOL versions of Python.

Spack is one of the only package managers in existence that I know of that lets you install something that isn't the latest version of a package. Homebrew and others may have separate packages for major version changes (python2 and python3) but none of them let you pick which version of Python 2 or 3 you get. Spack is trying to solve a much more difficult problem (how do we install every version of a package). Without deprecating old EOL software, this problem becomes intractable.

Spack is one of the only package managers in existence that I know of that lets you install something that isn't the latest version of a package. Homebrew and others may have separate packages for major version changes (python2 and python3) but none of them let you pick which version of Python 2 or 3 you get. Spack is trying to solve a much more difficult problem (how do we install every version of a package). Without deprecating old EOL software, this problem becomes intractable.

Just to note, conda, pip and most other Python and language packaging tools (including Homebrew and similar) allow installing any released version of a package just fine, and it is in fact a design requirement of PyPI, Conda-Forge, etc. that once a version is released, it must always remain available barring exceptional circumstances (illegal, malware, etc). In fact, doing so is important for reproducibility, to get the same runtime environment as the code ran in originally, particularly for applications (and by extension, data analysis scripts). AFAIK, its only OS (Linux distro, etc) package managers that tend not to allow this. Of course, this may not be a goal of Spack, which is perfectly fine (and I can very much sympathize with the maintenance burden of still supporting an obsolete Python version, having dropped it in all our packages long ago); just wanted to make sure this was clear.

If there's still a substantial number of users trapped on Python 2 due to the unfortunate reality of grant funding priorities, you could consider making 0.17 a LTS branch maintained by a separate team that accepts bug and security fixes on a best-effort basis. That's more or less what distros like Debian do.

For anyone following this discussion, we decided to revert this PR. Thanks to @scheibelp and @varioustoxins for volunteering to help maintain Python 2 support in Spack! I definitely still encourage users to request that packages that still require Python 2 be ported to Python 3, but many packages are no longer maintained yet are still needed for reproducing experiments. If anyone has time to volunteer, tools like 2to3 make it relatively easy to port software libraries from Python 2 to 3.