Skip to content

ParaView and Catalyst use python3.#11485

Merged
chuckatkins merged 6 commits intospack:developfrom
danlipsa:use_python3
Jun 10, 2019
Merged

ParaView and Catalyst use python3.#11485
chuckatkins merged 6 commits intospack:developfrom
danlipsa:use_python3

Conversation

@danlipsa
Copy link
Copy Markdown
Contributor

No description provided.

@danlipsa
Copy link
Copy Markdown
Contributor Author

@chuckatkins Please review.

@adamjstewart
Copy link
Copy Markdown
Member

It would be preferable if we could support both Python 2 and Python 3. Especially since older versions of these packages seem to only support Python 2?

@chuckatkins
Copy link
Copy Markdown

@adamjstewart this is part of paraview's transition path to dropping python2 support entirely. Kitware is essentially asserting that python2 is being deprecated in paraview and vtk and as part of that, the spack package will only be supporting python3 moving forward.

@adamjstewart
Copy link
Copy Markdown
Member

@chuckatkins what I'm trying to say is that if the current version of these packages only support Python 3, that's fine. But what about all the older versions? I think it should look something like:

depends_on('python')
depends_on('python@3:', when='@5.6:')
depends_on('python@:2', when='@:5.5')

@chuckatkins
Copy link
Copy Markdown

@adamjstewart That's fair. @danlipsa it should probably be something like this then:

extends('python', when='+python')
depends_on('python@3:', when='@5.6: +python', type=('build', 'link', 'run'))
depends_on('[email protected]:2.8', when='@:5.5.2 +python', type=('build', 'link', 'run'))

The extends will make the paraview python modules available with python instead of just pvpython while the depends_on will manage the versioning.

@danlipsa
Copy link
Copy Markdown
Contributor Author

@chuckatkins Sounds good.

@danlipsa
Copy link
Copy Markdown
Contributor Author

@chuckatkins @adamjstewart Please review.
Chuck, this has the problems I showed you before:
spack spec paraview+python
gives
paraview requires python version 3:, but spec asked for 2.7.16
spack spec [email protected]+python
works.
Even if the first command also choses 5.6.0 as paraview version.

@danlipsa
Copy link
Copy Markdown
Contributor Author

@chuckatkins Should we merge this? At least [email protected] and paraview@develop work. Right now in the develop branch no version works because they all use python2.

@chuckatkins
Copy link
Copy Markdown

Why switch to the vendored libtiff

@danlipsa
Copy link
Copy Markdown
Contributor Author

@chuckatkins I rebase on develop and that error went away - so I removed the internal libtiff.

@DarylGrunau
Copy link
Copy Markdown
Contributor

@danlipsa @chuckatkins, I know you're not asking me but I would be opposed to merging this PR, as it stands, on the basis that it alienates all PV versions less than 5.6.0 when the spec includes +python. Even if the concretizer is broken, I think there is a compromise I propose that will permit the build of downrev versions (I care about v5.5.2 because v5.6.0 contains a bug that is exposed by our production ASC code). If you would be willing to change your package.py line 52:

depends_on('[email protected]:2.8', when='@:5.5+python', type=('build', 'link', 'run'))

into these two lines:

depends_on('python', when='@5.3:5.5+python', type=('build', 'link', 'run'))
depends_on('[email protected]:2.8', when='@:5.2+python', type=('build', 'link', 'run'))

it would permit versions between 5.3 and 5.5 inclusive to be built with any python version (v3 will be required if +osmesa is in the spec). It is well-known that PV does not support python3 before 5.3 so, IMHO, this is the natural break. I am perfectly fine with requiring python3 for versions of [email protected] onward. I beg you, please don't deprecate older versions of PV by pushing this PR through unmodified.

@chuckatkins
Copy link
Copy Markdown

@DarylGrunau I believe #11553 will fix some of these issues so that @5.6: +python+osmesa will build with current meson-based mesa and python 3 and @:5.5 +python+osmesa will build will python 2 and the previous autotools version of mesa. This is currently on hold until that gets resolved.

@danlipsa
Copy link
Copy Markdown
Contributor Author

danlipsa commented Jun 3, 2019

@chuckatkins Not sure what's wrong with travis - this works on my machine. Any ideas?
I could successfully build paraview 5.6 (with python3) and paraview 5.5.2 (with python2)
@DarylGrunau Can you try this?

@danlipsa
Copy link
Copy Markdown
Contributor Author

danlipsa commented Jun 6, 2019

@chuckatkins Should we merge this?

@jrood-nrel
Copy link
Copy Markdown
Member

Thanks for all the work on this!

Workaround for
adding the following to your packages.yaml
packages:
  python:
    version: [3, 2]
without this you'll get:
paraview requires python version 3:, but spec asked for 2.7.16
for `spack spec paraview+python`
see spack pull request spack#11539
@danlipsa
Copy link
Copy Markdown
Contributor Author

danlipsa commented Jun 7, 2019

@chuckatkins I pushed the workaround. Is this ready?

@chuckatkins
Copy link
Copy Markdown

@danlipsa paraview+python works but paraview+python+osmesa still fails (even though the resulting spec is the same 🙁). I tried creating a separate variant for python and python3 and that seems to work in all cases: chuckatkins@619c8f2

Copy link
Copy Markdown

@chuckatkins chuckatkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to add:

if '+python' in spec or '+python3' in spec:

In both packages configure args then I think we should should be good to go finally.

@danlipsa
Copy link
Copy Markdown
Contributor Author

Needs to add:

if '+python' in spec or '+python3' in spec:

In both packages configure args then I think we should should be good to go finally.

Good catch. I'll do that.

@chuckatkins
Copy link
Copy Markdown

@danlipsa looks like travis wasn't reporting. See https://travis-ci.org/spack/spack/jobs/543763846 to fix the flake 8 error. Then I'll merge this once it passes.

@chuckatkins chuckatkins removed the WIP label Jun 10, 2019
@danlipsa
Copy link
Copy Markdown
Contributor Author

@chuckatkins What about now?

@chuckatkins chuckatkins merged commit c98190a into spack:develop Jun 10, 2019
@jrood-nrel
Copy link
Copy Markdown
Member

Rather than open an issue for this, I just realized that '-DVTK_USE_SYSTEM_MPI4PY:BOOL=%s' % variant_bool('+python+mpi'), needs the python3 variant to be able to work as well. For example, I can't install with python3 because the builtin Paraview 5.6.0 MPI4PY is broken.

@chuckatkins
Copy link
Copy Markdown

So close... So close...

@danlipsa
Copy link
Copy Markdown
Contributor Author

@jrood-nrel Thanks for the report. I'll push a fix shortly.

@danlipsa
Copy link
Copy Markdown
Contributor Author

danlipsa commented Jun 14, 2019

See #11707 for a fix.

carsonwoods pushed a commit to carsonwoods/spack that referenced this pull request Jun 27, 2019
Use python3 for latest paraview and catalyst versions.
dev-zero pushed a commit to dev-zero/spack that referenced this pull request Aug 13, 2019
Use python3 for latest paraview and catalyst versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants