Skip to content

Added Jupyter-related Python Packages#1439

Closed
jppelteret wants to merge 4 commits intospack:developfrom
jppelteret:features/numerous_py_packages
Closed

Added Jupyter-related Python Packages#1439
jppelteret wants to merge 4 commits intospack:developfrom
jppelteret:features/numerous_py_packages

Conversation

@jppelteret
Copy link
Copy Markdown
Contributor

Added and modified python packages that are dependencies for the Jupyter suite (#1297)

extends('python')
depends_on('py-setuptools', type='build')
depends_on('py-nose', type='build')
depends_on('py-setuptools', type='build')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

py-setuptools is already in this package.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @adamjstewart . Keen eye :-) I think that this must be a result of my rebaseing before committing. I'll fix this now.

@adamjstewart
Copy link
Copy Markdown
Member

For new Python packages, can you use setup_py( instead of python('setup.py',? See #950, #1424, and #1435 for details.

@citibeth
Copy link
Copy Markdown
Member

citibeth commented Aug 15, 2016

Should py-backports-abc (and other backport modules in this PR) have a guard to prevent it from being installed on newer Python versions that don't need it?

@citibeth
Copy link
Copy Markdown
Member

OK, more comments on the PR:

  1. Why was the following added to py-scipy?
+    depends_on('blas')
+    depends_on('lapack')
  1. Is there a way to eliminate the following code from py-scipy?
         if 'atlas' in spec:
             # libatlas.so actually isn't always installed, but this
             # seems to make the build autodetect things correctly.
-            env['ATLAS'] = join_path(spec['atlas'].prefix.lib, 'libatlas.' + dso_suffix)
+            env['ATLAS'] = join_path(spec['atlas'].prefix.lib,
+                                     'libatlas.' + dso_suffix)
         else:

Isn't this what setup_dependent_package() is for? See the following in openblas:

    def setup_dependent_package(self, module, dspec):
        # This is WIP for a prototype interface for virtual packages.
        # We can update this as more builds start depending on BLAS/LAPACK.
        libdir = find_library_path('libopenblas.a',
                                   self.prefix.lib64,
                                   self.prefix.lib)

        self.spec.blas_static_lib   = join_path(libdir, 'libopenblas.a')
        self.spec.lapack_static_lib = self.spec.blas_static_lib

        if '+shared' in self.spec:
            self.spec.blas_shared_lib   = join_path(libdir, 'libopenblas.%s' %
                                                    dso_suffix)
            self.spec.lapack_shared_lib = self.spec.blas_shared_lib
  1. Should we be concerned about Spack auto-installing root certificates taken off of github (py-certifi). This makes me nervous... Why can't we use the root certificates already on our machines? Or have a sysadmin install new ones if we need them?
  2. Why does py-jsomschema depend on py-setuptools, but the other python packages here that also use setup_py() do not?
  3. Of the Python packages added in this PR... are any of them Python2-only or Python3-only? Or should they work with all (common) versions of Python?

@adamjstewart
Copy link
Copy Markdown
Member

Of the Python packages added in this PR... are any of them Python2-only or Python3-only? Or should they work with all (common) versions of Python?

I've had terrible luck enforcing version constraints on Python extensions. The same things that worked for depends_on like @2.6:2.7,3.4:3.5 didn't work for extends. Maybe it's been fixed, idk. But we can at least add comments.

@davydden
Copy link
Copy Markdown
Member

@citibeth :

Why was the following added to py-scipy?

py-scipy does depend on them and use spec['blas'] and spec['lapack'] in install(). I think they were forgotten originally. The whole DAG happened to work as Blas/Lapack got injected from py-numpy.

Is there a way to eliminate the following code from py-scipy?

to be fair, this was not added by the OP author, so let's keep it for a separate pull request for brevity. Although i do agree that it should work without extra tests for Atlas or non-Atlas blas/lapack.

@citibeth
Copy link
Copy Markdown
Member

I've had terrible luck enforcing version constraints on Python extensions. The same things that worked for depends_on like @2.6:2.7,3.4:3.5 didn't work for extends. Maybe it's been fixed, idk. But we can at least add comments.

I have the following in one of my Python extension packages, maybe something similar would work here?

    extends('python', when='+python')
    depends_on('python@3:', when='+python')

Why was the following added to py-scipy?

py-scipy does depend on them and use spec['blas'] and spec['lapack'] in install(). I think they were forgotten originally. The whole DAG happened to work as Blas/Lapack got injected from py-numpy.

@tgamblin what is our policy on this?

@citibeth
Copy link
Copy Markdown
Member

@jppelteret Is there a reason this was closed? It looked like a promising PR...

@citibeth citibeth reopened this Oct 23, 2016
@jppelteret
Copy link
Copy Markdown
Contributor Author

@citibeth Yes -- I've abandoned my attempt to get Jupyter notebook working through spack, so these packages are no longer required. Unfortunately, at some point one needs to get on and do some actual work instead of messing about with installing packages.

@jppelteret
Copy link
Copy Markdown
Contributor Author

@citibeth Let me qualify my last statement (which could be interpreted in a way that I did not intend): What I mean to convey is that I had no reason to fight with getting Jupyter working through spack when a more immediate solution, namely installing it through their recommended method would suffice just fine. I spent quite a bit of time on this and eventually just needed to get on and have a working solution.

@citibeth citibeth changed the title Added and modified numerous python packages Added Jupyter-related Python Packages Oct 25, 2016
@citibeth
Copy link
Copy Markdown
Member

@jppelteret I understand; I use a MacPorts-installed Sphinx to build the Spack documentation on my laptop (shh...). In the meantime, are you able to give a quick summary of the state of the packages in this PR? They look like decent packages, and it seems they should be merged, even if they are not of immediate use to you.

@jppelteret
Copy link
Copy Markdown
Contributor Author

Well, they did all work for me when I last build Jupyter (which, by the way, built fine but I could just never get to run). What I think what remains to be done is the following:

  • Remove the changes to py-scipy, which has a number of changes since this PR was opened
  • Fix the conflict with py-bottleneck
  • Add depends_on('py-setuptools') where necessary
  • Enforce that PyBackportsAbc to be build for python@2 only

@tgamblin
Copy link
Copy Markdown
Member

Adding up-for-grabs label in case anyone wants to pick up on this! Thanks for all the work so far @jppelteret

@jppelteret
Copy link
Copy Markdown
Contributor Author

You're welcome. So can you offer an amicable solution as to how I get this off my list of open pull requests?

@adamjstewart
Copy link
Copy Markdown
Member

@citibeth Would you be willing to take over this PR? Personally, I think if we remove all changes to existing packages and merge the kind-of-working new packages, that would be fine. It will be easier for others to work out the bugs in them once they are merged.

@citibeth
Copy link
Copy Markdown
Member

I will take the branch out of your fork and put it into the main Spack repo. Then I will submit it as a new PR and we will close this one.

@jppelteret
Copy link
Copy Markdown
Contributor Author

Thanks, I'd appreciate that.

@citibeth citibeth closed this Oct 25, 2016
@jppelteret jppelteret deleted the features/numerous_py_packages branch October 25, 2016 16:30
@citibeth
Copy link
Copy Markdown
Member

Would anyone like to volunteer to take this to completion (eventually)?
On Oct 25, 2016 12:26 AM, "Jean-Paul Pelteret" [email protected]
wrote:

Well, they did all work for me when I last build Jupyter (which, by the
way, built fine but I could just never get to run). What I think what
remains to be done is the following:

  • Remove the changes to py-scipy, which has a number of changes since
    this PR was opened
  • Fix the conflict with py-bottleneck
  • Add depends_on('py-setuptools') where necessary
  • Enforce that PyBackportsAbc to be build for python@2 only


You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
#1439 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB1cdy2bgL5BniZ4d0BSDpu72-jcOQmvks5q3YTygaJpZM4JcoAF
.

olupton pushed a commit to olupton/spack that referenced this pull request Feb 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants