Ipython > 6 is python 3 only. Update pacakges and dependencies. by s-sajid-ali · Pull Request #8309 · spack/spack

s-sajid-ali · 2018-05-29T16:47:14Z

	modified:   var/spack/repos/builtin/packages/py-ipython-genutils/package.py
	modified:   var/spack/repos/builtin/packages/py-ipython/package.py
	new file:   var/spack/repos/builtin/packages/py-testpath/package.py
	modified:   var/spack/repos/builtin/packages/py-traitlets/package.py

I don't understand why ipython doesn't work when py-setuptools is specified as a dependency. Leaving it as is until someone understands how to fix it.

One package requires master and has to be installed with -n.

modified: var/spack/repos/builtin/packages/py-ipython-genutils/package.py modified: var/spack/repos/builtin/packages/py-ipython/package.py new file: var/spack/repos/builtin/packages/py-testpath/package.py modified: var/spack/repos/builtin/packages/py-traitlets/package.py

s-sajid-ali · 2018-05-29T16:51:38Z

Also, does spack have a plan for dealing with the flit buildsystem for python packages ?
refer to 1

modified: var/spack/repos/builtin/packages/py-backcall/package.py modified: var/spack/repos/builtin/packages/py-ipython-genutils/package.py modified: var/spack/repos/builtin/packages/py-ipython/package.py modified: var/spack/repos/builtin/packages/py-testpath/package.py modified: var/spack/repos/builtin/packages/py-traitlets/package.py

adamjstewart · 2018-05-29T17:30:29Z

I don't understand why ipython doesn't work when py-setuptools is specified as a dependency.

Because py-setuptools is not a dependency. Spack adds setuptools-specific flags to python setup.py install in order to handle things like eggs. If the package doesn't use setuptools, this will break.

adamjstewart · 2018-05-29T17:31:54Z

var/spack/repos/builtin/packages/py-backcall/package.py

+    """Specifications for callback functions passed in to an API"""
+
+    homepage = "https://github.com/takluyver/backcall"
+    url = "https://files.pythonhosted.org/packages/84/71/c8ca4f5bb1e08401b916c68003acf0a0655df935d74d93bf3f3364b310e0/backcall-0.1.0.tar.gz"


See https://github.com/spack/spack/pull/5015/files#diff-da0d72075ba6486328675e8e03ba2daeR139

s-sajid-ali · 2018-05-29T17:38:34Z

The install_requies section of the current setup.py lists setuptools as a requirement though and I did the same for the spack package.

adamjstewart · 2018-05-29T19:45:45Z

Oh, very interesting. For some reason they use the setup function from distutils instead of from setuptools, which explains why those install flags aren't working. You could override the default install phase to not add those flags even though setuptools is a dependency.

s-sajid-ali · 2018-05-30T19:18:00Z

So it looks like there's a choice for ipython. The setup.py has this comment :

# For some commands, use setuptools.  Note that we do NOT list install here!
# If you want a setuptools-enhanced install, just run 'setupegg.py install'

Should this be left here and not list setuptools as a dependency or should a variant for setuptools also be included ?

citibeth

Do not merge. Add necessary elements to #7926 instead.

citibeth · 2018-06-01T14:23:05Z

This cannot be merged because it breaks Python2. See #7926 for how to create packages that work for Python2 and Python3. HOWEVER... the core problem is that the concretizer cannot properly handle things like:

depends_on('py-backport', when='^python@:2')

@scheibelp we have had discussion about that, and the plan is to fix the concretizer. In the meantime, #7926 provides a "hack" of a +python3 variant you have to set manually, and then the concretizer is able to use that in depends_on() statements. So the above turns into:

depend_on('py-backport', when='~python3')

It has been decided that this stuff will not be merged into develop; but instead (for now), Python3 users will merge #7926. See #8018. If you wish this decision to be revisited, you need to petition core Spack maintainers such as @scheibelp. In any case... the good thing about #7926 is it doesn't break Python2; so in theory it COULD be merged.

Moving forward... I see this PR as basically as new versions of a bunch of things. Please do the following:

If the new package or version requires Python3, submit a new PR based on the branch efischer/Python3Fixes and notify me (@citibeth) when opening the PR. Please do one package per PR. I will merge it into that branch, where it becomes part of Successfully Concretize Python3-Based DAGs #7926.
If the new package can also work with Python2, please separate it into its own PR and re-submit to the general develop branch.

s-sajid-ali · 2018-06-01T14:25:13Z

So, I've made the default version to be 6.4.0 and this will work for python-3 only. All that needs to be done is to add an if statement to force it to pick versions lower than 6.0.0 if the python version is 2.7.x. I've updated the dependencies from their latest setup.py file and tested the install as well.

citibeth · 2018-06-02T16:41:50Z

All that needs to be done is to add an if statement to force it to pick versions lower than 6.0.0 if the python version is 2.7.x.

@s-sajid-ali That is done with depends_on('xyz', when='^python@...') --- which is unfortunately what doesn't work with the concretizer at this point. Please do submit a PR on top of the efischer/Python3Fixes branch.

s-sajid-ali · 2018-06-02T16:49:50Z

It looks like all the commits to develop are also included. Does that work ?

citibeth · 2018-06-02T16:53:28Z

I'll merge develop into `efischer/Python3Fixes`. If we both merge `develop` into the respective branches, then things should work well.

…

On Sat, Jun 2, 2018 at 12:49 PM, Sajid Ali ***@***.***> wrote: It looks like all the commits to develop are also included. I'll see if i can submit a PR for only this set of commits. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#8309 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB1cdzoKKMYJ6oCS3najqMYKOxgyKhdhks5t4sIxgaJpZM4UR2j9> .

s-sajid-ali · 2018-06-28T16:51:37Z

@adamjstewart : would it be possible to add the py-backcall package to develop so that it's easier for people who do use it ?

adamjstewart · 2018-06-28T17:05:38Z

You can add any package you want to develop, just submit a PR.

s-sajid-ali force-pushed the ipython_python_3 branch from 8ac7b7d to bf91046 Compare May 29, 2018 17:09

adamjstewart reviewed May 29, 2018

View reviewed changes

Update package.py

77f2a84

s-sajid-ali added 6 commits May 30, 2018 12:51

Update package.py

022ce91

Update package.py

c265ed8

Update package.py

1e3b918

Update package.py

79f0576

Update package.py

10ea591

Update package.py

61f7903

s-sajid-ali mentioned this pull request Jun 1, 2018

Successfully Concretize Python3-Based DAGs #7926

Closed

citibeth suggested changes Jun 1, 2018

View reviewed changes

citibeth closed this Jun 1, 2018

Conversation

s-sajid-ali commented May 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

s-sajid-ali commented May 29, 2018

Uh oh!

adamjstewart commented May 29, 2018

Uh oh!

adamjstewart May 29, 2018

Choose a reason for hiding this comment

Uh oh!

s-sajid-ali commented May 29, 2018

Uh oh!

adamjstewart commented May 29, 2018

Uh oh!

s-sajid-ali commented May 30, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

citibeth left a comment

Choose a reason for hiding this comment

Uh oh!

citibeth commented Jun 1, 2018

Uh oh!

s-sajid-ali commented Jun 1, 2018

Uh oh!

citibeth commented Jun 2, 2018

Uh oh!

s-sajid-ali commented Jun 2, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

citibeth commented Jun 2, 2018 via email

Uh oh!

s-sajid-ali commented Jun 28, 2018

Uh oh!

adamjstewart commented Jun 28, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

s-sajid-ali commented May 29, 2018 •

edited

Loading

s-sajid-ali commented May 30, 2018 •

edited

Loading

s-sajid-ali commented Jun 2, 2018 •

edited

Loading