New package: py-wub, with supporting fixes by hartzell · Pull Request #13773 · spack/spack

hartzell · 2019-11-18T17:32:29Z

add py-wub
add py-pycmd because py-wub needs it
update py-statsmodels, which needs at least v0.9.0 to work with
python3.7 because cython.

- add py-wub - add py-pycmd because py-wub needs it - update py-statsmodels, which needs at least v0.9.0 to work with python3.7 because cython.

var/spack/repos/builtin/packages/py-pycmd/package.py

var/spack/repos/builtin/packages/py-statsmodels/package.py

var/spack/repos/builtin/packages/py-wub/package.py

var/spack/repos/builtin/packages/py-statsmodels/package.py

var/spack/repos/builtin/packages/py-wub/package.py

The statsmodels tests weren't run in the previous version of the package. If I enable them, the fail. Update the package comment with the statsmodels issue I opened to track the problem: statsmodels/statsmodels#6263

var/spack/repos/builtin/packages/py-wub/package.py

hartzell · 2019-11-21T20:03:56Z

[edit: add ellipsis to test crash code block]

I'm trying to get the tests working. The folks over in statsmodels/statsmodels#6263 suggested that I use "older" versions of numpy and scipy, something from the same time period as 0.10.1. I've had to bump forward a bit from what was listed in their INSTALL.txt, but have it in place.

It also turns out that:

We require pytest and don't support running tests using setup.

I've done this:

-    # depends_on('py-nose', type='test')
+    depends_on('py-pytest', type='test')
+
+    def test(self):
+        pytest = which('pytest')
+        pytest('statsmodels')

but the wheels all fall off, spectacularly:

==> [2019-11-21-11:44:20.875975] '/Users/hartzell/tmp/spack/opt/spack/darwin-highsierra-x86_64/clang-10.0.0-apple/py-pytest-5.2.1-rwdps4klpalu5hdlonm4krdubp55jfki/bin/pytest' 'statsmodels'
============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /private/var/folders/f7/fcb363190nz0jbypndp3nj_m0000gn/T/hartzell/spack-stage/spack-stage-py-statsmodels-0.10.1-ifl2ehgyg4k3bzra6nuuyddirhmzrgmc/spack-src, inifile: setup.cfg, testpaths: statsmodels
collected 2960 items / 68 errors / 2892 selected

==================================== ERRORS ====================================
_______ ERROR collecting statsmodels/base/tests/test_generic_methods.py ________
ImportError while importing test module '/private/var/folders/f7/fcb363190nz0jbypndp3nj_m0000gn/T/hartzell/spack-stage/spack-stage-py-statsmodels-0.10.1-ifl2ehgyg4k3bzra6nuuyddirhmzrgmc/spack-src/statsmodels/base/tests/test_generic_methods.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
statsmodels/base/tests/test_generic_methods.py:20: in <module>
    import statsmodels.api as sm
statsmodels/api.py:9: in <module>
    from .regression.recursive_ls import RecursiveLS
statsmodels/regression/recursive_ls.py:14: in <module>
    from statsmodels.tsa.statespace.mlemodel import (
statsmodels/tsa/statespace/mlemodel.py:31: in <module>
    from .simulation_smoother import SimulationSmoother
statsmodels/tsa/statespace/simulation_smoother.py:10: in <module>
    from .kalman_smoother import KalmanSmoother
statsmodels/tsa/statespace/kalman_smoother.py:11: in <module>
    from statsmodels.tsa.statespace.representation import OptionWrapper
statsmodels/tsa/statespace/representation.py:10: in <module>
    from .tools import (
statsmodels/tsa/statespace/tools.py:16: in <module>
    from . import (_initialization, _representation, _kalman_filter,
E   ImportError: cannot import name '_initialization' from 'statsmodels.tsa.statespace' (/private/var/folders/f7/fcb363190nz0jbypndp3nj_m0000gn/T/hartzell/spack-stage/spack-stage-py-statsmodels-0.10.1-ifl2ehgyg4k3bzra6nuuyddirhmzrgmc/spack-src/statsmodels/tsa/statespace/__init__.py)
________ ERROR collecting statsmodels/base/tests/test_shrink_pickle.py _________
ImportError while importing test module '/private/var/folders/f7/fcb363190nz0jbypndp3nj_m0000gn/T/hartzell/spack-stage/spack-stage-py-statsmodels-0.10.1-ifl2ehgyg4k3bzra6nuuyddirhmzrgmc/spack-src/statsmodels/base/tests/test_shrink_pickle.py'.
[... and on and on and on ...]

The bit that's failing seems to be a C extension.

I cribbed the test setup from something I found in py-scikit-learn, it also sets up a working directory, but I don't think that's my problem here.

Suggestions?

adamjstewart · 2019-11-21T23:02:16Z

Flake8:

var/spack/repos/builtin/packages/py-statsmodels/package.py:40: [W391] blank line at end of file
var/spack/repos/builtin/packages/py-wub/package.py:1: [E902] TokenError: EOF in multi-line statement

adamjstewart · 2019-11-21T23:03:30Z

No idea on why the unit tests would be failing. Just keep working with the developers. You can restrict the package to older versions of numpy if you need to.

hartzell · 2019-11-21T23:20:17Z

Thanks. This is making me look forward to my next dentist appointment.... I appreciate the help!

hartzell · 2019-11-22T01:17:39Z

I have the tests running, and perhaps they'll even pass (20 minutes in, 40% done...), if I run them from this directory:

/private/var/folders/f7/fcb363190nz0jbypndp3nj_m0000gn/T/hartzell/spack-stage/spack-stage-py-statsmodels-0.10.1-zmdnjrvtr4lu7p64onk5t24rlgjyxmkl/spack-src/build/lib.macosx-10.13.6-x86_64-3.7

Is there a standard relationship of build/lib.macosx-10.13.6-x86_64-3.7 to prefix or do I need to cons it up for each platform?

adamjstewart · 2019-11-22T03:42:42Z

Hmm, you shouldn't need to run the tests from a specific directory. The only thing you might have a problem with is running the tests in the build directory, which I why I usually create a spack-test directory. But no, there isn't an easy way to predict that directory, I would just use globs.

hartzell · 2019-11-22T18:14:50Z

Hmm, you shouldn't need to run the tests from a specific directory.

Well, empirically they fail if I run them in spack-src and they fail if I mkdir spack-test and run them there. They work (yay) if I run them from build/lib*.

- need to use [email protected]: - need to run the tests from within the build/lib* dir

hartzell · 2019-11-22T22:28:59Z

I think that this is all in order, barring additional feedback.

Add py-wub, with supporting fixes

f4337ba

- add py-wub - add py-pycmd because py-wub needs it - update py-statsmodels, which needs at least v0.9.0 to work with python3.7 because cython.

ax3l added new-package new-version labels Nov 18, 2019

ax3l approved these changes Nov 18, 2019

View reviewed changes

adamjstewart requested changes Nov 20, 2019

View reviewed changes

Update based on Adam's comments

b8bdcc0

adamjstewart reviewed Nov 21, 2019

View reviewed changes

var/spack/repos/builtin/packages/py-statsmodels/package.py Outdated Show resolved Hide resolved

adamjstewart reviewed Nov 21, 2019

View reviewed changes

var/spack/repos/builtin/packages/py-wub/package.py Outdated Show resolved Hide resolved

ax3l assigned adamjstewart Nov 21, 2019

hartzell mentioned this pull request Nov 21, 2019

tests fail on Mac with Python 3.7.4, built with Spack. statsmodels/statsmodels#6263

Closed

hartzell added 2 commits November 21, 2019 10:02

Fix dependency types for py-six in py-wub

0fe6462

statsmodels tests fail, update comment w/ Issue #

2f2a5bd

The statsmodels tests weren't run in the previous version of the package. If I enable them, the fail. Update the package comment with the statsmodels issue I opened to track the problem: statsmodels/statsmodels#6263

adamjstewart requested changes Nov 21, 2019

View reviewed changes

Update dependency types in py-wub/package.py

9f40020

adamjstewart approved these changes Nov 21, 2019

View reviewed changes

flake8 cleanups

8bd56fd

Make statsmodels tests work

511031a

- need to use [email protected]: - need to run the tests from within the build/lib* dir

adamjstewart merged commit 48420d0 into spack:develop Nov 23, 2019

Conversation

hartzell commented Nov 18, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hartzell commented Nov 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamjstewart commented Nov 21, 2019

Uh oh!

adamjstewart commented Nov 21, 2019

Uh oh!

hartzell commented Nov 21, 2019

Uh oh!

hartzell commented Nov 22, 2019

Uh oh!

adamjstewart commented Nov 22, 2019

Uh oh!

hartzell commented Nov 22, 2019

Uh oh!

hartzell commented Nov 22, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hartzell commented Nov 21, 2019 •

edited

Loading