Skip to content

update to test arch#2

Merged
frankwillmore merged 112 commits intofrankwillmore:developfrom
spack:develop
May 20, 2020
Merged

update to test arch#2
frankwillmore merged 112 commits intofrankwillmore:developfrom
spack:develop

Conversation

@frankwillmore
Copy link
Copy Markdown
Owner

No description provided.

s-sajid-ali and others added 30 commits May 11, 2020 16:27
* Add pmi support (required by ucx, ofi, and gni backends)
* Add support for ucx backend
* Add dependency on MPI for pmi=simplepmi, slurmpmi, or slurmpmi2
* Remove charmpp as an MPI provider since the changes in this PR can
  add MPI as a dependency (mentioned previously)
* Install into transport_protocol-OS-arch subdirectory to match
  default charmpp installation behavior (which helps dependents find it)
- Added Fujitsu compiler options that have the same functions as the options added for GCC
- Added option to link C objects
`clingo` has some undocumented dependencies:
- `bison`, for which the default macos version is too old
- `re2c`, which isn't always available

Also, the python installation was not set up properly. Clingo by default
does an install in `~/.local`, so we need to disable that and add a few
other options to get things right.

- [x] add `bison` dependency
- [x] add `re2c` dependency
- [x] make `doxygen` dependency optional (and patch if needed)
- [x] add options to fix `python` build
- [x] make python build optional but on by default
-[x] `z3` needs a dependency on `py-setuptools`
-[x] `z3` has a run dependency on `python`, so we might as well make
     building with the python bindings default
- GCC has moved to Git
- The develop/master version requires flex
- Add myself as a maintainer
- Set the library path in the zstd patch
* py-dask: new version

* py-dask: update the constraints.

* py-dask: add a list of maintainers.
* opencv 3.2 requires cmake 3.11 or older.
caffe dependency is changed.

* revert opencv variant to caffe.
backport to opencv errors.
* r-knitr : dependencies and version updated

* Adding version conditions for dependencies
* r-withr : dependencies and version updated

* Adding version conditions for dependencies
* r-rversions : new package

* URL updated r-rversions
* r-purrr : dependencies and version update

* Dependencies updated for condition
* r-roxygen : dependencies and version updated

* Dependencies updated for condition
* r-rematch2 : new package

* Updating URL
* new version r-ellipsis

* r-ellipsis : R dependency updated

* Adding version conditions for dependencies
* new version r-cli

* r-cli : dependencies updated

* Adding version conditions for dependencies
* new version r-rcurl

* RCurl dependency R version updated

* Adding version conditions
* change print order for variants to avoid zsh parsing bugs

* change tests for new variant parse order
* revbayes: update for 1.1.0

* revbayes: Fix flake8
Modifications:

- [x] Travis now uses `bionic` as a default (`xenial` used for Python 3.5, `trusty` for Python 2.6)
- [x] Shell unit tests have been factored into their own run
- [x] `kcov` is built only for tests that upload coverage results

Overall with this we shave 3-4 mins. on each run and add an additional run of about 3 min. For some reason `kcov` 38 fails forwarding output when used with Python unit tests, so I used v34 for that and v38 (latest) for shell testing. Previously we were using v25.
* r-testthat : dependencies and version updated

* Adding version conditions for dependencies

* r-testthat: Fix dependecies condition
* py-plotly: unfix dependency versions (fixes #16544)

* py-plotly: add missing dependencies for #16546

* py-plotly: match setup.py instead of requirements.txt for #16546
h-denpo and others added 26 commits May 18, 2020 10:04
* New package: py-pyeda

* py-pyeda: added dep for python 3.3 or greater

* [py-pyeda] Added space in depends_on python line
* Add new package: hyperscan

* hyperscan: remove cmake_args()
Update Arrow to the latest version.

Also add an additional dependency on `thrift` when building with Parquet support.
Adding a variant to enable/disable kits and making the default use kits.
* angsd: add 0.933, add misc programs to install

* angsd: add r variant
* Add new package: http-get

* refine version name
* Add new package: http-load

* refine version format

* fix date/version error
Also retry jobs always, as script failure is not confined to only our
spack ci rebuild command exit code.
* Add new package: http-ping

* refine version name
* Add new package: http-post

* refine version name
The openssl build process can use the wrong perl for
various reasons, including:
* Wrong value in PERL env var
* The build process first looks for `perl5`, which the
  spack system does not provide, but some other
  distributions provide it. That way, the build process can
  end up using the wrong perl.

Stop all of these problems by explicitly setting PERL to
the to be used dependency.
… release (#16722)

Problems in 0.0.4 should be fixed for 1.0.0
@frankwillmore frankwillmore merged commit ff9b295 into frankwillmore:develop May 20, 2020
frankwillmore pushed a commit that referenced this pull request Jul 6, 2020
Whenever attempting to use any ncurses functionality within cscope, a
page fault would result within the ncurses library.

    Program received signal SIGSEGV, Segmentation fault.
    0x00007ffff7fad3cf in termattrs_sp () from .../lib/libncursesw.so.6
    (gdb) bt
    #0  0x00007ffff7fad3cf in termattrs_sp () from .../lib/libncursesw.so.6
    #1  0x00007ffff7faa794 in _nc_setupscreen_sp () from .../lib/libncursesw.so.6
    #2  0x00007ffff7fa614c in newterm_sp () from .../lib/libncursesw.so.6
    #3  0x00007ffff7fa65b9 in newterm () from .../lib/libncursesw.so.6
    #4  0x00007ffff7fa2970 in initscr () from .../lib/libncursesw.so.6
    #5  0x0000000000403dc2 in main (argc=<optimized out>, argv=0x7fffffffcea8) at main.c:574

This is due to a conflict between libtinfo.so and libtinfow.so. Both are
linked into cscope:

    $ ldd $(which cscope)
    /bin/bash: .../lib/libtinfo.so.6: no version information available (required by /bin/bash)
        linux-vdso.so.1 (0x00007fff5dbcb000)
        libncursesw.so.6 => .../lib/libncursesw.so.6 (0x00007f435cc69000)
        libtinfo.so.6 => .../lib/libtinfo.so.6 (0x00007f435cc2c000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f435ca29000)
        libtinfow.so.6 => .../lib/libtinfow.so.6 (0x00007f435c9e8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f435cca7000)

Specifically linking libtinfow.so instead of libtinfo.so resolves the
issue.

All instances of '...' above represent the path to the installed ncurses
for Spack.
frankwillmore pushed a commit that referenced this pull request Oct 19, 2021
Git 2.24 introduced a feature flag for repositories with many files, see:
https://github.blog/2019-11-03-highlights-from-git-2-24/#feature-macros

Since Spack's Git repository contains roughly 8,500 files, it can be
worthwhile to enable this, especially on slow file systems such as NFS:
```
$ hyperfine --warmup 3 'cd spack-default; git status' 'cd spack-manyfiles; git status'
Benchmark #1: cd spack-default; git status
  Time (mean ± σ):      3.388 s ±  0.095 s    [User: 256.2 ms, System: 625.8 ms]
  Range (min … max):    3.168 s …  3.535 s    10 runs

Benchmark #2: cd spack-manyfiles; git status
  Time (mean ± σ):     168.7 ms ±  10.9 ms    [User: 98.6 ms, System: 126.1 ms]
  Range (min … max):   144.8 ms … 188.0 ms    19 runs

Summary
  'cd spack-manyfiles; git status' ran
   20.09 ± 1.42 times faster than 'cd spack-default; git status'
```
frankwillmore pushed a commit that referenced this pull request May 12, 2023
1. support version 3.1.3, which now depends on sundials@6

2. support version 3.1.2:, which broke the two patch files and
   therefore the two patch files have been replaced by more flexible
   filter_file() commands inside a patch() function.

3. rename the variant for python extension from using the package name
   "+pyuqtk" to the more standard "+python"

4. add maintainers @omsai and the upstream developer @bjdebus who
   offered to help with the spack packaging.

5. swig should only be a build-time dependency.  swig is only
   necessary until @:3.1.0

6. confirmed python dependencies are correct by inspecting imports,
   subset python dependencies type to build, run, and confirmed all
   31 build-time tests pass including the 9 python tests:

```console
$ spack env create uqtk-dev
$ spack add [email protected]
$ spack install --test root && cat $(spack location -i uqtk)/.spack/install-time-test-log.txt
==> Testing package uqtk-3.1.3-nok6fut
==> [2023-04-19-14:56:25.005361] Running build-time tests
==> [2023-04-19-14:56:25.005536] RUN-TESTS: build-time tests [check]
==> [2023-04-19-14:56:25.009543] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'test'
==> [2023-04-19-14:56:25.014903] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' 'test'
Running tests...
/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/cmake-3.26.3-zjmsfz23j5l4ytniz26uzvxonlu5qebr/bin/ctest --force-new-ctest-process
Test project /tmp/omsai/spack-stage/spack-stage-uqtk-3.1.3-nok6fut47h42cnaau7wkoohgqy5f2qqa/spack-build-nok6fut
      Start  1: ArrayReadAndWrite
      Start  2: ArrayDelColumn
      Start  3: Array1DMiscTest
      Start  4: Array2DMiscTest
 1/31 Test  #1: ArrayReadAndWrite ................   Passed    0.01 sec
      Start  5: ArraySortTest
 2/31 Test  #2: ArrayDelColumn ...................   Passed    0.01 sec
      Start  6: MultiIndexTest
 3/31 Test  #3: Array1DMiscTest ..................   Passed    0.01 sec
      Start  7: CorrTest
 4/31 Test  #4: Array2DMiscTest ..................   Passed    0.01 sec
      Start  8: QuadLUTest
 5/31 Test  #5: ArraySortTest ....................   Passed    0.02 sec
      Start  9: MCMC2dTest
 6/31 Test  #6: MultiIndexTest ...................   Passed    0.01 sec
      Start 10: MCMCRandomTest
 7/31 Test  #8: QuadLUTest .......................   Passed    0.02 sec
      Start 11: MCMCNestedTest
 8/31 Test #10: MCMCRandomTest ...................   Passed    0.02 sec
      Start 12: Deriv1dTest
 9/31 Test #12: Deriv1dTest ......................   Passed    0.01 sec
      Start 13: SecondDeriv1dTest
10/31 Test #13: SecondDeriv1dTest ................   Passed    0.01 sec
      Start 14: GradHessianTest
11/31 Test #11: MCMCNestedTest ...................   Passed    0.03 sec
      Start 15: GradientPCETest
12/31 Test #14: GradHessianTest ..................   Passed    0.01 sec
      Start 16: PCE1dTest
13/31 Test #15: GradientPCETest ..................   Passed    0.01 sec
      Start 17: PCEImplTest
14/31 Test #16: PCE1dTest ........................   Passed    0.01 sec
      Start 18: PCELogTest
15/31 Test #18: PCELogTest .......................   Passed    0.01 sec
      Start 19: Hessian2dTest
16/31 Test #19: Hessian2dTest ....................   Passed    0.01 sec
      Start 20: BCS1dTest
17/31 Test #20: BCS1dTest ........................   Passed    0.01 sec
      Start 21: BCS2dTest
18/31 Test #21: BCS2dTest ........................   Passed    0.01 sec
      Start 22: LowRankRegrTest
19/31 Test #22: LowRankRegrTest ..................   Passed    0.01 sec
      Start 23: PyModTest
20/31 Test #17: PCEImplTest ......................   Passed    0.07 sec
      Start 24: PyArrayTest
21/31 Test #23: PyModTest ........................   Passed    0.08 sec
      Start 25: PyArrayTest2
22/31 Test #25: PyArrayTest2 .....................   Passed    0.30 sec
      Start 26: PyQuadTest
23/31 Test #24: PyArrayTest ......................   Passed    1.44 sec
      Start 27: PyBCSTest1D
24/31 Test #26: PyQuadTest .......................   Passed    1.68 sec
      Start 28: PyBCSTest2D
25/31 Test #27: PyBCSTest1D ......................   Passed    1.66 sec
      Start 29: PyBADPTest
26/31 Test  #7: CorrTest .........................   Passed    3.43 sec
      Start 30: PyRegressionTest
27/31 Test #28: PyBCSTest2D ......................   Passed    1.50 sec
      Start 31: PyGalerkinTest
28/31 Test  #9: MCMC2dTest .......................   Passed    3.90 sec
29/31 Test #29: PyBADPTest .......................   Passed    1.66 sec
30/31 Test #30: PyRegressionTest .................   Passed    1.72 sec
31/31 Test #31: PyGalerkinTest ...................   Passed    1.63 sec

100% tests passed, 0 tests failed out of 31

Total Test time (real) =   5.35 sec
==> [2023-04-19-14:56:30.382797] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'check'
==> [2023-04-19-14:56:30.385983] Target 'check' not found in Makefile
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.