Skip to content

merge upstream#11

Merged
luke-dt merged 41 commits intodantaslab:package/pharokka-cleanfrom
spack:develop
Dec 5, 2022
Merged

merge upstream#11
luke-dt merged 41 commits intodantaslab:package/pharokka-cleanfrom
spack:develop

Conversation

@luke-dt
Copy link
Copy Markdown
Member

@luke-dt luke-dt commented Dec 5, 2022

No description provided.

iarspider and others added 30 commits December 1, 2022 17:19
* Herwig3: make njet, vbfnlo dependencies optional...
  also drop openloops dependency when building on PowerPC
* Update package.py
…psafe (#32849)

* py-flask: add 2.2.2, py-werkzeug: add 2.2.2, py-markupsafe: add 2.1.1

* Remove py-dataclasses dependency
…4225)

* Bugfix: Fetch should not force use of curl to check url existence

* Switch type hints from comments to actual hints
* py-keyrings-alt: add 4.2.0

* Add missing py-jaraco-classes dependency
All Spec attributes are now represented as `attr(attribute_name, ... args ...)`, e.g.
`attr(node, "hdf5")` instead of `node("hdf5")`, as we *have* to maintain the `attr()`
form anyway, and it simplifies the encoding to just maintain one form of the Spec
information.

Background
----------

In #20644, we unified the way conditionals are done in the concretizer, but this
introduced a nasty aspect to the encoding: we have to maintain everything we want in
general conditions in two forms: `predicate(...)` and `attr("predicate", ...)`. For
example, here's the start of the table of spec attributes we had to maintain:

```prolog
node(Package)                      :- attr("node", Package).
virtual_node(Virtual)              :- attr("virtual_node", Virtual).
hash(Package, Hash)                :- attr("hash", Package, Hash).
version(Package, Version)          :- attr("version", Package, Version).
...
```

```prolog
attr("node", Package)              :- node(Package).
attr("virtual_node", Virtual)      :- virtual_node(Virtual).
attr("hash", Package, Hash)        :- hash(Package, Hash).
attr("version", Package, Version)  :- version(Package, Version).
...
```

This adds cognitive load to understanding how the concretizer works, as you have to
understand the equivalence between the two forms of spec attributes. It also makes the
general condition logic in #20644 hard to explain, and it's easy to forget to add a new
equivalence to this list when adding new spec attributes (at least two people have been
bitten by this).

Solution
--------

- [x] remove the equivalence list from `concretize.lp`
- [x] simplify `spec_clauses()`, `condition()`, and other functions in `asp.py` that need
      to deal with `Spec` attributes.
- [x] Convert all old-form spec attributes in `concretize.lp` to the `attr()` form
- [x] Simplify `display.lp`, where we also had to maintain a list of spec attributes. Now
      we only need to show `attr/2`, `attr/3`, and `attr/4`.
- [x] Simplify model extraction logic in `asp.py`.

Performance
-----------

This seems to result in a smaller grounded problem (as there are no longer duplicated
`attr("foo", ...)` / `foo(...)` predicates in the program), but it also adds a slight
performance overhead vs. develop. Ultimately, simplifying the encoding will be a win,
particularly for improving error messages.

Notes
-----

This will simplify future node refactors in `concretize.lp` (e.g., not identifying nodes
by package name, which we need for separate build dependencies).

I'm still not entirely used to reading `attr()` notation, but I thnk it's ultimately
clearer than what we did before. We need more uniform naming, and it's now clear what is
part of a solution. We should probably continue making the encoding of `concretize.lp`
simpler and more self-explanatory. It may make sense to rename `attr` to something like
`node_attr` and to simplify the names of node attributes. It also might make sense to do
something similar for other types of predicates in `concretize.lp`.
* feat(Hermes): update version to 0.9.0-beta.
   This is the latest release.
* feat(Hermes): fix checksum.
  Credit: @tldahlgren
…conditional variants (#34244)

* ROOT: add GSL/math dependency
* ROOT: use conditional variants instead of conflicts
* patch command: add concretizer args
* tab completion
* py-alphafold: update to 2.2.4, update dependencies

* style
* vecgeom: add new 1.2.1 version
* vecgeom: introduce conflict between gcc/cuda

Recent tests of vecgeom in Spack environments have shown that the build
with +cuda fails with GCC >= 11.3 and CUDA < 11.7 with error

...lib/gcc/x86_64-pc-linux-gnu/11.3.0/include/serializeintrin.h(41):
error: identifier "__builtin_ia32_serialize" is undefined

1 error detected in the compilation of
".../VecGeom/source/BVHManager.cu".

Other GCC/CUDA combinations appear o.k.

Avoid this error in spack, and document it for users, with a conflict
directive to express the restriction.
* doxygen: add build-tool tag
   This allows it to be included automatically as an external. No one links against doxygen so this should be ok.
* doxygen: add self as maintainer
We've stopped supporting Python 2, and contributors are noticing that our CI no longer
allows Python 2.7 comment type hints. They end up having to adapt them, but this adds
extra unrelated work to PRs.

- [x] Move to 3.6 type hints across the entire code base
* py-mne: add 1.2.2 and depencendy packages

* py-mne: add 1.2.3

* Remove unnecessary when statement
We no longer support Python <3.6, so we don't need to check whether Python supports SSL
verification in `spack.util.web`.

- [x] Remove a bunch of logic we needed to appease Python 2
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
glennpj and others added 11 commits December 5, 2022 11:58
All the vermin annotations we were using were for optional features introduced in early
Python 3 versions. We no longer need any of them, as we only support Python 3.6+. If we
start optionally using features from newer Pythons than 3.6, we will need more vermin
annotations.

Co-authored-by: Harmen Stoppels <[email protected]>

Co-authored-by: Harmen Stoppels <[email protected]>
* py-nbclassic: add 0.4.8 and new package py-notebook-shim

* Add missing dependencies
* Add the very first version of cernlib
* Update package.py
* Update package.py

Co-authored-by: Andrii Verbytskyi <[email protected]>
* epsic: add epsic package to spack

* psrcat: add psrcat to spack

* psrchive: add psarchive to spack

* tempo: add tempo package to spack
The 0.9.0-beta requires yaml-cpp for parsing the configuration file format in YAML.

P.S. I'm using https://www.conventionalcommits.org/en/v1.0.0/#specification for this commit message.
* LAMMPS: Add version 20220803 and 20220623.1

* LAMMPS: Add 20220915, 20221103, and 20220623.2
@luke-dt luke-dt merged commit d1b740c into dantaslab:package/pharokka-clean Dec 5, 2022
luke-dt pushed a commit that referenced this pull request Jun 6, 2023
1. support version 3.1.3, which now depends on sundials@6

2. support version 3.1.2:, which broke the two patch files and
   therefore the two patch files have been replaced by more flexible
   filter_file() commands inside a patch() function.

3. rename the variant for python extension from using the package name
   "+pyuqtk" to the more standard "+python"

4. add maintainers @omsai and the upstream developer @bjdebus who
   offered to help with the spack packaging.

5. swig should only be a build-time dependency.  swig is only
   necessary until @:3.1.0

6. confirmed python dependencies are correct by inspecting imports,
   subset python dependencies type to build, run, and confirmed all
   31 build-time tests pass including the 9 python tests:

```console
$ spack env create uqtk-dev
$ spack add [email protected]
$ spack install --test root && cat $(spack location -i uqtk)/.spack/install-time-test-log.txt
==> Testing package uqtk-3.1.3-nok6fut
==> [2023-04-19-14:56:25.005361] Running build-time tests
==> [2023-04-19-14:56:25.005536] RUN-TESTS: build-time tests [check]
==> [2023-04-19-14:56:25.009543] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'test'
==> [2023-04-19-14:56:25.014903] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' 'test'
Running tests...
/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/cmake-3.26.3-zjmsfz23j5l4ytniz26uzvxonlu5qebr/bin/ctest --force-new-ctest-process
Test project /tmp/omsai/spack-stage/spack-stage-uqtk-3.1.3-nok6fut47h42cnaau7wkoohgqy5f2qqa/spack-build-nok6fut
      Start  1: ArrayReadAndWrite
      Start  2: ArrayDelColumn
      Start  3: Array1DMiscTest
      Start  4: Array2DMiscTest
 1/31 Test  #1: ArrayReadAndWrite ................   Passed    0.01 sec
      Start  5: ArraySortTest
 2/31 Test  #2: ArrayDelColumn ...................   Passed    0.01 sec
      Start  6: MultiIndexTest
 3/31 Test  #3: Array1DMiscTest ..................   Passed    0.01 sec
      Start  7: CorrTest
 4/31 Test  #4: Array2DMiscTest ..................   Passed    0.01 sec
      Start  8: QuadLUTest
 5/31 Test  #5: ArraySortTest ....................   Passed    0.02 sec
      Start  9: MCMC2dTest
 6/31 Test  #6: MultiIndexTest ...................   Passed    0.01 sec
      Start 10: MCMCRandomTest
 7/31 Test  #8: QuadLUTest .......................   Passed    0.02 sec
      Start 11: MCMCNestedTest
 8/31 Test #10: MCMCRandomTest ...................   Passed    0.02 sec
      Start 12: Deriv1dTest
 9/31 Test #12: Deriv1dTest ......................   Passed    0.01 sec
      Start 13: SecondDeriv1dTest
10/31 Test #13: SecondDeriv1dTest ................   Passed    0.01 sec
      Start 14: GradHessianTest
11/31 Test #11: MCMCNestedTest ...................   Passed    0.03 sec
      Start 15: GradientPCETest
12/31 Test spack#14: GradHessianTest ..................   Passed    0.01 sec
      Start 16: PCE1dTest
13/31 Test spack#15: GradientPCETest ..................   Passed    0.01 sec
      Start 17: PCEImplTest
14/31 Test spack#16: PCE1dTest ........................   Passed    0.01 sec
      Start 18: PCELogTest
15/31 Test spack#18: PCELogTest .......................   Passed    0.01 sec
      Start 19: Hessian2dTest
16/31 Test spack#19: Hessian2dTest ....................   Passed    0.01 sec
      Start 20: BCS1dTest
17/31 Test spack#20: BCS1dTest ........................   Passed    0.01 sec
      Start 21: BCS2dTest
18/31 Test spack#21: BCS2dTest ........................   Passed    0.01 sec
      Start 22: LowRankRegrTest
19/31 Test spack#22: LowRankRegrTest ..................   Passed    0.01 sec
      Start 23: PyModTest
20/31 Test spack#17: PCEImplTest ......................   Passed    0.07 sec
      Start 24: PyArrayTest
21/31 Test spack#23: PyModTest ........................   Passed    0.08 sec
      Start 25: PyArrayTest2
22/31 Test spack#25: PyArrayTest2 .....................   Passed    0.30 sec
      Start 26: PyQuadTest
23/31 Test spack#24: PyArrayTest ......................   Passed    1.44 sec
      Start 27: PyBCSTest1D
24/31 Test spack#26: PyQuadTest .......................   Passed    1.68 sec
      Start 28: PyBCSTest2D
25/31 Test spack#27: PyBCSTest1D ......................   Passed    1.66 sec
      Start 29: PyBADPTest
26/31 Test  #7: CorrTest .........................   Passed    3.43 sec
      Start 30: PyRegressionTest
27/31 Test spack#28: PyBCSTest2D ......................   Passed    1.50 sec
      Start 31: PyGalerkinTest
28/31 Test  #9: MCMC2dTest .......................   Passed    3.90 sec
29/31 Test spack#29: PyBADPTest .......................   Passed    1.66 sec
30/31 Test spack#30: PyRegressionTest .................   Passed    1.72 sec
31/31 Test spack#31: PyGalerkinTest ...................   Passed    1.63 sec

100% tests passed, 0 tests failed out of 31

Total Test time (real) =   5.35 sec
==> [2023-04-19-14:56:30.382797] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'check'
==> [2023-04-19-14:56:30.385983] Target 'check' not found in Makefile
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.