Update HPX Boost dependencies by msimberg · Pull Request #1 · haampie/spack

msimberg · 2021-03-23T12:12:39Z

No description provided.

var/spack/repos/builtin/packages/hpx/package.py

msimberg · 2021-03-23T12:15:48Z

var/spack/repos/builtin/packages/hpx/package.py


+    depends_on('[email protected]:', when='@1.5')
+    depends_on('[email protected]:', when='@1.6')
+


These don't 100% belong here so I can move them to a separate PR if that's easier.

Let's keep it here

msimberg · 2021-03-23T12:16:29Z

Note, I'm building a few variants to sanity check the changes, hence a draft.

msimberg · 2021-03-23T16:09:34Z

I'm not sure where this is coming from, but if I specify some (any?) non-default settings for the HPX package I get hpx requires boost variant cxxstd=17, but spec asked for cxxstd=98 (e.g. spack spec [email protected] or hpx +tools) gives me that. This doesn't happen on develop, but does seem to happen both before this PR and after it. Do you have any idea why it might do that?

haampie · 2021-04-06T10:00:42Z

I'm not sure where this is coming from, but if I specify some (any?) non-default settings for the HPX package I get hpx requires boost variant cxxstd=17, but spec asked for cxxstd=98 (e.g. spack spec [email protected] or hpx +tools) gives me that. This doesn't happen on develop, but does seem to happen both before this PR and after it. Do you have any idea why it might do that?

I think the old concretizer is not smart enough (iiuc it just does a depth-first search, but doesn't really do backtracking). Installing clingo should be easier these days: https://spack.readthedocs.io/en/latest/getting_started.html#optional-bootstrapping-clingo

haampie · 2021-04-06T11:22:21Z

New concretizer:

$ spack spec [email protected]
[email protected]%[email protected]~async_cuda~async_mpi~cuda~examples~generic_coroutines~ipo~tools build_type=RelWithDebInfo cuda_arch=none cxxstd=17 instrumentation=none malloc=tcmalloc max_cpu_count=64 networking=tcp arch=linux-ubuntu20.04-zen2
    ^[email protected]%[email protected]~atomic~chrono~clanglibcpp~container~context~coroutine~date_time~debug~exception~fiber~filesystem~graph~icu~iostreams~locale~log~math~mpi+multithreaded~numpy~pic+program_options~python~random~regex~serialization+shared~signals~singlethreaded~system~taggedlayout~test~thread~timer~versionedlayout~wave cxxstd=17 patches=f2409bfa0e69e44817a5f8799e25c2e9e5ee50876a5aaacefd32fa647b80472f visibility=hidden arch=linux-ubuntu20.04-zen2

$ spack spec hpx +tools
[email protected]%[email protected]~async_cuda~async_mpi~cuda~examples~generic_coroutines~ipo+tools build_type=RelWithDebInfo cuda_arch=none cxxstd=17 instrumentation=none malloc=tcmalloc max_cpu_count=64 networking=tcp arch=linux-ubuntu20.04-zen2
    ^[email protected]%[email protected]~atomic~chrono~clanglibcpp~container~context~coroutine~date_time~debug~exception~fiber~filesystem~graph~icu~iostreams~locale~log~math~mpi+multithreaded~numpy~pic~program_options~python~random+regex~serialization+shared~signals~singlethreaded~system~taggedlayout~test~thread~timer~versionedlayout~wave cxxstd=17 patches=f2409bfa0e69e44817a5f8799e25c2e9e5ee50876a5aaacefd32fa647b80472f visibility=hidden arch=linux-ubuntu20.04-zen2

For some reason old gcc's don't work (yet). i'm checking what that is about

haampie · 2021-04-06T11:24:32Z

Okay, I didn't have gcc 7 installed is all 😆

$ spack spec hpx %gcc@:7

[email protected]%[email protected]~async_cuda~async_mpi~cuda~examples~generic_coroutines~ipo~tools build_type=RelWithDebInfo cuda_arch=none cxxstd=17 instrumentation=none malloc=tcmalloc max_cpu_count=64 networking=tcp arch=linux-ubuntu20.04-zen
    ^[email protected]%[email protected]~atomic~chrono~clanglibcpp~container~context~coroutine~date_time~debug~exception~fiber+filesystem~graph~icu~iostreams~locale~log~math~mpi+multithreaded~numpy~pic~program_options~python~random~regex~serialization+shared~signals~singlethreaded~system~taggedlayout~test~thread~timer~versionedlayout~wave cxxstd=17 patches=f2409bfa0e69e44817a5f8799e25c2e9e5ee50876a5aaacefd32fa647b80472f visibility=hidden arch=linux-ubuntu20.04-zen

haampie · 2021-04-06T11:31:59Z

Can I merge this @msimberg?

msimberg · 2021-04-06T11:36:13Z

Can I merge this @msimberg?

Please go ahead. Thanks for the fixes!

Before: ``` $ hyperfine '~/spack/bin/spack -e . build-env rocfft' Benchmark #1: ~/spack/bin/spack -e . build-env rocfft Time (mean ± σ): 1.593 s ± 0.016 s [User: 1.468 s, System: 0.126 s] Range (min … max): 1.575 s … 1.628 s 10 runs ``` After: ``` $ hyperfine '~/spack/bin/spack -e . build-env rocfft' Benchmark #1: ~/spack/bin/spack -e . build-env rocfft Time (mean ± σ): 1.407 s ± 0.020 s [User: 1.280 s, System: 0.127 s] Range (min … max): 1.393 s … 1.455 s 10 runs ```

Git 2.24 introduced a feature flag for repositories with many files, see: https://github.blog/2019-11-03-highlights-from-git-2-24/#feature-macros Since Spack's Git repository contains roughly 8,500 files, it can be worthwhile to enable this, especially on slow file systems such as NFS: ``` $ hyperfine --warmup 3 'cd spack-default; git status' 'cd spack-manyfiles; git status' Benchmark #1: cd spack-default; git status Time (mean ± σ): 3.388 s ± 0.095 s [User: 256.2 ms, System: 625.8 ms] Range (min … max): 3.168 s … 3.535 s 10 runs Benchmark #2: cd spack-manyfiles; git status Time (mean ± σ): 168.7 ms ± 10.9 ms [User: 98.6 ms, System: 126.1 ms] Range (min … max): 144.8 ms … 188.0 ms 19 runs Summary 'cd spack-manyfiles; git status' ran 20.09 ± 1.42 times faster than 'cd spack-default; git status' ```

@omsai

1. support version 3.1.3, which now depends on sundials@6 2. support version 3.1.2:, which broke the two patch files and therefore the two patch files have been replaced by more flexible filter_file() commands inside a patch() function. 3. rename the variant for python extension from using the package name "+pyuqtk" to the more standard "+python" 4. add maintainers @omsai and the upstream developer @bjdebus who offered to help with the spack packaging. 5. swig should only be a build-time dependency. swig is only necessary until @:3.1.0 6. confirmed python dependencies are correct by inspecting imports, subset python dependencies type to build, run, and confirmed all 31 build-time tests pass including the 9 python tests: ```console $ spack env create uqtk-dev $ spack add [email protected] $ spack install --test root && cat $(spack location -i uqtk)/.spack/install-time-test-log.txt ==> Testing package uqtk-3.1.3-nok6fut ==> [2023-04-19-14:56:25.005361] Running build-time tests ==> [2023-04-19-14:56:25.005536] RUN-TESTS: build-time tests [check] ==> [2023-04-19-14:56:25.009543] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'test' ==> [2023-04-19-14:56:25.014903] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' 'test' Running tests... /home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/cmake-3.26.3-zjmsfz23j5l4ytniz26uzvxonlu5qebr/bin/ctest --force-new-ctest-process Test project /tmp/omsai/spack-stage/spack-stage-uqtk-3.1.3-nok6fut47h42cnaau7wkoohgqy5f2qqa/spack-build-nok6fut Start 1: ArrayReadAndWrite Start 2: ArrayDelColumn Start 3: Array1DMiscTest Start 4: Array2DMiscTest 1/31 Test #1: ArrayReadAndWrite ................ Passed 0.01 sec Start 5: ArraySortTest 2/31 Test #2: ArrayDelColumn ................... Passed 0.01 sec Start 6: MultiIndexTest 3/31 Test #3: Array1DMiscTest .................. Passed 0.01 sec Start 7: CorrTest 4/31 Test #4: Array2DMiscTest .................. Passed 0.01 sec Start 8: QuadLUTest 5/31 Test #5: ArraySortTest .................... Passed 0.02 sec Start 9: MCMC2dTest 6/31 Test #6: MultiIndexTest ................... Passed 0.01 sec Start 10: MCMCRandomTest 7/31 Test #8: QuadLUTest ....................... Passed 0.02 sec Start 11: MCMCNestedTest 8/31 Test #10: MCMCRandomTest ................... Passed 0.02 sec Start 12: Deriv1dTest 9/31 Test #12: Deriv1dTest ...................... Passed 0.01 sec Start 13: SecondDeriv1dTest 10/31 Test #13: SecondDeriv1dTest ................ Passed 0.01 sec Start 14: GradHessianTest 11/31 Test #11: MCMCNestedTest ................... Passed 0.03 sec Start 15: GradientPCETest 12/31 Test #14: GradHessianTest .................. Passed 0.01 sec Start 16: PCE1dTest 13/31 Test #15: GradientPCETest .................. Passed 0.01 sec Start 17: PCEImplTest 14/31 Test #16: PCE1dTest ........................ Passed 0.01 sec Start 18: PCELogTest 15/31 Test #18: PCELogTest ....................... Passed 0.01 sec Start 19: Hessian2dTest 16/31 Test #19: Hessian2dTest .................... Passed 0.01 sec Start 20: BCS1dTest 17/31 Test #20: BCS1dTest ........................ Passed 0.01 sec Start 21: BCS2dTest 18/31 Test #21: BCS2dTest ........................ Passed 0.01 sec Start 22: LowRankRegrTest 19/31 Test #22: LowRankRegrTest .................. Passed 0.01 sec Start 23: PyModTest 20/31 Test #17: PCEImplTest ...................... Passed 0.07 sec Start 24: PyArrayTest 21/31 Test #23: PyModTest ........................ Passed 0.08 sec Start 25: PyArrayTest2 22/31 Test #25: PyArrayTest2 ..................... Passed 0.30 sec Start 26: PyQuadTest 23/31 Test #24: PyArrayTest ...................... Passed 1.44 sec Start 27: PyBCSTest1D 24/31 Test #26: PyQuadTest ....................... Passed 1.68 sec Start 28: PyBCSTest2D 25/31 Test #27: PyBCSTest1D ...................... Passed 1.66 sec Start 29: PyBADPTest 26/31 Test #7: CorrTest ......................... Passed 3.43 sec Start 30: PyRegressionTest 27/31 Test #28: PyBCSTest2D ...................... Passed 1.50 sec Start 31: PyGalerkinTest 28/31 Test #9: MCMC2dTest ....................... Passed 3.90 sec 29/31 Test #29: PyBADPTest ....................... Passed 1.66 sec 30/31 Test #30: PyRegressionTest ................. Passed 1.72 sec 31/31 Test #31: PyGalerkinTest ................... Passed 1.63 sec 100% tests passed, 0 tests failed out of 31 Total Test time (real) = 5.35 sec ==> [2023-04-19-14:56:30.382797] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'check' ==> [2023-04-19-14:56:30.385983] Target 'check' not found in Makefile ```

Update HPX Boost dependencies

ffb3171

msimberg commented Mar 23, 2021

View reviewed changes

var/spack/repos/builtin/packages/hpx/package.py Outdated Show resolved Hide resolved

msimberg commented Mar 23, 2021

View reviewed changes

var/spack/repos/builtin/packages/hpx/package.py Show resolved Hide resolved

msimberg commented Mar 23, 2021

View reviewed changes

Change compiler ^ to %

ec109c3

haampie marked this pull request as ready for review April 6, 2021 11:31

haampie merged commit ecc94b0 into haampie:feature/make-boost-composable Apr 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update HPX Boost dependencies#1

Update HPX Boost dependencies#1
haampie merged 2 commits intohaampie:feature/make-boost-composablefrom
msimberg:feature/make-boost-composable

msimberg commented Mar 23, 2021

Uh oh!

Uh oh!

Uh oh!

msimberg Mar 23, 2021

Uh oh!

haampie Apr 6, 2021

Uh oh!

msimberg commented Mar 23, 2021

Uh oh!

msimberg commented Mar 23, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

msimberg commented Apr 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		depends_on('[email protected]:', when='@1.5')
		depends_on('[email protected]:', when='@1.6')

Conversation

msimberg commented Mar 23, 2021

Uh oh!

Uh oh!

Uh oh!

msimberg Mar 23, 2021

Choose a reason for hiding this comment

Uh oh!

haampie Apr 6, 2021

Choose a reason for hiding this comment

Uh oh!

msimberg commented Mar 23, 2021

Uh oh!

msimberg commented Mar 23, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

haampie commented Apr 6, 2021

Uh oh!

msimberg commented Apr 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants