uqtk: bump by omsai · Pull Request #36670 · spack/spack

omsai · 2023-04-05T21:10:57Z

support version 3.1.3, which now depends on sundials@6.
support version 3.1.2:, which broke the two patch files and therefore the two patch files have been replaced by more flexible filter_file() commands inside a patch() function.
rename variant for python extension from using the package name to the more standard +python.
add maintainers @omsai and the upstream developer @bjdebus who offered to help with the spack packaging in pytests out of date in release 3.1.2 sandialabs/UQTk#7 (comment)

Edit: Fixed the minor formatting issue caught by the black check of spack style.

spackbot-app · 2023-04-05T21:11:08Z

@bjdebus can you review this PR?

This PR modifies the following package(s), for which you are listed as a maintainer:

uqtk

omsai · 2023-04-06T15:35:51Z

@bjdebus can you review this PR?

This PR modifies the following package(s), for which you are listed as a maintainer:

uqtk

@bjdebus I think you can ignore the @spackbot-app's message for now and start reviewing the package when you're more familiar with spack. Thanks!

tldahlgren · 2023-04-06T16:31:40Z

@bjdebus can you review this PR?
This PR modifies the following package(s), for which you are listed as a maintainer:

uqtk

@bjdebus I think you can ignore the @spackbot-app's message for now and start reviewing the package when you're more familiar with spack. Thanks!

Or you can comment on the build.

@bjdebus Please confirm you are willing to be listed as a maintainer on the package. (See https://spack.readthedocs.io/en/latest/packaging_guide.html#maintainers for more information.)

tldahlgren · 2023-04-06T16:34:02Z

@padamson As a significant contributor to this package within the last year, would you be interested in commenting on this PR?

tldahlgren

Confirmed the two version sha256.

tldahlgren · 2023-04-06T16:35:39Z

@adamjstewart Thought I'd ping you on this PR since it changes the package to extend python just in case you might want to comment on it. (I know you're really busy though.)

bjdebus · 2023-04-06T18:16:36Z

Yes, I am willing to be listed as a maintainer for this package. Note, I am new to Spack, and am also super swamped with some other engagements, so it may be a week or so before I can contribute in a meaningful way.

adamjstewart · 2023-04-06T18:53:33Z

var/spack/repos/builtin/packages/uqtk/package.py

+    depends_on("py-scipy", when="+python")
+    depends_on("py-matplotlib", when="+python")
+    depends_on("py-pymc3", when="+python")
+    depends_on("swig", when="+python")


Are these dependencies documented anywhere? Most of these should be type=("build", "run"), although swig is usually type="build"

UQTk no longer depends on swig.

@adamjstewart thanks for catching the swig build time only dependency issue. Indeed, there isn't a requirements.txt file for the uqtk python extension whereby one might quickly verify the python extension dependency list and any minimum python dependency versions. My understanding is that some of those dependencies (for example py-pymc3) are meant to support the examples and are not part of the core package, so maybe they should only be pulled in with a +examples variant (I checked that several spack packages also use an "examples" variant) when enabling the +python variant. I'll comb through the package and make those changes.

Or you can comment on the build.

@tldahlgren yes, my mistake! I appreciate your nudge for reviewers and I'm sorry for getting ahead of myself!

@adamjstewart -

I've fixed swig to be a build-time only dependency. Also the swig dependency was removed after @3.1.0 per sandialabs/UQTk@370e8a7 (those directories contained the swig wrapper files)

I had a video conference with @bjdebus yesterday and the python wrapper is a work in progress. Eventually it will become its own package on pypi. Currently, as you saw, there isn't any setup.py or requirements.txt, etc in the python wrapper to check dependencies against, but I did confirm that the current python dependencies are all required and limited to those packages by checking the imports and excluding the example programs:
$ git clone https://github.com/sandialabs/UQTk $ cd UQTk/ $ find PyUQTk/ \ -name '*.py' \ -not -path 'PyUQTk/pytests/*' \ -exec grep -h -w import {} + | grep -E -v '[[:space:]]*#|[[:space:]]*from [.]' | awk '{$1=$1;print}' | sort | uniq
... and also running all the Python unit tests which I've included in the latest commit message of 0ff1316 (The 9 python tests among the 31 total tests start with Py*)

So everything looks okay to me.

Python deps still need to be type=("build", "run") at least

@adamjstewart Whoops, my mistake! Sorry for not reading your first comment more carefully. I've fixed the python deps with your instruction. Also, I squashed the commits to reduce the noise in the log.

1. swig should only be a build-time dependency. 2. swig is only necessary until @:3.1.0 3. confirmed python dependencies are correct and that all tests pass: ```console $ spack env create uqtk-dev $ spack add [email protected] $ spack install -v --test root && cat $(spack location -i uqtk)/.spack/install-time-test-log.txt ==> Testing package uqtk-3.1.3-nok6fut ==> [2023-04-19-14:56:25.005361] Running build-time tests ==> [2023-04-19-14:56:25.005536] RUN-TESTS: build-time tests [check] ==> [2023-04-19-14:56:25.009543] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'test' ==> [2023-04-19-14:56:25.014903] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' 'test' Running tests... /home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/cmake-3.26.3-zjmsfz23j5l4ytniz26uzvxonlu5qebr/bin/ctest --force-new-ctest-process Test project /tmp/omsai/spack-stage/spack-stage-uqtk-3.1.3-nok6fut47h42cnaau7wkoohgqy5f2qqa/spack-build-nok6fut Start 1: ArrayReadAndWrite Start 2: ArrayDelColumn Start 3: Array1DMiscTest Start 4: Array2DMiscTest 1/31 Test #1: ArrayReadAndWrite ................ Passed 0.01 sec Start 5: ArraySortTest 2/31 Test #2: ArrayDelColumn ................... Passed 0.01 sec Start 6: MultiIndexTest 3/31 Test #3: Array1DMiscTest .................. Passed 0.01 sec Start 7: CorrTest 4/31 Test #4: Array2DMiscTest .................. Passed 0.01 sec Start 8: QuadLUTest 5/31 Test #5: ArraySortTest .................... Passed 0.02 sec Start 9: MCMC2dTest 6/31 Test #6: MultiIndexTest ................... Passed 0.01 sec Start 10: MCMCRandomTest 7/31 Test #8: QuadLUTest ....................... Passed 0.02 sec Start 11: MCMCNestedTest 8/31 Test #10: MCMCRandomTest ................... Passed 0.02 sec Start 12: Deriv1dTest 9/31 Test #12: Deriv1dTest ...................... Passed 0.01 sec Start 13: SecondDeriv1dTest 10/31 Test #13: SecondDeriv1dTest ................ Passed 0.01 sec Start 14: GradHessianTest 11/31 Test #11: MCMCNestedTest ................... Passed 0.03 sec Start 15: GradientPCETest 12/31 Test #14: GradHessianTest .................. Passed 0.01 sec Start 16: PCE1dTest 13/31 Test #15: GradientPCETest .................. Passed 0.01 sec Start 17: PCEImplTest 14/31 Test #16: PCE1dTest ........................ Passed 0.01 sec Start 18: PCELogTest 15/31 Test #18: PCELogTest ....................... Passed 0.01 sec Start 19: Hessian2dTest 16/31 Test #19: Hessian2dTest .................... Passed 0.01 sec Start 20: BCS1dTest 17/31 Test #20: BCS1dTest ........................ Passed 0.01 sec Start 21: BCS2dTest 18/31 Test #21: BCS2dTest ........................ Passed 0.01 sec Start 22: LowRankRegrTest 19/31 Test #22: LowRankRegrTest .................. Passed 0.01 sec Start 23: PyModTest 20/31 Test #17: PCEImplTest ...................... Passed 0.07 sec Start 24: PyArrayTest 21/31 Test #23: PyModTest ........................ Passed 0.08 sec Start 25: PyArrayTest2 22/31 Test #25: PyArrayTest2 ..................... Passed 0.30 sec Start 26: PyQuadTest 23/31 Test #24: PyArrayTest ...................... Passed 1.44 sec Start 27: PyBCSTest1D 24/31 Test #26: PyQuadTest ....................... Passed 1.68 sec Start 28: PyBCSTest2D 25/31 Test #27: PyBCSTest1D ...................... Passed 1.66 sec Start 29: PyBADPTest 26/31 Test #7: CorrTest ......................... Passed 3.43 sec Start 30: PyRegressionTest 27/31 Test #28: PyBCSTest2D ...................... Passed 1.50 sec Start 31: PyGalerkinTest 28/31 Test #9: MCMC2dTest ....................... Passed 3.90 sec 29/31 Test #29: PyBADPTest ....................... Passed 1.66 sec 30/31 Test #30: PyRegressionTest ................. Passed 1.72 sec 31/31 Test #31: PyGalerkinTest ................... Passed 1.63 sec 100% tests passed, 0 tests failed out of 31 Total Test time (real) = 5.35 sec ==> [2023-04-19-14:56:30.382797] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'check' ==> [2023-04-19-14:56:30.385983] Target 'check' not found in Makefile ```

@omsai

1. support version 3.1.3, which now depends on sundials@6 2. support version 3.1.2:, which broke the two patch files and therefore the two patch files have been replaced by more flexible filter_file() commands inside a patch() function. 3. rename the variant for python extension from using the package name "+pyuqtk" to the more standard "+python" 4. add maintainers @omsai and the upstream developer @bjdebus who offered to help with the spack packaging. 5. swig should only be a build-time dependency. swig is only necessary until @:3.1.0 6. confirmed python dependencies are correct by inspecting imports, subset python dependencies type to build, run, and confirmed all 31 build-time tests pass including the 9 python tests: ```console $ spack env create uqtk-dev $ spack add [email protected] $ spack install --test root && cat $(spack location -i uqtk)/.spack/install-time-test-log.txt ==> Testing package uqtk-3.1.3-nok6fut ==> [2023-04-19-14:56:25.005361] Running build-time tests ==> [2023-04-19-14:56:25.005536] RUN-TESTS: build-time tests [check] ==> [2023-04-19-14:56:25.009543] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'test' ==> [2023-04-19-14:56:25.014903] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' 'test' Running tests... /home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/cmake-3.26.3-zjmsfz23j5l4ytniz26uzvxonlu5qebr/bin/ctest --force-new-ctest-process Test project /tmp/omsai/spack-stage/spack-stage-uqtk-3.1.3-nok6fut47h42cnaau7wkoohgqy5f2qqa/spack-build-nok6fut Start 1: ArrayReadAndWrite Start 2: ArrayDelColumn Start 3: Array1DMiscTest Start 4: Array2DMiscTest 1/31 Test #1: ArrayReadAndWrite ................ Passed 0.01 sec Start 5: ArraySortTest 2/31 Test #2: ArrayDelColumn ................... Passed 0.01 sec Start 6: MultiIndexTest 3/31 Test #3: Array1DMiscTest .................. Passed 0.01 sec Start 7: CorrTest 4/31 Test #4: Array2DMiscTest .................. Passed 0.01 sec Start 8: QuadLUTest 5/31 Test #5: ArraySortTest .................... Passed 0.02 sec Start 9: MCMC2dTest 6/31 Test #6: MultiIndexTest ................... Passed 0.01 sec Start 10: MCMCRandomTest 7/31 Test #8: QuadLUTest ....................... Passed 0.02 sec Start 11: MCMCNestedTest 8/31 Test #10: MCMCRandomTest ................... Passed 0.02 sec Start 12: Deriv1dTest 9/31 Test #12: Deriv1dTest ...................... Passed 0.01 sec Start 13: SecondDeriv1dTest 10/31 Test #13: SecondDeriv1dTest ................ Passed 0.01 sec Start 14: GradHessianTest 11/31 Test #11: MCMCNestedTest ................... Passed 0.03 sec Start 15: GradientPCETest 12/31 Test #14: GradHessianTest .................. Passed 0.01 sec Start 16: PCE1dTest 13/31 Test #15: GradientPCETest .................. Passed 0.01 sec Start 17: PCEImplTest 14/31 Test #16: PCE1dTest ........................ Passed 0.01 sec Start 18: PCELogTest 15/31 Test #18: PCELogTest ....................... Passed 0.01 sec Start 19: Hessian2dTest 16/31 Test #19: Hessian2dTest .................... Passed 0.01 sec Start 20: BCS1dTest 17/31 Test #20: BCS1dTest ........................ Passed 0.01 sec Start 21: BCS2dTest 18/31 Test #21: BCS2dTest ........................ Passed 0.01 sec Start 22: LowRankRegrTest 19/31 Test #22: LowRankRegrTest .................. Passed 0.01 sec Start 23: PyModTest 20/31 Test #17: PCEImplTest ...................... Passed 0.07 sec Start 24: PyArrayTest 21/31 Test #23: PyModTest ........................ Passed 0.08 sec Start 25: PyArrayTest2 22/31 Test #25: PyArrayTest2 ..................... Passed 0.30 sec Start 26: PyQuadTest 23/31 Test #24: PyArrayTest ...................... Passed 1.44 sec Start 27: PyBCSTest1D 24/31 Test #26: PyQuadTest ....................... Passed 1.68 sec Start 28: PyBCSTest2D 25/31 Test #27: PyBCSTest1D ...................... Passed 1.66 sec Start 29: PyBADPTest 26/31 Test #7: CorrTest ......................... Passed 3.43 sec Start 30: PyRegressionTest 27/31 Test #28: PyBCSTest2D ...................... Passed 1.50 sec Start 31: PyGalerkinTest 28/31 Test #9: MCMC2dTest ....................... Passed 3.90 sec 29/31 Test #29: PyBADPTest ....................... Passed 1.66 sec 30/31 Test #30: PyRegressionTest ................. Passed 1.72 sec 31/31 Test #31: PyGalerkinTest ................... Passed 1.63 sec 100% tests passed, 0 tests failed out of 31 Total Test time (real) = 5.35 sec ==> [2023-04-19-14:56:30.382797] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'check' ==> [2023-04-19-14:56:30.385983] Target 'check' not found in Makefile ```

adamjstewart

Python bits look good, will let someone else review the rest

tldahlgren

Confirmed the two sha256 and that 'spack spec' of the latest version succeeds.

spackbot-app bot added dependencies extends new-version update-package labels Apr 5, 2023

omsai force-pushed the packages/uqtk-3.1.3 branch from 103057e to 4b83d2e Compare April 6, 2023 13:53

tldahlgren reviewed Apr 6, 2023

View reviewed changes

tldahlgren self-assigned this Apr 6, 2023

adamjstewart reviewed Apr 6, 2023

View reviewed changes

tldahlgren assigned adamjstewart Apr 6, 2023

omsai force-pushed the packages/uqtk-3.1.3 branch from 0ff1316 to f683179 Compare April 20, 2023 14:15

adamjstewart approved these changes Apr 20, 2023

View reviewed changes

tldahlgren approved these changes Apr 21, 2023

View reviewed changes

tldahlgren merged commit 0b606b0 into spack:develop Apr 21, 2023

adamjstewart mentioned this pull request Apr 22, 2023

Installation issue: py-numpy #37135

Closed

4 tasks

Conversation

omsai commented Apr 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spackbot-app bot commented Apr 5, 2023

Uh oh!

omsai commented Apr 6, 2023

Uh oh!

tldahlgren commented Apr 6, 2023

Uh oh!

tldahlgren commented Apr 6, 2023

Uh oh!

tldahlgren left a comment

Choose a reason for hiding this comment

Uh oh!

tldahlgren commented Apr 6, 2023

Uh oh!

bjdebus commented Apr 6, 2023

Uh oh!

adamjstewart Apr 6, 2023

Choose a reason for hiding this comment

Uh oh!

bjdebus Apr 6, 2023

Choose a reason for hiding this comment

Uh oh!

omsai Apr 10, 2023

Choose a reason for hiding this comment

Uh oh!

omsai Apr 19, 2023

Choose a reason for hiding this comment

Uh oh!

adamjstewart Apr 19, 2023

Choose a reason for hiding this comment

Uh oh!

omsai Apr 20, 2023

Choose a reason for hiding this comment

Uh oh!

adamjstewart left a comment

Choose a reason for hiding this comment

Uh oh!

tldahlgren left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

omsai commented Apr 5, 2023 •

edited

Loading