Can you post some log/artifact from a test run to show that the tests are running? I forget if it's running "make test" or what, if that's the case then there should be some output in install-time-test-log.txt

@AlexanderRichert-NOAA for your request, it runs make check.

Since there's two versions of ESMF that are being installed from the workflows (8.6.1, and 8.8.0) it is run 4 times. Attached are the four results.
install-time-test-log-gcc-esmf-8.6.1.txt
install-time-test-log-gcc-esmf-8.8.0.txt
install-time-test-log-oneapi-esmf-8.6.1.txt
install-time-test-log-oneapi-esmf-8.8.0.txt

I could be wrong, but I think the 'installcheck' target is different from 'test', the latter which is the one that actually runs the battery of unit tests (I think 'installcheck' is just compiling a small executable to make sure various installed files can be found based on ESMFMKFILE). I've had issues with some other packages where 'installcheck' output was overwriting 'test' output in install-time-test-log.txt, I don't know if that's what's happening here. In any case, it would be helpful to have some indication that a) it's running the unit tests and b) a failed unit test would cause 'spack install' to fail.

I'll give a try and run things manually to see if it's actually running test and see if it can generate some kind of testing log either via ctest or other.

You might keep an eye on install-time-test-log.txt while it's building-- if initially you see unit test output then it's probably getting overwritten during installcheck() (in which case we can tweak the ESMF recipe as a workaround).

I manually ran the install step via cli and I see this in the terminal:

...
The stdout files for the system_tests can be found at:
/tmp/spack-stack/cache/build_stage/ubuntu/spack-stage-esmf-8.8.0-mfk6i5sxkojoxmb6xyntbsp2rzju7xpl/spack-src/test/testO/Linux.gfortran.64.openmpi.default


Found 51 multi-processor system tests, 51 passed and 0 failed.

make[2]: Leaving directory '/tmp/spack-stack/cache/build_stage/ubuntu/spack-stage-esmf-8.8.0-mfk6i5sxkojoxmb6xyntbsp2rzju7xpl/spack-src'


SYSTEM TESTS SUMMARY
Found 51 multi-processor system tests, 51 passed and 0 failed.



UNIT TESTS SUMMARY
Found 8666 non-exhaustive multi-processor unit tests, 8666 passed and 0 failed.
...

So I think the installing with tests is doing a couple of things:

1. Running the ESMF tests and unit tests
2. Doing an install check via spack

Yeah, that makes sense... Does any of that output get preserved anywhere under .spack/? We should have some sort of proof of unit tests running with each installation.

Sadly, searching and the install-and-test-log.txt from the .spack does not show this, and I only got it by running it with the verbose flag.

Is that something we'd like to add to the 04_SpackInstall.sh? I think at that point we will have the info we want, but it'll definitely increase log sizes tremendously.

That's interesting, I guess I wouldn't have expected it to be in the verbose output based on my past experience... Does it show up in spack-build-out.txt? 🤞
If not then I'm open to ignoring it for now (especially if we can at least establish that the 'spack install' command will fail if one of the unit tests fails), and wait for a fix for spack/spack#49200.

Yes, the spack-build-out.txt.gz does contain this information.

Awesome