spack ci: add support for running stand-alone tests by tldahlgren · Pull Request #27877 · spack/spack

tldahlgren · 2021-12-09T02:42:53Z

The goal of this work is to run stand-alone tests from spack ci when the --tests option is used. When the option is used, the stand-alone tests are run after a successful (re)build of the package. Test results are collected and report(able) using CDash.

This PR adds the following features:

Adds -t and --tests to spack ci rebuild to run stand-alone tests;
Adds --fail-fast to stop stand-alone tests after the first failure;
Ensures a single InstallError across packages (i.e., removes second class from build environment);
Captures skipping tests for externals and uninstalled packages (for CDash reporting);
Copies test logs and outputs to the CI artifacts directory to facilitate debugging;
Parses stand-alone test results to report outputs from each run_test as separate test parts (CDash reporting);
Logs a test completion message to allow capture of timing of the last run_test part;
Adds the runner description to the CDash site to better distinguish entries in CDash tables;
Adds gitlab-ci broken-tests-packages to CI configuration to skip stand-alone testing for packages with known issues;
Changes spack ci --help so description of each subcommand is a single, concise line;
Changes spack ci <subcommand> --help to provide the full description of each command (versus no description); and
Ensures junit test log file ends in an .xml extension (versus default where it does not).

Update (2022 Mar 28): Handling of concretization failures that do not get reported to CDash is expected to be addressed in a subsequent issue/PR.

Update (2022 Apr 7): spack ci rebuild --tests should not be be trying to run tests for job spec dependencies, only the job spec. If the build jobs of dependency specs fail to upload to the build cache (e.g., configuration, permissions), then those jobs should be failing. (TBD: Should this PR ensure that check or is it better to have a separate PR?)

Tasks:

Include the equivalent of the architecture information, or at least the host target, in the CDash output
Upload stand-alone test results files as test artifacts
Confirm tests are run in GitLab
Ensure CDash results are uploaded as artifacts
Resolve issues with CDash build-and test results appearing on same row of the table
Add unit tests as needed
Investigate why some (dependency) packages don't have test results (e.g., related from other pipelines)
~~- [ ] Possibly add full hash in build name -- Scott Wittenburg is working on a significant change in this area ATM~~
Ensure proper parsing and reporting of skipped tests (as not run) .. post- Testing: Optionally run tests on externally installed packages #28701 merge
~~- [ ] Determine why concretization failure jobs and associated reports -- actually just Update.xml -- are not showing up in CDash~~
Restore the proper CDash URLand or mirror ONCE out-of-band testing completed

tldahlgren · 2022-01-27T04:31:10Z

@zackgalbreath @tgamblin @becker33

(Update: Adding VendorString attribute to the Site element per an off-line discussion.)
(Update: Replaced generated output for the same package as of commit XYZ.)

Commit ec03ceb generated the output below from a run of spack test run --log-format cdash libxml2.

~~I would prefer to be able to output the architecture as well but I'm not sure how to best represent it given the XSD we've been using.~~

<?xml version="1.0" encoding="UTF-8"?>
<Site BuildName="libxml2-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q"
      BuildStamp="20220128-1706-Experimental"
      Name="ruby961"
      Generator="spack-0.17.1-1009-ec03cebdcd"
      Hostname="ruby961"
      OSName="Linux"
      OSRelease="3.10.0-1160.45.1.1chaos.ch6.x86_64"
      VendorString="cascadelake"
>
  <Testing>
    <StartTime>1643418415</StartTime>
    <Test Status="passed">
      <Name>test:_Performing_simple_import_test</Name>
      <FullCommand>unknown</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.000228</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: Performing simple import test</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Completed</Value>
        </NamedMeasurement>
      </Results>
    </Test>
    <Test Status="passed">
      <Name>xml2-config_--version</Name>
      <FullCommand>'/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xml2-config' '--version'</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.00592</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: xml2-config: expect command status in [0]</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Completed</Value>
        </NamedMeasurement>
        <Measurement>
          <Value>2.9.12
PASSED</Value>
        </Measurement>
      </Results>
    </Test>
    <Test Status="passed">
      <Name>xmllint_--auto_-o_test.xml</Name>
      <FullCommand>'/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--auto' '-o' 'test.xml'</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.012523</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: xmllint: expect command status in [0]</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Completed</Value>
        </NamedMeasurement>
        <Measurement>
          <Value>PASSED</Value>
        </Measurement>
      </Results>
    </Test>
    <Test Status="passed">
      <Name>xmllint_--postvalid_test.xml</Name>
      <FullCommand>'/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--postvalid' 'test.xml'</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.010513</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: xmllint: expect command status in [3]</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Expected to fail</Value>
        </NamedMeasurement>
        <Measurement>
          <Value>&lt;?xml version="1.0"?&gt;
&lt;info&gt;abc&lt;/info&gt;
validity error : no DTD found!
Document test.xml does not validate
PASSED</Value>
        </Measurement>
      </Results>
    </Test>
    <Test Status="passed">
      <Name>xmllint_--dtdvalid_info.dtd_test.xml</Name>
      <FullCommand>'/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--dtdvalid' '/g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.dtd' 'test.xml'</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.009787</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: xmllint: expect command status in [3]</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Expected to fail</Value>
        </NamedMeasurement>
        <Measurement>
          <Value>&lt;?xml version="1.0"?&gt;
&lt;info&gt;abc&lt;/info&gt;
test.xml:2: element info: validity error : Element info content does not follow the DTD, expecting (data), got (CDATA)
Document test.xml does not validate against /g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.dtd
PASSED</Value>
        </Measurement>
      </Results>
    </Test>
    <Test Status="passed">
      <Name>xmllint_--dtdvalid_info.dtd_info.xml</Name>
      <FullCommand>'/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--dtdvalid' '/g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.dtd' '/g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.xml'</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.010327</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: xmllint: expect command status in [0]</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Completed</Value>
        </NamedMeasurement>
        <Measurement>
          <Value>&lt;?xml version="1.0"?&gt;
&lt;info&gt;
&lt;data&gt;abc&lt;/data&gt;
&lt;/info&gt;
PASSED</Value>
        </Measurement>
      </Results>
    </Test>
    <Test Status="passed">
      <Name>xmlcatalog_--create</Name>
      <FullCommand>'/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmlcatalog' '--create'</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.0</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Description">
          <Value>test: xmlcatalog: expect command status in [0]</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>Completed</Value>
        </NamedMeasurement>
        <Measurement>
          <Value>&lt;?xml version="1.0"?&gt;
&lt;!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"&gt;
&lt;catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/&gt;
PASSED</Value>
        </Measurement>
      </Results>
    </Test>
    <EndTestTime>1643418415</EndTestTime>
  </Testing>

This corresponds to the spack test run output text file:

==> Testing package libxml2-2.9.12-pr2t5oa
==> [2022-01-28-17:07:07.892801] test: Performing simple import test
==> [2022-01-28-17:07:07.894672] test: xml2-config: expect command status in [0]
==> [2022-01-28-17:07:07.894900] '/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xml2-config' '--version'
2.9.12
PASSED
==> [2022-01-28-17:07:07.900820] test: xmllint: expect command status in [0]
==> [2022-01-28-17:07:07.901089] '/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--auto' '-o' 'test.xml'
PASSED
==> [2022-01-28-17:07:07.913612] test: xmllint: expect command status in [3]
==> [2022-01-28-17:07:07.913883] '/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--postvalid' 'test.xml'
<?xml version="1.0"?>
<info>abc</info>
validity error : no DTD found!
Document test.xml does not validate
PASSED
==> [2022-01-28-17:07:07.924396] test: xmllint: expect command status in [3]
==> [2022-01-28-17:07:07.924671] '/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--dtdvalid' '/g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.dtd' 'test.xml'
<?xml version="1.0"?>
<info>abc</info>
test.xml:2: element info: validity error : Element info content does not follow the DTD, expecting (data), got (CDATA)
Document test.xml does not validate against /g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.dtd
PASSED
==> [2022-01-28-17:07:07.934458] test: xmllint: expect command status in [0]
==> [2022-01-28-17:07:07.934732] '/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmllint' '--dtdvalid' '/g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.dtd' '/g/g21/dahlgren/.spack/test/2wwlsgepwo4ygldrjc4doero3qzyuncu/libxml2-2.9.12-pr2t5oa/data/libxml2/info.xml'
<?xml version="1.0"?>
<info>
<data>abc</data>
</info>
PASSED
==> [2022-01-28-17:07:07.945059] test: xmlcatalog: expect command status in [0]
==> [2022-01-28-17:07:07.945330] '/usr/WS1/dahlgren/spack/opt/spack/linux-rhel7-cascadelake/intel-19.0.4.227/libxml2-2.9.12-pr2t5oa33wfrxb3mwr5xcl2gz4cawn3q/bin/xmlcatalog' '--create'
<?xml version="1.0"?>
<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/>
PASSED

lib/spack/spack/reporters/extract.py

tldahlgren · 2022-01-29T01:12:50Z

Since we want to know if a package does not have tests to run, I created a dummy test part so we can flag/track such packages (starting with commit 2c1ac7c). What do others think of reporting packages in this manner?

An example output from running at the command line for the blt package is given below.

<?xml version="1.0" encoding="UTF-8"?>
<Site BuildName="blt-m6hnxcd346nyhmzwagcmyfzv35piomjy"
      BuildStamp="20220128-1706-Experimental"
      Name="ruby961"
      Generator="spack-0.17.1-1009-ec03cebdcd"
      Hostname="ruby961"
      OSName="Linux"
      OSRelease="3.10.0-1160.45.1.1chaos.ch6.x86_64"
      VendorString="cascadelake"
>
  <Testing>
    <StartTime>1643418411</StartTime>
    <Test Status="notrun">
      <Name>blt</Name>
      <FullCommand>Not Applicable</FullCommand>
      <Results>
        <NamedMeasurement type="numeric/double" name="Execution Time">
          <Value>0.0</Value>
        </NamedMeasurement>
        <NamedMeasurement type="text/string" name="Completion Status">
          <Value>No tests to run</Value>
        </NamedMeasurement>
      </Results>
    </Test>
    <EndTestTime>1643418411</EndTestTime>
  </Testing>

Is there anything missing or you would want changed?

tldahlgren · 2022-02-17T03:08:12Z

Interesting test failures in what appear to be a section untouched by this PR (Updated example below):

_______________________________ test_config_edit _______________________________
[2540](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2540)

[2541](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2541)
    def test_config_edit():
[2542](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2542)
        """Ensure `spack config edit` edits the right paths."""
[2543](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2543)
    
[2544](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2544)
        dms = spack.config.default_modify_scope('compilers')
[2545](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2545)
        dms_path = spack.config.config.scopes[dms].path
[2546](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2546)
        user_path = spack.config.config.scopes['user'].path
[2547](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2547)
    
[2548](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2548)
        comp_path = os.path.join(dms_path, 'compilers.yaml')
[2549](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2549)
        repos_path = os.path.join(user_path, 'repos.yaml')
[2550](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2550)
    
[2551](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2551)
>       assert config('edit', '--print-file', 'compilers').strip() == comp_path
[2552](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2552)
E       AssertionError: assert '/tmp/pytest-...st/spack.yaml' == '/tmp/pytest-o...ompilers.yaml'
[2553](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2553)
E         - /tmp/pytest-of-spack-test/pytest-0/mock-env-path87/test/spack.yaml
[2554](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2554)
E         + /tmp/pytest-of-spack-test/pytest-0/mock-env-path87/test/spack.yaml/compilers.yaml
[2555](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2555)
E         ?                                                                   +++++++++++++++
[2556](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2556)

[2557](https://github.com/spack/spack/runs/5236546178?check_suite_focus=true#step:6:2557)
lib/spack/spack/test/cmd/config.py:100: AssertionError
FAILED lib/spack/spack/test/cmd/config.py::test_config_edit - AssertionError:...

Note the error seems to be related to a file path being treated as a directory:

>       assert config('edit', '--print-file', 'compilers').strip() == comp_path
E       AssertionError: assert '/tmp/pytest-...st/spack.yaml' == '/tmp/pytest-o...ompilers.yaml')
E         - /tmp/pytest-of-spack-test/pytest-0/mock-env-path87/test/spack.yaml
E         + /tmp/pytest-of-spack-test/pytest-0/mock-env-path87/test/spack.yaml/compilers.yaml
E         ?                                                                   +++++++++++++++

tldahlgren · 2022-08-04T23:41:28Z

@spackbot style --fix

That looks like the appropriate spack command, but to get spackbot to respond, you have to do it like this.

So I should've gone with my first instinct instead of what I had in my notes. 😜

spackbot-app · 2022-08-04T23:44:07Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/build_environment.py
  lib/spack/spack/ci.py
  lib/spack/spack/cmd/__init__.py
  lib/spack/spack/cmd/ci.py
  lib/spack/spack/cmd/install.py
  lib/spack/spack/cmd/test.py
  lib/spack/spack/environment/environment.py
  lib/spack/spack/install_test.py
  lib/spack/spack/installer.py
  lib/spack/spack/package_base.py
  lib/spack/spack/relocate.py
  lib/spack/spack/report.py
  lib/spack/spack/reporters/cdash.py
  lib/spack/spack/reporters/extract.py
  lib/spack/spack/reporters/junit.py
  lib/spack/spack/schema/gitlab_ci.py
  lib/spack/spack/test/ci.py
  lib/spack/spack/test/cmd/ci.py
  lib/spack/spack/test/cmd/config.py
  lib/spack/spack/test/cmd/install.py
  lib/spack/spack/test/cmd/test.py
  lib/spack/spack/test/reporters.py
  lib/spack/spack/test/test_suite.py
  lib/spack/spack/test/util/path.py
  lib/spack/spack/util/path.py
==> Bootstrapping py-black from pre-built binaries
==> Bootstrapping py-black@:21 %gcc target=x86_64 from sources
==> Bootstrapping clingo from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.2/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-dtwevigmwgke4g6ee5byktpmzmrp2kvx.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.2/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstrap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-dtwevigmwgke4g6ee5byktpmzmrp2kvx.spack
==> Installing "clingo-bootstrap@spack%[email protected]~docs~ipo+python+static_libstdcpp build_type=Release arch=linux-centos7-x86_64" from a buildcache
==> Bootstrapping patchelf from pre-built binaries
==> Bootstrapping [email protected]:0.13.99 %gcc target=x86_64 from sources
==> Waiting for patchelf-0.13.1-lfwjnfm4uicf4quipjouw3kj6huqqrve
�[1F�[J==> Installing patchelf-0.13.1-lfwjnfm4uicf4quipjouw3kj6huqqrve
==> No binary for patchelf-0.13.1-lfwjnfm4uicf4quipjouw3kj6huqqrve found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/08/08c0237e89be74d61ddf8f6ff218439cdd62af572d568fb38913b53e222831de.tar.gz
==> No patches needed for patchelf
==> patchelf: Executing phase: 'autoreconf'
==> patchelf: Executing phase: 'configure'
==> patchelf: Executing phase: 'build'
==> patchelf: Executing phase: 'install'
==> patchelf: Successfully installed patchelf-0.13.1-lfwjnfm4uicf4quipjouw3kj6huqqrve
  Fetch: 0.04s.  Build: 8.88s.  Total: 8.92s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/patchelf-0.13.1-lfwjnfm4uicf4quipjouw3kj6huqqrve
[+] /usr/local (external python-3.7-xf6fhfbk7y4or2ghkaw3remwp2cyhhay)
==> Waiting for py-pip-22.1.2-k5outm3hsfhplnxfztto5p7xsyfn5mfk
�[1F�[J==> Installing py-pip-22.1.2-k5outm3hsfhplnxfztto5p7xsyfn5mfk
==> No binary for py-pip-22.1.2-k5outm3hsfhplnxfztto5p7xsyfn5mfk found: installing from source
==> Fetching https://files.pythonhosted.org/packages/py3/p/pip/pip-22.1.2-py3-none-any.whl
==> No patches needed for py-pip
==> py-pip: Executing phase: 'install'
==> py-pip: Successfully installed py-pip-22.1.2-k5outm3hsfhplnxfztto5p7xsyfn5mfk
  Fetch: 0.72s.  Build: 3.31s.  Total: 4.03s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-pip-22.1.2-k5outm3hsfhplnxfztto5p7xsyfn5mfk
==> Waiting for py-wheel-0.37.1-iyk25i7t4z7mxovk6hbkkz3z6oevsirt
�[1F�[J==> Installing py-wheel-0.37.1-iyk25i7t4z7mxovk6hbkkz3z6oevsirt
==> No binary for py-wheel-0.37.1-iyk25i7t4z7mxovk6hbkkz3z6oevsirt found: installing from source
==> Fetching https://files.pythonhosted.org/packages/py2.py3/w/wheel/wheel-0.37.1-py2.py3-none-any.whl
==> No patches needed for py-wheel
==> py-wheel: Executing phase: 'install'
==> py-wheel: Successfully installed py-wheel-0.37.1-iyk25i7t4z7mxovk6hbkkz3z6oevsirt
  Fetch: 0.67s.  Build: 0.66s.  Total: 1.32s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-wheel-0.37.1-iyk25i7t4z7mxovk6hbkkz3z6oevsirt
==> Waiting for py-setuptools-63.0.0-f4bjue7wk54mnigds4wn5pde2qzth3qe
�[1F�[J==> Installing py-setuptools-63.0.0-f4bjue7wk54mnigds4wn5pde2qzth3qe
==> No binary for py-setuptools-63.0.0-f4bjue7wk54mnigds4wn5pde2qzth3qe found: installing from source
==> Fetching https://files.pythonhosted.org/packages/py3/s/setuptools/setuptools-63.0.0-py3-none-any.whl
==> No patches needed for py-setuptools
==> py-setuptools: Executing phase: 'install'
==> py-setuptools: Successfully installed py-setuptools-63.0.0-f4bjue7wk54mnigds4wn5pde2qzth3qe
  Fetch: 0.66s.  Build: 1.32s.  Total: 1.98s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-setuptools-63.0.0-f4bjue7wk54mnigds4wn5pde2qzth3qe
==> Waiting for py-tomli-1.2.2-6cg5xhj7tukj6hzvu7ymt7hq7uxogzbx
�[1F�[J==> Installing py-tomli-1.2.2-6cg5xhj7tukj6hzvu7ymt7hq7uxogzbx
==> No binary for py-tomli-1.2.2-6cg5xhj7tukj6hzvu7ymt7hq7uxogzbx found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/f0/f04066f68f5554911363063a30b108d2b5a5b1a010aa8b6132af78489fe3aade
==> No patches needed for py-tomli
==> py-tomli: Executing phase: 'install'
==> py-tomli: Successfully installed py-tomli-1.2.2-6cg5xhj7tukj6hzvu7ymt7hq7uxogzbx
  Fetch: 0.04s.  Build: 0.64s.  Total: 0.68s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-tomli-1.2.2-6cg5xhj7tukj6hzvu7ymt7hq7uxogzbx
==> Waiting for py-flit-core-3.6.0-7m5wgiknumtuzu4jwqfvk53vzyxmf55e
�[1F�[J==> Installing py-flit-core-3.6.0-7m5wgiknumtuzu4jwqfvk53vzyxmf55e
==> No binary for py-flit-core-3.6.0-7m5wgiknumtuzu4jwqfvk53vzyxmf55e found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/58/5892962ab8b8ea945835b3a288fe9dd69316f1903d5288c3f5cafdcdd04756ad.tar.gz
==> No patches needed for py-flit-core
==> py-flit-core: Executing phase: 'install'
==> py-flit-core: Successfully installed py-flit-core-3.6.0-7m5wgiknumtuzu4jwqfvk53vzyxmf55e
  Fetch: 0.06s.  Build: 0.91s.  Total: 0.97s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-flit-core-3.6.0-7m5wgiknumtuzu4jwqfvk53vzyxmf55e
==> Waiting for py-typed-ast-1.4.3-7zhfzivbg6leml2gs76u4n5542s2ziy6
�[1F�[J==> Installing py-typed-ast-1.4.3-7zhfzivbg6leml2gs76u4n5542s2ziy6
==> No binary for py-typed-ast-1.4.3-7zhfzivbg6leml2gs76u4n5542s2ziy6 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/fb/fb1bbeac803adea29cedd70781399c99138358c26d05fcbd23c13016b7f5ec65.tar.gz
==> No patches needed for py-typed-ast
==> py-typed-ast: Executing phase: 'install'
==> py-typed-ast: Successfully installed py-typed-ast-1.4.3-7zhfzivbg6leml2gs76u4n5542s2ziy6
  Fetch: 0.05s.  Build: 14.34s.  Total: 14.39s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-typed-ast-1.4.3-7zhfzivbg6leml2gs76u4n5542s2ziy6
==> Waiting for py-mypy-extensions-0.4.3-m2ljne7zbs6jcqlrnqdyxssv6z2byleq
�[1F�[J==> Installing py-mypy-extensions-0.4.3-m2ljne7zbs6jcqlrnqdyxssv6z2byleq
==> No binary for py-mypy-extensions-0.4.3-m2ljne7zbs6jcqlrnqdyxssv6z2byleq found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/2d/2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8.tar.gz
==> No patches needed for py-mypy-extensions
==> py-mypy-extensions: Executing phase: 'install'
==> py-mypy-extensions: Successfully installed py-mypy-extensions-0.4.3-m2ljne7zbs6jcqlrnqdyxssv6z2byleq
  Fetch: 0.04s.  Build: 1.84s.  Total: 1.88s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-mypy-extensions-0.4.3-m2ljne7zbs6jcqlrnqdyxssv6z2byleq
==> Waiting for py-pyparsing-3.0.6-el3auh2snyditdnpvf5haexklkcrtfpg
�[1F�[J==> Installing py-pyparsing-3.0.6-el3auh2snyditdnpvf5haexklkcrtfpg
==> No binary for py-pyparsing-3.0.6-el3auh2snyditdnpvf5haexklkcrtfpg found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/d9/d9bdec0013ef1eb5a84ab39a3b3868911598afa494f5faa038647101504e2b81.tar.gz
==> No patches needed for py-pyparsing
==> py-pyparsing: Executing phase: 'install'
==> py-pyparsing: Successfully installed py-pyparsing-3.0.6-el3auh2snyditdnpvf5haexklkcrtfpg
  Fetch: 0.10s.  Build: 1.93s.  Total: 2.03s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-pyparsing-3.0.6-el3auh2snyditdnpvf5haexklkcrtfpg
==> Waiting for py-pathspec-0.9.0-mi3tri4otven5swebmiqkj26s7g2uaai
�[1F�[J==> Installing py-pathspec-0.9.0-mi3tri4otven5swebmiqkj26s7g2uaai
==> No binary for py-pathspec-0.9.0-mi3tri4otven5swebmiqkj26s7g2uaai found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/e5/e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1.tar.gz
==> No patches needed for py-pathspec
==> py-pathspec: Executing phase: 'install'
==> py-pathspec: Successfully installed py-pathspec-0.9.0-mi3tri4otven5swebmiqkj26s7g2uaai
  Fetch: 0.06s.  Build: 1.85s.  Total: 1.92s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-pathspec-0.9.0-mi3tri4otven5swebmiqkj26s7g2uaai
==> Waiting for py-typing-extensions-4.3.0-wq4kdytjxiqztkg57sctbssdye3tcziu
�[1F�[J==> Installing py-typing-extensions-4.3.0-wq4kdytjxiqztkg57sctbssdye3tcziu
==> No binary for py-typing-extensions-4.3.0-wq4kdytjxiqztkg57sctbssdye3tcziu found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/t/typing_extensions/typing_extensions-4.3.0.tar.gz
==> No patches needed for py-typing-extensions
==> py-typing-extensions: Executing phase: 'install'
==> py-typing-extensions: Successfully installed py-typing-extensions-4.3.0-wq4kdytjxiqztkg57sctbssdye3tcziu
  Fetch: 0.70s.  Build: 1.47s.  Total: 2.17s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-typing-extensions-4.3.0-wq4kdytjxiqztkg57sctbssdye3tcziu
==> Waiting for py-packaging-21.3-u3r2vov2sjes4tma3rcnmibvealiselb
�[1F�[J==> Installing py-packaging-21.3-u3r2vov2sjes4tma3rcnmibvealiselb
==> No binary for py-packaging-21.3-u3r2vov2sjes4tma3rcnmibvealiselb found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/dd/dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb.tar.gz
==> No patches needed for py-packaging
==> py-packaging: Executing phase: 'install'
==> py-packaging: Successfully installed py-packaging-21.3-u3r2vov2sjes4tma3rcnmibvealiselb
  Fetch: 0.07s.  Build: 1.71s.  Total: 1.78s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-packaging-21.3-u3r2vov2sjes4tma3rcnmibvealiselb
==> Waiting for py-setuptools-scm-6.3.2-t3qr4663ghwjfhyrgmp4id2fpuiakwcg
�[1F�[J==> Installing py-setuptools-scm-6.3.2-t3qr4663ghwjfhyrgmp4id2fpuiakwcg
==> No binary for py-setuptools-scm-6.3.2-t3qr4663ghwjfhyrgmp4id2fpuiakwcg found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/a4/a49aa8081eeb3514eb9728fa5040f2eaa962d6c6f4ec9c32f6c1fba88f88a0f2.tar.gz
==> No patches needed for py-setuptools-scm
==> py-setuptools-scm: Executing phase: 'install'
==> py-setuptools-scm: Successfully installed py-setuptools-scm-6.3.2-t3qr4663ghwjfhyrgmp4id2fpuiakwcg
  Fetch: 0.06s.  Build: 1.60s.  Total: 1.67s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-setuptools-scm-6.3.2-t3qr4663ghwjfhyrgmp4id2fpuiakwcg
==> Waiting for py-platformdirs-2.4.1-ond7orp24xtp77tvxuezuc4nkb657gj5
�[1F�[J==> Installing py-platformdirs-2.4.1-ond7orp24xtp77tvxuezuc4nkb657gj5
==> No binary for py-platformdirs-2.4.1-ond7orp24xtp77tvxuezuc4nkb657gj5 found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/p/platformdirs/platformdirs-2.4.1.tar.gz
==> No patches needed for py-platformdirs
==> py-platformdirs: Executing phase: 'install'
==> py-platformdirs: Successfully installed py-platformdirs-2.4.1-ond7orp24xtp77tvxuezuc4nkb657gj5
  Fetch: 0.67s.  Build: 1.99s.  Total: 2.66s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-platformdirs-2.4.1-ond7orp24xtp77tvxuezuc4nkb657gj5
==> Waiting for py-zipp-3.6.0-fo2kfs7faaeis42bnatslc4uhzlge66u
�[1F�[J==> Installing py-zipp-3.6.0-fo2kfs7faaeis42bnatslc4uhzlge66u
==> No binary for py-zipp-3.6.0-fo2kfs7faaeis42bnatslc4uhzlge66u found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/71/71c644c5369f4a6e07636f0aa966270449561fcea2e3d6747b8d23efaa9d7832.tar.gz
==> No patches needed for py-zipp
==> py-zipp: Executing phase: 'install'
==> py-zipp: Successfully installed py-zipp-3.6.0-fo2kfs7faaeis42bnatslc4uhzlge66u
  Fetch: 0.05s.  Build: 1.79s.  Total: 1.84s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-zipp-3.6.0-fo2kfs7faaeis42bnatslc4uhzlge66u
==> Waiting for py-importlib-metadata-4.11.1-cmx5z2bxp3dvtbroifktoiceec2mqinu
�[1F�[J==> Installing py-importlib-metadata-4.11.1-cmx5z2bxp3dvtbroifktoiceec2mqinu
==> No binary for py-importlib-metadata-4.11.1-cmx5z2bxp3dvtbroifktoiceec2mqinu found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/17/175f4ee440a0317f6e8d81b7f8d4869f93316170a65ad2b007d2929186c8052c.tar.gz
==> No patches needed for py-importlib-metadata
==> py-importlib-metadata: Executing phase: 'install'
==> py-importlib-metadata: Successfully installed py-importlib-metadata-4.11.1-cmx5z2bxp3dvtbroifktoiceec2mqinu
  Fetch: 0.04s.  Build: 1.62s.  Total: 1.67s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-importlib-metadata-4.11.1-cmx5z2bxp3dvtbroifktoiceec2mqinu
==> Waiting for py-click-8.0.3-m6vrp6cow4qw4wmmgx2ci7xcglcecqlv
�[1F�[J==> Installing py-click-8.0.3-m6vrp6cow4qw4wmmgx2ci7xcglcecqlv
==> No binary for py-click-8.0.3-m6vrp6cow4qw4wmmgx2ci7xcglcecqlv found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/41/410e932b050f5eed773c4cda94de75971c89cdb3155a72a0831139a79e5ecb5b.tar.gz
==> No patches needed for py-click
==> py-click: Executing phase: 'install'
==> py-click: Successfully installed py-click-8.0.3-m6vrp6cow4qw4wmmgx2ci7xcglcecqlv
  Fetch: 0.05s.  Build: 1.80s.  Total: 1.85s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-click-8.0.3-m6vrp6cow4qw4wmmgx2ci7xcglcecqlv
==> Waiting for py-black-21.12b0-vuekotdgktt6qwecccj56nr36tk5yniu
�[1F�[J==> Installing py-black-21.12b0-vuekotdgktt6qwecccj56nr36tk5yniu
==> No binary for py-black-21.12b0-vuekotdgktt6qwecccj56nr36tk5yniu found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/b/black/black-21.12b0.tar.gz
==> No patches needed for py-black
==> py-black: Executing phase: 'install'
==> py-black: Successfully installed py-black-21.12b0-vuekotdgktt6qwecccj56nr36tk5yniu
  Fetch: 0.71s.  Build: 1.99s.  Total: 2.70s.
[+] /root/.spack/bootstrap/store/linux-debian11-x86_64/gcc-10.2.1/py-black-21.12b0-vuekotdgktt6qwecccj56nr36tk5yniu
==> Running isort checks
  isort checks were clean
==> Running mypy checks
lib/spack/spack/operating_systems/windows_os.py:40: error: No overload variant of "check_output" matches argument types "List[str]", "Dict[str, str]"  [call-overload]
lib/spack/spack/operating_systems/windows_os.py:40: note: Possible overload variants:
lib/spack/spack/operating_systems/windows_os.py:40: note:     def check_output(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., stdin: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., env: Optional[Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]]]] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, timeout: Optional[float] = ..., input: Optional[Union[bytes, str]] = ..., encoding: Optional[str] = ..., errors: Optional[str] = ..., text: Literal[True]) -> str
lib/spack/spack/operating_systems/windows_os.py:40: note:     def check_output(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., stdin: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., env: Optional[Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]]]] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, timeout: Optional[float] = ..., input: Optional[Union[bytes, str]] = ..., encoding: str, errors: Optional[str] = ..., text: Optional[bool] = ...) -> str
lib/spack/spack/operating_systems/windows_os.py:40: note:     def check_output(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., stdin: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., env: Optional[Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]]]] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, timeout: Optional[float] = ..., input: Optional[Union[bytes, str]] = ..., encoding: Optional[str] = ..., errors: str, text: Optional[bool] = ...) -> str
lib/spack/spack/operating_systems/windows_os.py:40: note:     def check_output(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., stdin: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., env: Optional[Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]]]] = ..., *, universal_newlines: Literal[True], startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., timeout: Optional[float] = ..., input: Optional[Union[bytes, str]] = ..., encoding: Optional[str] = ..., errors: Optional[str] = ..., text: Optional[bool] = ...) -> str
lib/spack/spack/operating_systems/windows_os.py:40: note:     def check_output(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., stdin: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., env: Optional[Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]]]] = ..., universal_newlines: Literal[False] = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, timeout: Optional[float] = ..., input: Optional[Union[bytes, str]] = ..., encoding: None = ..., errors: None = ..., text: Optional[Literal[False]] = ...) -> bytes
lib/spack/spack/operating_systems/windows_os.py:40: note:     def check_output(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., stdin: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Optional[Union[str, bytes, PathLike[str], PathLike[bytes]]] = ..., env: Optional[Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]]]] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, timeout: Optional[float] = ..., input: Optional[Union[bytes, str]] = ..., encoding: Optional[str] = ..., errors: Optional[str] = ..., text: Optional[bool] = ...) -> Any
Found 1 error in 1 file (checked 557 source files)
  mypy found errors
==> Running black checks
reformatted lib/spack/spack/cmd/ci.py
reformatted lib/spack/spack/cmd/install.py
reformatted lib/spack/spack/ci.py
reformatted lib/spack/spack/install_test.py
reformatted lib/spack/spack/installer.py
reformatted lib/spack/spack/report.py
reformatted lib/spack/spack/package_base.py
reformatted lib/spack/spack/reporters/extract.py
reformatted lib/spack/spack/reporters/junit.py
reformatted lib/spack/spack/schema/gitlab_ci.py
reformatted lib/spack/spack/reporters/cdash.py
reformatted lib/spack/spack/test/ci.py
reformatted lib/spack/spack/test/cmd/ci.py
reformatted lib/spack/spack/test/reporters.py
reformatted lib/spack/spack/test/cmd/install.py
reformatted lib/spack/spack/test/test_suite.py
reformatted lib/spack/spack/test/util/path.py
All done! ✨ 🍰 ✨
17 files reformatted, 9 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

tldahlgren · 2022-08-08T20:32:32Z

@spackbot fix style

spackbot-app · 2022-08-08T20:32:33Z

Let me see if I can fix that for you!

spackbot-app · 2022-08-08T20:34:33Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/build_environment.py
  lib/spack/spack/ci.py
  lib/spack/spack/cmd/__init__.py
  lib/spack/spack/cmd/ci.py
  lib/spack/spack/cmd/install.py
  lib/spack/spack/cmd/test.py
  lib/spack/spack/environment/environment.py
  lib/spack/spack/install_test.py
  lib/spack/spack/installer.py
  lib/spack/spack/package_base.py
  lib/spack/spack/relocate.py
  lib/spack/spack/report.py
  lib/spack/spack/reporters/cdash.py
  lib/spack/spack/reporters/extract.py
  lib/spack/spack/reporters/junit.py
  lib/spack/spack/schema/gitlab_ci.py
  lib/spack/spack/test/ci.py
  lib/spack/spack/test/cmd/ci.py
  lib/spack/spack/test/cmd/config.py
  lib/spack/spack/test/cmd/install.py
  lib/spack/spack/test/cmd/test.py
  lib/spack/spack/test/reporters.py
  lib/spack/spack/test/test_suite.py
  lib/spack/spack/test/util/path.py
  lib/spack/spack/util/path.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 557 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/cmd/ci.py
reformatted lib/spack/spack/cmd/install.py
reformatted lib/spack/spack/ci.py
reformatted lib/spack/spack/install_test.py
reformatted lib/spack/spack/installer.py
reformatted lib/spack/spack/package_base.py
reformatted lib/spack/spack/report.py
reformatted lib/spack/spack/reporters/extract.py
reformatted lib/spack/spack/reporters/junit.py
reformatted lib/spack/spack/schema/gitlab_ci.py
reformatted lib/spack/spack/reporters/cdash.py
reformatted lib/spack/spack/test/ci.py
reformatted lib/spack/spack/test/cmd/ci.py
reformatted lib/spack/spack/test/cmd/install.py
reformatted lib/spack/spack/test/test_suite.py
reformatted lib/spack/spack/test/reporters.py
reformatted lib/spack/spack/test/util/path.py
All done! ✨ 🍰 ✨
17 files reformatted, 9 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

tldahlgren · 2022-08-08T21:31:22Z

Moved Reduce CI Noise commit (except 1 line) to #31998 .

tldahlgren · 2022-08-08T23:49:34Z

@spackbot fix style

spackbot-app · 2022-08-08T23:49:36Z

Let me see if I can fix that for you!

spackbot-app · 2022-08-08T23:51:37Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/build_environment.py
  lib/spack/spack/ci.py
  lib/spack/spack/cmd/__init__.py
  lib/spack/spack/cmd/ci.py
  lib/spack/spack/cmd/install.py
  lib/spack/spack/cmd/test.py
  lib/spack/spack/environment/environment.py
  lib/spack/spack/install_test.py
  lib/spack/spack/installer.py
  lib/spack/spack/package_base.py
  lib/spack/spack/relocate.py
  lib/spack/spack/report.py
  lib/spack/spack/reporters/cdash.py
  lib/spack/spack/reporters/extract.py
  lib/spack/spack/reporters/junit.py
  lib/spack/spack/schema/gitlab_ci.py
  lib/spack/spack/test/ci.py
  lib/spack/spack/test/cmd/ci.py
  lib/spack/spack/test/cmd/config.py
  lib/spack/spack/test/cmd/install.py
  lib/spack/spack/test/cmd/test.py
  lib/spack/spack/test/reporters.py
  lib/spack/spack/test/test_suite.py
  lib/spack/spack/test/util/path.py
  lib/spack/spack/util/path.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 557 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/cmd/ci.py
reformatted lib/spack/spack/cmd/install.py
reformatted lib/spack/spack/ci.py
reformatted lib/spack/spack/install_test.py
reformatted lib/spack/spack/installer.py
reformatted lib/spack/spack/report.py
reformatted lib/spack/spack/package_base.py
reformatted lib/spack/spack/reporters/extract.py
reformatted lib/spack/spack/reporters/junit.py
reformatted lib/spack/spack/schema/gitlab_ci.py
reformatted lib/spack/spack/reporters/cdash.py
reformatted lib/spack/spack/test/ci.py
reformatted lib/spack/spack/test/cmd/ci.py
reformatted lib/spack/spack/test/cmd/install.py
reformatted lib/spack/spack/test/reporters.py
reformatted lib/spack/spack/test/test_suite.py
reformatted lib/spack/spack/test/util/path.py
All done! ✨ 🍰 ✨
17 files reformatted, 9 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

tldahlgren · 2022-08-09T01:38:06Z

FYI. Just confirmed the commit to skip stand-alone testing for packages listed in gitlab-ci's broken-tests-packages works for gptune (see https://gitlab.spack.io/spack/spack/-/jobs/2962398) with the warning:

Warning: Unable to run stand-alone tests since listed in gitlab-ci's 'broken-tests-packages'

tldahlgren · 2022-08-09T02:07:04Z

@spackbot fix style

spackbot-app · 2022-08-09T02:07:06Z

Let me see if I can fix that for you!

spackbot-app · 2022-08-09T02:09:07Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/build_environment.py
  lib/spack/spack/ci.py
  lib/spack/spack/cmd/__init__.py
  lib/spack/spack/cmd/ci.py
  lib/spack/spack/cmd/install.py
  lib/spack/spack/cmd/test.py
  lib/spack/spack/environment/environment.py
  lib/spack/spack/install_test.py
  lib/spack/spack/installer.py
  lib/spack/spack/package_base.py
  lib/spack/spack/relocate.py
  lib/spack/spack/report.py
  lib/spack/spack/reporters/cdash.py
  lib/spack/spack/reporters/extract.py
  lib/spack/spack/reporters/junit.py
  lib/spack/spack/schema/gitlab_ci.py
  lib/spack/spack/test/ci.py
  lib/spack/spack/test/cmd/ci.py
  lib/spack/spack/test/cmd/config.py
  lib/spack/spack/test/cmd/install.py
  lib/spack/spack/test/cmd/test.py
  lib/spack/spack/test/reporters.py
  lib/spack/spack/test/test_suite.py
  lib/spack/spack/test/util/path.py
  lib/spack/spack/util/path.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 557 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/cmd/ci.py
reformatted lib/spack/spack/cmd/install.py
reformatted lib/spack/spack/ci.py
reformatted lib/spack/spack/install_test.py
reformatted lib/spack/spack/installer.py
reformatted lib/spack/spack/package_base.py
reformatted lib/spack/spack/report.py
reformatted lib/spack/spack/reporters/extract.py
reformatted lib/spack/spack/reporters/junit.py
reformatted lib/spack/spack/schema/gitlab_ci.py
reformatted lib/spack/spack/reporters/cdash.py
reformatted lib/spack/spack/test/ci.py
reformatted lib/spack/spack/test/cmd/ci.py
reformatted lib/spack/spack/test/cmd/install.py
reformatted lib/spack/spack/test/reporters.py
reformatted lib/spack/spack/test/util/path.py
reformatted lib/spack/spack/test/test_suite.py
All done! ✨ 🍰 ✨
17 files reformatted, 9 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

tldahlgren · 2022-08-09T18:56:48Z

@spackbot fix style

spackbot-app · 2022-08-09T18:56:50Z

Let me see if I can fix that for you!

tldahlgren · 2022-08-09T19:00:49Z

@spackbot fix style

spackbot-app · 2022-08-09T19:00:51Z

Let me see if I can fix that for you!

spackbot-app · 2022-08-09T19:02:51Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/build_environment.py
  lib/spack/spack/ci.py
  lib/spack/spack/cmd/__init__.py
  lib/spack/spack/cmd/ci.py
  lib/spack/spack/cmd/install.py
  lib/spack/spack/cmd/test.py
  lib/spack/spack/environment/environment.py
  lib/spack/spack/install_test.py
  lib/spack/spack/installer.py
  lib/spack/spack/main.py
  lib/spack/spack/package_base.py
  lib/spack/spack/report.py
  lib/spack/spack/reporters/cdash.py
  lib/spack/spack/reporters/extract.py
  lib/spack/spack/reporters/junit.py
  lib/spack/spack/schema/gitlab_ci.py
  lib/spack/spack/test/ci.py
  lib/spack/spack/test/cmd/ci.py
  lib/spack/spack/test/cmd/config.py
  lib/spack/spack/test/cmd/install.py
  lib/spack/spack/test/cmd/test.py
  lib/spack/spack/test/reporters.py
  lib/spack/spack/test/test_suite.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 557 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/cmd/ci.py
reformatted lib/spack/spack/cmd/install.py
reformatted lib/spack/spack/ci.py
reformatted lib/spack/spack/install_test.py
reformatted lib/spack/spack/installer.py
reformatted lib/spack/spack/report.py
reformatted lib/spack/spack/reporters/cdash.py
reformatted lib/spack/spack/reporters/extract.py
reformatted lib/spack/spack/reporters/junit.py
reformatted lib/spack/spack/schema/gitlab_ci.py
reformatted lib/spack/spack/package_base.py
reformatted lib/spack/spack/test/ci.py
reformatted lib/spack/spack/test/cmd/ci.py
reformatted lib/spack/spack/test/cmd/install.py
reformatted lib/spack/spack/test/reporters.py
reformatted lib/spack/spack/test/test_suite.py
All done! ✨ 🍰 ✨
16 files reformatted, 7 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

tldahlgren · 2022-08-09T23:28:20Z

The 2.7, clingo unit tests check failed while installing system packages with:

E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/c/cmake/cmake-data_3.16.3-1ubuntu1_all.deb  Connection failed [IP: 52.154.174.208 80]
[79](https://github.com/spack/spack/runs/7753927130?check_suite_focus=true#step:4:80)
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
[80](https://github.com/spack/spack/runs/7753927130?check_suite_focus=true#step:4:81)
Error: Process completed with exit code 100.

tldahlgren · 2022-08-10T02:03:38Z

Rebased to develop in the hopes that that odd 2.7 clingo check will not encounter system package installation issues this time.

tldahlgren · 2022-08-10T15:48:35Z

Everything is currently passing so I am going to drop the temporary mirror, etc.

This support requires adding the '--tests' option to 'spack ci rebuild'. Packages whose stand-alone tests are broken (in the CI environment) can be configured in gitlab-ci to be skipped by adding them to broken-tests-packages. Highlights include: - Restructured 'spack ci' help to provide better subcommand summaries; - Ensured only one InstallError (i.e., installer's) rather than allowing build_environment to have its own; and - Refactored CI and CDash reporting to keep CDash-related properties and behavior in a separate class.

tldahlgren · 2022-08-10T19:24:39Z

@tgamblin Looks like all checks have passed and this PR is RFR.

tldahlgren · 2022-08-15T16:46:29Z

ping @tgamblin

tldahlgren · 2022-08-22T16:47:11Z

@tgamblin There are a number of new CI-related PRs that, if merged before this one, may result in yet another round of rebasing and fixes for this PR (e.g., #32187, #32290, #32300). Should I split this PR to make it easier to review since it has at least a few sets of changes?

tgamblin

This looks great to me @tldahlgren! Thanks for all the work debugging pipelines to get this running properly.

alalazo · 2022-12-27T21:00:24Z

lib/spack/spack/report.py

        self.cls = cls
        self.function = function
        self.filename = None
+        self.ctest_parsing = getattr(args, "ctest_parsing", False)


@zackgalbreath @tldahlgren Where is args.ctest_parsing defined? I'm currently trying to refactor these classes to make them independent of argparse namespaces and for some reason I can't find references to this argument.

spackbot-app bot added the commands label Dec 9, 2021

tldahlgren force-pushed the spack-ci-stand-alone-testing branch 3 times, most recently from 52cd1d8 to dc273fb Compare December 15, 2021 03:36

spackbot-app bot added the tests General test capability(ies) label Dec 15, 2021

tldahlgren force-pushed the spack-ci-stand-alone-testing branch 2 times, most recently from 3e47927 to 9ef24b3 Compare January 6, 2022 02:46

tldahlgren force-pushed the spack-ci-stand-alone-testing branch from 578b64d to e1df1cf Compare January 14, 2022 18:17

spackbot-app bot added the gitlab Issues related to gitlab integration label Jan 15, 2022

tldahlgren force-pushed the spack-ci-stand-alone-testing branch from 2d88f26 to 817941b Compare January 25, 2022 00:57

tldahlgren force-pushed the spack-ci-stand-alone-testing branch from 4d32484 to c291bcc Compare January 28, 2022 18:05

zackgalbreath reviewed Jan 28, 2022

View reviewed changes

lib/spack/spack/reporters/extract.py Outdated Show resolved Hide resolved

tldahlgren force-pushed the spack-ci-stand-alone-testing branch 2 times, most recently from 7012ba2 to eb0310e Compare February 3, 2022 19:16

tldahlgren force-pushed the spack-ci-stand-alone-testing branch 7 times, most recently from 7dda56f to de1fd1d Compare February 16, 2022 18:40

tldahlgren force-pushed the spack-ci-stand-alone-testing branch 3 times, most recently from 9c2454e to ebb1e82 Compare February 17, 2022 02:10

tldahlgren force-pushed the spack-ci-stand-alone-testing branch 2 times, most recently from c4bfe48 to 3b50789 Compare February 18, 2022 01:20

tldahlgren mentioned this pull request Feb 22, 2022

Testing: Summarize test results and add verbose output #28700

Merged

tldahlgren mentioned this pull request Aug 9, 2022

Reduce installation noise: filter path padding and make some details debug level 2 #31998

Merged

tldahlgren mentioned this pull request Aug 12, 2022

Update CI Pipeline docs with stand-alone test feature #32048

Merged

tgamblin approved these changes Aug 23, 2022

View reviewed changes

greenc-FNAL mentioned this pull request Sep 9, 2022

Changes required for Cetmodules dependencies. FNALssi/spack#31

Merged

alalazo reviewed Dec 27, 2022

View reviewed changes

Conversation

tldahlgren commented Dec 9, 2021 • edited by tgamblin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tldahlgren commented Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

tldahlgren commented Jan 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tldahlgren commented Feb 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tldahlgren commented Aug 4, 2022

Uh oh!

spackbot-app bot commented Aug 4, 2022

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 9, 2022

Uh oh!

tldahlgren commented Aug 9, 2022

Uh oh!

spackbot-app bot commented Aug 9, 2022

Uh oh!

spackbot-app bot commented Aug 9, 2022

Uh oh!

tldahlgren commented Aug 9, 2022

Uh oh!

spackbot-app bot commented Aug 9, 2022

Uh oh!

tldahlgren commented Aug 9, 2022

Uh oh!

spackbot-app bot commented Aug 9, 2022

Uh oh!

spackbot-app bot commented Aug 9, 2022

Uh oh!

tldahlgren commented Aug 9, 2022

Uh oh!

tldahlgren commented Aug 10, 2022

Uh oh!

tldahlgren commented Aug 10, 2022

Uh oh!

tldahlgren commented Aug 10, 2022

Uh oh!

tldahlgren commented Aug 15, 2022

Uh oh!

tldahlgren commented Aug 22, 2022

Uh oh!

tgamblin left a comment

Choose a reason for hiding this comment

Uh oh!

alalazo Dec 27, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tldahlgren commented Dec 9, 2021 •

edited by tgamblin

Loading

tldahlgren commented Jan 27, 2022 •

edited

Loading

tldahlgren commented Jan 29, 2022 •

edited

Loading

tldahlgren commented Feb 17, 2022 •

edited

Loading