Reduce installation noise: filter path padding and make some details debug level 2 by tldahlgren · Pull Request #31998 · spack/spack

tldahlgren · 2022-08-08T20:29:46Z

CI logs, especially for packages with lots of dependencies, can fill up before the installation completes. This can lead to problems identifying failures that occur after the log is truncated (e.g., post-install or stand-alone testing).

This PR adds path padding filters to the installation process, including prefix directory-related outputs (e.g., directory creation and installation success). A number of "extra" debug messages in binary_distribution and relocate are also changed from level 1 to level 2 (i.e., -dd will be needed to see them).

Both of these changes were needed to help identify a failure with post-installation stand-alone testing in a package built in CI stage 13 .. up to 19.

…to level 2

tldahlgren · 2022-08-08T21:30:18Z

@spackbot fix style

spackbot-app · 2022-08-08T21:30:20Z

Let me see if I can fix that for you!

spackbot-app · 2022-08-08T21:39:25Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/installer.py
  lib/spack/spack/relocate.py
  lib/spack/spack/test/util/path.py
  lib/spack/spack/util/path.py
==> Bootstrapping py-black from pre-built binaries
==> Bootstrapping py-black@:21 %gcc target=x86_64 from sources
==> Bootstrapping clingo from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.2/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-dtwevigmwgke4g6ee5byktpmzmrp2kvx.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.2/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstrap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-dtwevigmwgke4g6ee5byktpmzmrp2kvx.spack
==> Installing "clingo-bootstrap@spack%[email protected]~docs~ipo+python+static_libstdcpp build_type=Release arch=linux-centos7-x86_64" from a buildcache
==> Bootstrapping patchelf from pre-built binaries
==> Bootstrapping [email protected]:0.13.99 %gcc target=x86_64 from sources
==> Waiting for patchelf-0.13.1-eoqgkon43rpy523egehhgtnjn4rywwap
�[1F�[J==> Installing patchelf-0.13.1-eoqgkon43rpy523egehhgtnjn4rywwap
==> No binary for patchelf-0.13.1-eoqgkon43rpy523egehhgtnjn4rywwap found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/08/08c0237e89be74d61ddf8f6ff218439cdd62af572d568fb38913b53e222831de.tar.gz
==> No patches needed for patchelf
==> patchelf: Executing phase: 'autoreconf'
==> patchelf: Executing phase: 'configure'
==> patchelf: Executing phase: 'build'
==> patchelf: Executing phase: 'install'
==> patchelf: Successfully installed patchelf-0.13.1-eoqgkon43rpy523egehhgtnjn4rywwap
  Fetch: 0.10s.  Build: 11.82s.  Total: 11.92s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/patchelf-0.13.1-eoqgkon43rpy523egehhgtnjn4rywwap
[+] /usr/local (external python-3.7-m5tyrabctwmb6w2yqcjyn6vkjnqfe4eo)
==> Waiting for py-pip-22.1.2-hge3libdbaqm3xzcsms63zqpr6wwtsom
�[1F�[J==> Installing py-pip-22.1.2-hge3libdbaqm3xzcsms63zqpr6wwtsom
==> No binary for py-pip-22.1.2-hge3libdbaqm3xzcsms63zqpr6wwtsom found: installing from source
==> Fetching https://files.pythonhosted.org/packages/py3/p/pip/pip-22.1.2-py3-none-any.whl
==> No patches needed for py-pip
==> py-pip: Executing phase: 'install'
==> py-pip: Successfully installed py-pip-22.1.2-hge3libdbaqm3xzcsms63zqpr6wwtsom
  Fetch: 1.09s.  Build: 10.77s.  Total: 11.86s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-pip-22.1.2-hge3libdbaqm3xzcsms63zqpr6wwtsom
==> Waiting for py-wheel-0.37.1-p4tw5z4vfqnxj747ditmhatt44xs2c7g
�[1F�[J==> Installing py-wheel-0.37.1-p4tw5z4vfqnxj747ditmhatt44xs2c7g
==> No binary for py-wheel-0.37.1-p4tw5z4vfqnxj747ditmhatt44xs2c7g found: installing from source
==> Fetching https://files.pythonhosted.org/packages/py2.py3/w/wheel/wheel-0.37.1-py2.py3-none-any.whl
==> No patches needed for py-wheel
==> py-wheel: Executing phase: 'install'
==> py-wheel: Successfully installed py-wheel-0.37.1-p4tw5z4vfqnxj747ditmhatt44xs2c7g
  Fetch: 0.71s.  Build: 0.97s.  Total: 1.68s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-wheel-0.37.1-p4tw5z4vfqnxj747ditmhatt44xs2c7g
==> Waiting for py-setuptools-63.0.0-pq7ogdo33ivcf2vpeglaopiim4blydx5
�[1F�[J==> Installing py-setuptools-63.0.0-pq7ogdo33ivcf2vpeglaopiim4blydx5
==> No binary for py-setuptools-63.0.0-pq7ogdo33ivcf2vpeglaopiim4blydx5 found: installing from source
==> Fetching https://files.pythonhosted.org/packages/py3/s/setuptools/setuptools-63.0.0-py3-none-any.whl
==> No patches needed for py-setuptools
==> py-setuptools: Executing phase: 'install'
==> py-setuptools: Successfully installed py-setuptools-63.0.0-pq7ogdo33ivcf2vpeglaopiim4blydx5
  Fetch: 0.78s.  Build: 2.03s.  Total: 2.82s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-setuptools-63.0.0-pq7ogdo33ivcf2vpeglaopiim4blydx5
==> Waiting for py-flit-core-3.6.0-ze6ytlutraufsemkpocowy3o5z7t64zo
�[1F�[J==> Installing py-flit-core-3.6.0-ze6ytlutraufsemkpocowy3o5z7t64zo
==> No binary for py-flit-core-3.6.0-ze6ytlutraufsemkpocowy3o5z7t64zo found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/58/5892962ab8b8ea945835b3a288fe9dd69316f1903d5288c3f5cafdcdd04756ad.tar.gz
==> No patches needed for py-flit-core
==> py-flit-core: Executing phase: 'install'
==> py-flit-core: Successfully installed py-flit-core-3.6.0-ze6ytlutraufsemkpocowy3o5z7t64zo
  Fetch: 0.05s.  Build: 1.42s.  Total: 1.46s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-flit-core-3.6.0-ze6ytlutraufsemkpocowy3o5z7t64zo
==> Waiting for py-tomli-1.2.2-g3lsxkhroyvco4kcbdrlzbbeux5mlmin
�[1F�[J==> Installing py-tomli-1.2.2-g3lsxkhroyvco4kcbdrlzbbeux5mlmin
==> No binary for py-tomli-1.2.2-g3lsxkhroyvco4kcbdrlzbbeux5mlmin found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/f0/f04066f68f5554911363063a30b108d2b5a5b1a010aa8b6132af78489fe3aade
==> No patches needed for py-tomli
==> py-tomli: Executing phase: 'install'
==> py-tomli: Successfully installed py-tomli-1.2.2-g3lsxkhroyvco4kcbdrlzbbeux5mlmin
  Fetch: 0.05s.  Build: 0.83s.  Total: 0.88s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-tomli-1.2.2-g3lsxkhroyvco4kcbdrlzbbeux5mlmin
==> Waiting for py-pathspec-0.9.0-b37veol3hto4y35reodhgws5btuppvoh
�[1F�[J==> Installing py-pathspec-0.9.0-b37veol3hto4y35reodhgws5btuppvoh
==> No binary for py-pathspec-0.9.0-b37veol3hto4y35reodhgws5btuppvoh found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/e5/e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1.tar.gz
==> No patches needed for py-pathspec
==> py-pathspec: Executing phase: 'install'
==> py-pathspec: Successfully installed py-pathspec-0.9.0-b37veol3hto4y35reodhgws5btuppvoh
  Fetch: 0.05s.  Build: 1.82s.  Total: 1.87s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-pathspec-0.9.0-b37veol3hto4y35reodhgws5btuppvoh
==> Waiting for py-typed-ast-1.4.3-rd64us7fgdomp7wc4vds37cwk6qgfoz2
�[1F�[J==> Installing py-typed-ast-1.4.3-rd64us7fgdomp7wc4vds37cwk6qgfoz2
==> No binary for py-typed-ast-1.4.3-rd64us7fgdomp7wc4vds37cwk6qgfoz2 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/fb/fb1bbeac803adea29cedd70781399c99138358c26d05fcbd23c13016b7f5ec65.tar.gz
==> No patches needed for py-typed-ast
==> py-typed-ast: Executing phase: 'install'
==> py-typed-ast: Successfully installed py-typed-ast-1.4.3-rd64us7fgdomp7wc4vds37cwk6qgfoz2
  Fetch: 0.07s.  Build: 16.05s.  Total: 16.13s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-typed-ast-1.4.3-rd64us7fgdomp7wc4vds37cwk6qgfoz2
==> Waiting for py-pyparsing-3.0.6-n33r7gon6t6xyeq6dqiuvof3xypuryv7
�[1F�[J==> Installing py-pyparsing-3.0.6-n33r7gon6t6xyeq6dqiuvof3xypuryv7
==> No binary for py-pyparsing-3.0.6-n33r7gon6t6xyeq6dqiuvof3xypuryv7 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/d9/d9bdec0013ef1eb5a84ab39a3b3868911598afa494f5faa038647101504e2b81.tar.gz
==> No patches needed for py-pyparsing
==> py-pyparsing: Executing phase: 'install'
==> py-pyparsing: Successfully installed py-pyparsing-3.0.6-n33r7gon6t6xyeq6dqiuvof3xypuryv7
  Fetch: 0.07s.  Build: 2.60s.  Total: 2.68s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-pyparsing-3.0.6-n33r7gon6t6xyeq6dqiuvof3xypuryv7
==> Waiting for py-mypy-extensions-0.4.3-oerst7xeypdgzi7y47umxay2d7c6upkj
�[1F�[J==> Installing py-mypy-extensions-0.4.3-oerst7xeypdgzi7y47umxay2d7c6upkj
==> No binary for py-mypy-extensions-0.4.3-oerst7xeypdgzi7y47umxay2d7c6upkj found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/2d/2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8.tar.gz
==> No patches needed for py-mypy-extensions
==> py-mypy-extensions: Executing phase: 'install'
==> py-mypy-extensions: Successfully installed py-mypy-extensions-0.4.3-oerst7xeypdgzi7y47umxay2d7c6upkj
  Fetch: 0.07s.  Build: 1.81s.  Total: 1.88s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-mypy-extensions-0.4.3-oerst7xeypdgzi7y47umxay2d7c6upkj
==> Waiting for py-typing-extensions-4.3.0-4uc3eokghp27gpqqtapfgc7ipkmuqbes
�[1F�[J==> Installing py-typing-extensions-4.3.0-4uc3eokghp27gpqqtapfgc7ipkmuqbes
==> No binary for py-typing-extensions-4.3.0-4uc3eokghp27gpqqtapfgc7ipkmuqbes found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/t/typing_extensions/typing_extensions-4.3.0.tar.gz
==> No patches needed for py-typing-extensions
==> py-typing-extensions: Executing phase: 'install'
==> py-typing-extensions: Successfully installed py-typing-extensions-4.3.0-4uc3eokghp27gpqqtapfgc7ipkmuqbes
  Fetch: 0.75s.  Build: 1.58s.  Total: 2.33s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-typing-extensions-4.3.0-4uc3eokghp27gpqqtapfgc7ipkmuqbes
==> Waiting for py-packaging-21.3-2rcpfowsvy5oxooasu4vz6qofvmdwg4p
�[1F�[J==> Installing py-packaging-21.3-2rcpfowsvy5oxooasu4vz6qofvmdwg4p
==> No binary for py-packaging-21.3-2rcpfowsvy5oxooasu4vz6qofvmdwg4p found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/dd/dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb.tar.gz
==> No patches needed for py-packaging
==> py-packaging: Executing phase: 'install'
==> py-packaging: Successfully installed py-packaging-21.3-2rcpfowsvy5oxooasu4vz6qofvmdwg4p
  Fetch: 0.06s.  Build: 1.96s.  Total: 2.02s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-packaging-21.3-2rcpfowsvy5oxooasu4vz6qofvmdwg4p
==> Waiting for py-setuptools-scm-6.3.2-b4cqub7cdlwibdxr53wu244ebsvj37df
�[1F�[J==> Installing py-setuptools-scm-6.3.2-b4cqub7cdlwibdxr53wu244ebsvj37df
==> No binary for py-setuptools-scm-6.3.2-b4cqub7cdlwibdxr53wu244ebsvj37df found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/a4/a49aa8081eeb3514eb9728fa5040f2eaa962d6c6f4ec9c32f6c1fba88f88a0f2.tar.gz
==> No patches needed for py-setuptools-scm
==> py-setuptools-scm: Executing phase: 'install'
==> py-setuptools-scm: Successfully installed py-setuptools-scm-6.3.2-b4cqub7cdlwibdxr53wu244ebsvj37df
  Fetch: 0.05s.  Build: 1.92s.  Total: 1.97s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-setuptools-scm-6.3.2-b4cqub7cdlwibdxr53wu244ebsvj37df
==> Waiting for py-zipp-3.6.0-q2b6smxtnio7bib6p2xoxc6imm7goija
�[1F�[J==> Installing py-zipp-3.6.0-q2b6smxtnio7bib6p2xoxc6imm7goija
==> No binary for py-zipp-3.6.0-q2b6smxtnio7bib6p2xoxc6imm7goija found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/71/71c644c5369f4a6e07636f0aa966270449561fcea2e3d6747b8d23efaa9d7832.tar.gz
==> No patches needed for py-zipp
==> py-zipp: Executing phase: 'install'
==> py-zipp: Successfully installed py-zipp-3.6.0-q2b6smxtnio7bib6p2xoxc6imm7goija
  Fetch: 0.06s.  Build: 1.80s.  Total: 1.86s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-zipp-3.6.0-q2b6smxtnio7bib6p2xoxc6imm7goija
==> Waiting for py-platformdirs-2.4.1-pbuoscetficyn7tfiep2ysqimnojbtwm
�[1F�[J==> Installing py-platformdirs-2.4.1-pbuoscetficyn7tfiep2ysqimnojbtwm
==> No binary for py-platformdirs-2.4.1-pbuoscetficyn7tfiep2ysqimnojbtwm found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/p/platformdirs/platformdirs-2.4.1.tar.gz
==> No patches needed for py-platformdirs
==> py-platformdirs: Executing phase: 'install'
==> py-platformdirs: Successfully installed py-platformdirs-2.4.1-pbuoscetficyn7tfiep2ysqimnojbtwm
  Fetch: 0.86s.  Build: 2.68s.  Total: 3.54s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-platformdirs-2.4.1-pbuoscetficyn7tfiep2ysqimnojbtwm
==> Waiting for py-importlib-metadata-4.11.1-6csfxvkvqf3ms7v6jx4usdowp2fx6qv7
�[1F�[J==> Installing py-importlib-metadata-4.11.1-6csfxvkvqf3ms7v6jx4usdowp2fx6qv7
==> No binary for py-importlib-metadata-4.11.1-6csfxvkvqf3ms7v6jx4usdowp2fx6qv7 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/17/175f4ee440a0317f6e8d81b7f8d4869f93316170a65ad2b007d2929186c8052c.tar.gz
==> No patches needed for py-importlib-metadata
==> py-importlib-metadata: Executing phase: 'install'
==> py-importlib-metadata: Successfully installed py-importlib-metadata-4.11.1-6csfxvkvqf3ms7v6jx4usdowp2fx6qv7
  Fetch: 0.05s.  Build: 2.40s.  Total: 2.45s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-importlib-metadata-4.11.1-6csfxvkvqf3ms7v6jx4usdowp2fx6qv7
==> Waiting for py-click-8.0.3-6fmk7fvpnldsa6g4yfmrzohow6oal6yk
�[1F�[J==> Installing py-click-8.0.3-6fmk7fvpnldsa6g4yfmrzohow6oal6yk
==> No binary for py-click-8.0.3-6fmk7fvpnldsa6g4yfmrzohow6oal6yk found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/41/410e932b050f5eed773c4cda94de75971c89cdb3155a72a0831139a79e5ecb5b.tar.gz
==> No patches needed for py-click
==> py-click: Executing phase: 'install'
==> py-click: Successfully installed py-click-8.0.3-6fmk7fvpnldsa6g4yfmrzohow6oal6yk
  Fetch: 0.45s.  Build: 2.66s.  Total: 3.11s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-click-8.0.3-6fmk7fvpnldsa6g4yfmrzohow6oal6yk
==> Waiting for py-black-21.12b0-ki5r6swxc7hzemj2pe4ewhs3ijemag33
�[1F�[J==> Installing py-black-21.12b0-ki5r6swxc7hzemj2pe4ewhs3ijemag33
==> No binary for py-black-21.12b0-ki5r6swxc7hzemj2pe4ewhs3ijemag33 found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/b/black/black-21.12b0.tar.gz
==> No patches needed for py-black
==> py-black: Executing phase: 'install'
==> py-black: Successfully installed py-black-21.12b0-ki5r6swxc7hzemj2pe4ewhs3ijemag33
  Fetch: 0.70s.  Build: 3.10s.  Total: 3.81s.
[+] /root/.spack/bootstrap/store/linux-debian10-x86_64/gcc-8.3.0/py-black-21.12b0-ki5r6swxc7hzemj2pe4ewhs3ijemag33
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 555 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/test/util/path.py
All done! ✨ 🍰 ✨
1 file reformatted, 4 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

But it looks like I'm not able to push to your branch. 😭️ Did you check maintainer can edit when you opened the PR?

tldahlgren · 2022-08-08T23:22:33Z

@spackbot fix style

spackbot-app · 2022-08-08T23:22:35Z

Let me see if I can fix that for you!

spackbot-app · 2022-08-08T23:24:24Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, mypy, black, flake8
==> Modified files
  lib/spack/spack/binary_distribution.py
  lib/spack/spack/installer.py
  lib/spack/spack/relocate.py
  lib/spack/spack/test/util/path.py
  lib/spack/spack/util/path.py
==> Running isort checks
  isort checks were clean
==> Running mypy checks
Success: no issues found in 555 source files
  mypy checks were clean
==> Running black checks
reformatted lib/spack/spack/test/util/path.py
All done! ✨ 🍰 ✨
1 file reformatted, 4 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with isort fixes.

tldahlgren · 2022-08-09T01:32:02Z

These are the changes that allowed #27877 to NOT truncate the CI log for gptune when it failed in the stand-alone tests phase, post-successful install. From some PRs I looked at earlier, it could help cut the noise in other failing package builds.

scottwittenburg

Looks good to me, and I'm looking forward to having less cluttered logs in gitlab CI!

scottwittenburg · 2022-08-09T17:16:09Z

lib/spack/spack/util/path.py

    entirety at least one time. e.g., "/spack/" would not be filtered, but
    "/__spack_path_placeholder__/spack/" would be.

+    Note that only the first padded path in the string is filtered.


Did you encounter cases where the padded path appeared multiple times in a string? How hard would it be to filter every instance? Not sure if it's just a matter of supplying the count parameter to sub(), or supplying a combination of flags e.g. re.MULTILINE | re.DOTALL?

The comment is per-call so, IIRC, the path filtering works fine where it has been added. For example, the patchelf command that was causing issues with the gptune log for #27877 that we discussed before shows each dependency path padded as shown below:

==> [2022-08-09-00:53:57.354561] '/usr/bin/patchelf' '--force-rpath' '--set-rpath' '/home/software/spack/[padded-to-512-chars]/linux-ubuntu22.04-x86_64/gcc-11.2.0/pkgconf-1.8.0-wsqf3pdlaknj3vt7bq7skjmztdd5flps/lib:/home/software/spack/[padded-to-600-chars]/morepadding/linux-ubuntu22.04-x86_64/gcc-11.2.0/pkgconf-1.8.0-wsqf3pdlaknj3vt7bq7skjmztdd5flps/lib:/home/software/spack/[padded-to-612-chars]/linux-ubuntu22.04-x86_64/gcc-11.2.0/pkgconf-1.8.0-wsqf3pdlaknj3vt7bq7skjmztdd5flps/lib64:/home/software/spack/[padded-to-602-chars]/morepadding/linux-ubuntu22.04-x86_64/gcc-11.2.0/pkgconf-1.8.0-wsqf3pdlaknj3vt7bq7skjmztdd5flps/lib64' '/home/software/spack/[padded-to-512-chars]/linux-ubuntu22.04-x86_64/gcc-11.2.0/pkgconf-1.8.0-wsqf3pdlaknj3vt7bq7skjmztdd5flps/bin/pkgconf'

(From https://gitlab.spack.io/spack/spack/-/jobs/2875679.)

As for filtering each instance, the API could certainly be changed. I tried to keep the modifications to a minimum for this PR though.

alalazo · 2022-08-09T17:22:18Z

Thanks Tammy!

tldahlgren added 2 commits August 8, 2022 13:07

Reduce CI noise: filter path padding; switch selected debug messages …

a726585

…to level 2

Try change so don't skip new unit test for windows

f53a88e

spackbot-app bot added binary-packages core PR affects Spack core functionality tests General test capability(ies) utilities labels Aug 8, 2022

tldahlgren mentioned this pull request Aug 8, 2022

spack ci: add support for running stand-alone tests #27877

Merged

9 tasks

[@spackbot] updating style on behalf of tldahlgren

a894f16

tldahlgren requested review from alalazo and scottwittenburg August 9, 2022 01:29

scottwittenburg approved these changes Aug 9, 2022

View reviewed changes

alalazo approved these changes Aug 9, 2022

View reviewed changes

alalazo merged commit 3320c98 into spack:develop Aug 9, 2022

tldahlgren mentioned this pull request Aug 9, 2022

update ncurses- add v6.3, symlink libcurses -> libncurses #31993

Merged

greenc-FNAL mentioned this pull request Sep 9, 2022

Changes required for Cetmodules dependencies. FNALssi/spack#31

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce installation noise: filter path padding and make some details debug level 2#31998

Reduce installation noise: filter path padding and make some details debug level 2#31998
alalazo merged 3 commits intospack:developfrom
tldahlgren:reduce-ci-noise

tldahlgren commented Aug 8, 2022 •

edited

Loading

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 9, 2022 •

edited

Loading

Uh oh!

scottwittenburg left a comment

Uh oh!

scottwittenburg Aug 9, 2022

Uh oh!

tldahlgren Aug 9, 2022

Uh oh!

tldahlgren Aug 9, 2022

Uh oh!

alalazo commented Aug 9, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tldahlgren commented Aug 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

spackbot-app bot commented Aug 8, 2022

Uh oh!

tldahlgren commented Aug 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

scottwittenburg left a comment

Choose a reason for hiding this comment

Uh oh!

scottwittenburg Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

tldahlgren Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

tldahlgren Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

alalazo commented Aug 9, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tldahlgren commented Aug 8, 2022 •

edited

Loading

tldahlgren commented Aug 9, 2022 •

edited

Loading