Skip to content

Consolidate how Spack uses git#34700

Merged
tgamblin merged 6 commits intodevelopfrom
fix-git-in-tests
Dec 28, 2022
Merged

Consolidate how Spack uses git#34700
tgamblin merged 6 commits intodevelopfrom
fix-git-in-tests

Conversation

@tgamblin
Copy link
Copy Markdown
Member

@tgamblin tgamblin commented Dec 26, 2022

Local git tests will fail with fatal: transport 'file' not allowed when using git 2.38.1 or higher, due to a fix for CVE-2022-39253.

This was fixed in CI in #33429, but that doesn't help the issue for anyone's local environment. Instead of fixing this with git config in CI, we should ensure that the tests run anywhere.

  • Introduce spack.util.git.
  • Use spack.util.git.get_git() to get a git executable, instead of which("git") everywhere.
  • Make all git tests use a git fixture that goes through spack.util.git.get_git().
  • Add -c protocol.file.allow=always to all git invocations under pytest.
  • Revert changes from FIX CI after git update #33429, which are no longer needed.

@tgamblin tgamblin requested a review from alalazo December 26, 2022 20:08
@spackbot-app spackbot-app bot added commands core PR affects Spack core functionality fetching tests General test capability(ies) workflow labels Dec 26, 2022
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This basically LGTM. I just left a few minor comments:

  1. The rename spack.util.git.get_git() -> spack.util.git.git() is absolutely minor. I think we are pretty inconsistent at the moment, so I'd be fine with either if you prefer the current state.
  2. I need to double check if skipif can be applied on a fixture (I can't find clear docs on it, all the examples seem to suggest we need to use the imperative method during the setup phase)

Other than that it's 💯 to unify code!

Comment on lines +23 to +30
# If we're running under pytest, add this to ignore the fix for CVE-2022-39253 in
# git 2.38.1+. Do this in one place; we need git to do this in all parts of Spack.
if "pytest" in sys.modules:
git.add_default_arg("-c")
git.add_default_arg("protocol.file.allow=always")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we work around a CVE without letting users know? Where would this be relevant besides CI?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We know the CVE doesn't affect us because it's not a multi-user environment and we created the directories ourselves.

@alalazo alalazo self-assigned this Dec 27, 2022
@tgamblin
Copy link
Copy Markdown
Member Author

@alalazo all should be fixed

@tgamblin tgamblin requested a review from alalazo December 27, 2022 18:19
@tgamblin tgamblin force-pushed the fix-git-in-tests branch 2 times, most recently from 48a8607 to 55b15cf Compare December 27, 2022 20:52
@spackbot-app spackbot-app bot added the documentation Improvements or additions to documentation label Dec 27, 2022
Local `git` tests will fail with `fatal: transport 'file' not allowed` when using git
2.38.1 or higher, due to a fix for `CVE-2022-39253`.

This was fixed in CI in #33429, but that doesn't help the issue for anyone's local
environment. Instead of fixing this with git config in CI, we should ensure that the
tests run anywhere.

- [x] Introduce `spack.util.git`.
- [x] Use `spack.util.git.get_git()` to get a `git` executable, instead of
      using `which("git")` everywhere.
- [x] Make all `git` tests use a `git` fixture that goes through `spack.util.git.get_git()`.
- [x] Add `-c protocol.file.allow=always` to all `git` invocations under `pytest`.
- [x] Revert changes from #33429, which are no longer needed.
@tgamblin tgamblin force-pushed the fix-git-in-tests branch 3 times, most recently from 69ce8ed to bb4cdee Compare December 27, 2022 23:02
@tgamblin
Copy link
Copy Markdown
Member Author

tgamblin commented Dec 28, 2022

@alalazo I can't seem to get past this sphinx issue for Raises::

/Users/gamblin2/src/spack/lib/spack/spack/util/git.py:docstring of spack.util.git.git:1: WARNING: py:exc reference target not found: CommandNotFoundError

It doesn't seem to matter if it's fully qualified or if I add it to nitpick_ignore in conf.py.

But everything else is done.

Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sphinx tests seems to be passing on readthedocs. I think this one just needs to pass Gitlab, once it is available again.

@tgamblin tgamblin merged commit 5f8c706 into develop Dec 28, 2022
@tgamblin tgamblin deleted the fix-git-in-tests branch December 28, 2022 08:44
stephenmsachs pushed a commit to stephenmsachs/spack that referenced this pull request Jan 3, 2023
Local `git` tests will fail with `fatal: transport 'file' not allowed` when using git 2.38.1 or higher, due to a fix for `CVE-2022-39253`.

This was fixed in CI in spack#33429, but that doesn't help the issue for anyone's local environment. Instead of fixing this with git config in CI, we should ensure that the tests run anywhere.

- [x] Introduce `spack.util.git`.
- [x] Use `spack.util.git.get_git()` to get a git executable, instead of `which("git")` everywhere.
- [x] Make all `git` tests use a `git` fixture that goes through `spack.util.git.get_git()`.
- [x] Add `-c protocol.file.allow=always` to all `git` invocations under `pytest`.
- [x] Revert changes from spack#33429, which are no longer needed.
RikkiButler20 pushed a commit to RikkiButler20/spack that referenced this pull request Jan 24, 2023
Local `git` tests will fail with `fatal: transport 'file' not allowed` when using git 2.38.1 or higher, due to a fix for `CVE-2022-39253`.

This was fixed in CI in spack#33429, but that doesn't help the issue for anyone's local environment. Instead of fixing this with git config in CI, we should ensure that the tests run anywhere.

- [x] Introduce `spack.util.git`.
- [x] Use `spack.util.git.get_git()` to get a git executable, instead of `which("git")` everywhere.
- [x] Make all `git` tests use a `git` fixture that goes through `spack.util.git.get_git()`.
- [x] Add `-c protocol.file.allow=always` to all `git` invocations under `pytest`.
- [x] Revert changes from spack#33429, which are no longer needed.
amd-toolchain-support pushed a commit to amd-toolchain-support/spack that referenced this pull request Feb 16, 2023
Local `git` tests will fail with `fatal: transport 'file' not allowed` when using git 2.38.1 or higher, due to a fix for `CVE-2022-39253`.

This was fixed in CI in spack#33429, but that doesn't help the issue for anyone's local environment. Instead of fixing this with git config in CI, we should ensure that the tests run anywhere.

- [x] Introduce `spack.util.git`.
- [x] Use `spack.util.git.get_git()` to get a git executable, instead of `which("git")` everywhere.
- [x] Make all `git` tests use a `git` fixture that goes through `spack.util.git.get_git()`.
- [x] Add `-c protocol.file.allow=always` to all `git` invocations under `pytest`.
- [x] Revert changes from spack#33429, which are no longer needed.
techxdave pushed a commit to Tech-XCorp/spack that referenced this pull request Feb 17, 2023
Local `git` tests will fail with `fatal: transport 'file' not allowed` when using git 2.38.1 or higher, due to a fix for `CVE-2022-39253`.

This was fixed in CI in spack#33429, but that doesn't help the issue for anyone's local environment. Instead of fixing this with git config in CI, we should ensure that the tests run anywhere.

- [x] Introduce `spack.util.git`.
- [x] Use `spack.util.git.get_git()` to get a git executable, instead of `which("git")` everywhere.
- [x] Make all `git` tests use a `git` fixture that goes through `spack.util.git.get_git()`.
- [x] Add `-c protocol.file.allow=always` to all `git` invocations under `pytest`.
- [x] Revert changes from spack#33429, which are no longer needed.
jmcarcell pushed a commit to key4hep/spack that referenced this pull request Apr 13, 2023
Local `git` tests will fail with `fatal: transport 'file' not allowed` when using git 2.38.1 or higher, due to a fix for `CVE-2022-39253`.

This was fixed in CI in spack#33429, but that doesn't help the issue for anyone's local environment. Instead of fixing this with git config in CI, we should ensure that the tests run anywhere.

- [x] Introduce `spack.util.git`.
- [x] Use `spack.util.git.get_git()` to get a git executable, instead of `which("git")` everywhere.
- [x] Make all `git` tests use a `git` fixture that goes through `spack.util.git.get_git()`.
- [x] Add `-c protocol.file.allow=always` to all `git` invocations under `pytest`.
- [x] Revert changes from spack#33429, which are no longer needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands core PR affects Spack core functionality documentation Improvements or additions to documentation fetching tests General test capability(ies) utilities workflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants