-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Always capture clone logs in tests and remove or update stale workarounds #5906
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In commit 904145f of PR git-lfs#306 the original version of our t/t-happy-path.sh example test script was added, with an introductory comment describing the one test that was in the file at the time as a sample test of the Git LFS client. We have since copied that comment into several other test scripts, likely by accident, so we remove those comments now. We also move the comment in the t/t-happy-path.sh script so it directly precedes the original "happy path" test, as the script now contains a number of other tests that are not generic tests intended to be used as examples.
In commit 32244d9 of PR git-lfs#1825 the APPVEYOR_REPO_COMMIT_MESSAGE variable was added to our CI test suite environment in order to demonstrate that a problem exposed by the prior commit 0951c69 in the same PR had been addressed. Specifically, the earlier commit had the string "GIT_SSH" in its message, and the AppVeyor CI service automatically provided the message from the most recent commit in a Git branch under test in the APPVEYOR_REPO_COMMIT_MESSAGE environment variable. At the time, the Environ() function in our "lfs" package would populate the map it returns with the names and values of any environment variables whose names or values contained the string "GIT_". The result was that the APPVEYOR_REPO_COMMIT_MESSAGE variable was included in the returned environment data, which would cause the tests in what is now our t/t-env.sh test script to fail. This problem was resolved by adjusting the Environ() function to only include environment variables in its map when their name contained the string "GIT_". To validate this fix, our test script environment was updated to write a fixed value into the APPVEYOR_REPO_COMMIT_MESSAGE environment variable (overriding whatever the AppVeyor service set) with an example string containing the string "GIT_". The example string used was the commit message from the commit which exposed the problem. Subsequently, we further revised the Environ() function in commit f4f8fae of PR git-lfs#4269 so it only includes environment variables whose names start with "GIT_", excluding those whose names merely contain that string. As we no longer use the AppVeyor service since we migrated our CI jobs to GitHub Actions in PR git-lfs#3808, and because the intent of the APPVEYOR_REPO_COMMIT_MESSAGE variable is now somewhat obscure, we replace it with a dedicated TEST_GIT_EXAMPLE environment variable that contains the "GIT_" string in both its name and value. We define this TEST_GIT_EXAMPLE variable only in the test scripts where the output of "git lfs env" is checked, and we preface its definition with some comments to clarify its purpose. If a regression were ever to be introduced into the Environ() function such that it did not exclude environment variables with "GIT_" in their names or values, the tests in the t/t-env.sh and t/t-worktree.sh scripts should detect the problem due to the presence of the TEST_GIT_EXAMPLE variable.
In commit 9a00eb3 of PR git-lfs#1893 we altered our "clone ClientCert" test of the Git LFS client's support for TLS/SSL client certificates to deal with an exception raised by the version of libcurl provided by macOS at the time. As was reported in curl/curl#1105, the libcurl installed as part of macOS 10.12 ("Sierra") failed to find some TLS/SSL client certificates and instead raised an NSInvalidArgumentException. See the notes in our PR git-lfs#1869 for more details: git-lfs#1869 (comment) To work around this issue, we skip the "clone ClientCert" test if any NSInvalidArgumentException messages are found in the output of the Git clone operation performed by the clone_repo_clientcert() function of our t/testhelpers.sh shell library. The same workaround was then copied into the "clone ClientCert with homedir certs" test when it was added to our t/t-clone.sh test script in commit c54c9da of PR git-lfs#5657. The underlying problem in libcurl was resolved by the changes in commit curl/curl@7c9b9ad, which was included with libcurl version 7.52.0 and so is available in all contemporary versions of macOS. For instance, by the time of macOS 10.14 ("Mojave"), the version of libcurl packaged with the OS was v7.54.0. We can therefore drop our workarounds, as they are no longer applicable to any supported versions of macOS. (Note, too, that the GitHub Actions runners we now use for our CI jobs provide a libcurl version installed by Homebrew, which takes precedence over the one supplied by macOS.)
The clone_repo*() family of helper functions defined in our t/testhelpers.sh shell library are used extensively throughout our test suite to perform "git clone" commands and log their output for verification. After running the "git clone" command, these functions change the current working directory to the cloned repository's working tree and then write the log of the command's output into that directory. Before creating the log file, however, they call the "git config" command to register our git-credential-lfstest test utility as the program Git should use to store and retrieve credentials. One consequence of the way the majority of these functions operate is that if the "git clone" command should fail, then because we run all our tests with the "set -e" shell option, no log file is generated. Instead, the functions return immediately and do not change the working directory or write a log file. The clone_repo_clientcert() function is an exception to this pattern, as a result of the changes introduced to it in commit 9a00eb3 of PR git-lfs#1893, which were made to work around a problem caused by the version of libcurl installed with macOS version 10.12 ("Sierra"). In the clone_repo_clientcert() function, the "set +e" shell option was temporarily set before running the "git clone" command, and command's exit code captured before setting the "set -e" option again. This allowed the function to check the command's output for the macOS error message specific to the libcurl problem, and if it was present, add a flag string to the log file and not return an error code. The calling test could then simply skip the remainder of its normal actions if it detected the flag string in the log file. In a prior commit in this PR we reversed some of these changes because the underlying issue is fixed in all recent macOS and libcurl releases, and so we no longer need to try to detect specific exceptions in the output of the clone operation in the clone_repo_clientcert() function. We did not reverse the use of the "set +e" shell option, though, because there is some value in trying to avoid an immediate test failure if the "git clone" command does not succeed, as will occur with all the other clone_repo*() functions. Under most circumstances the behaviour of the other functions is reasonable, as any error messages from the "git clone" commands will still appear in the stdout and stderr streams captured by our CI job test framework, and so may be analyzed there. However, this leaves no way for tests using these functions to programmatically check the output of a failed "git clone" attempt. For instance, if they are looking for a specific string to indicate that the clone was unable to find a valid Git executable and ran an invalid one instead (as per CVE-2022-24826), we may want a test to be able to search the log file from a clone operation even after it fails. Therefore, in order to ensure that clone operation log traces are saved in all cases, we change all of the clone_repo*() functions to pipe the output of the "git clone" command into a tee(1) command, which then writes the log file. The advantage of the "tee" command over a simple redirection of stdout is that when the "set -e" option is set, the shell only considers the exit code of the final command in a pipeline to determine the success of the pipeline, so a non-zero exit code from the "git clone" command will not cause the function to return immediately. We then retrieve the "git clone" command's exit code from the shell's PIPESTATUS[0] variable and return if the code is non-zero. If the clone operation was successful, though, we proceed to change the working directory and move the log file into the new repository's working tree, where our tests expect to find it, and then return after calling the "git config" command. In the clone_repo_clientcert() function specifically, this change now allows us to remove the use of the "set +e" shell option, as we are able to read the exit code from the "git clone" command without it. We also no longer need separate statements to write the log file under both success and failure conditions.
larsxschneider
approved these changes
Nov 11, 2024
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Nov 18, 2024
In commit 365d615 of PR git-lfs#1478 a pair of tests in what is now our t/t-clone.sh test script were enhanced to check that the "git lfs clone" command did not create a directory named "lfs" within the top level of a cloned repository's working tree, as the "lfs" directory should instead be created within the ".git" directory. This check has since been replicated in a number of other tests in the t/t-clone.sh script, but not all of them, so we add this check in the relevant sections of the tests where it is not yet performed. We also adjust one check which dates from commit 365d615 to use the same -e shell test as all the other checks, rather than the -f test. These changes are not especially important, but in conjunction with the revisions we will make in other commits in this PR, along with those we made recently in PRs git-lfs#5882 and git-lfs#5906, they will help synchronize the tests in the t/t-clone.sh script so they are more consistent with each other and complete.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates our test suite to remove or refine a number of stale comments and code, including detection of an old TLS/SSL client certificate problem with libcurl on macOS 10.12 ("Sierra") from 2016, which was used to skip some of our tests of the
git lfs clonecommand, and the obscure name of an environment variable intended to validate the behaviour of thegit lfs envcommand.We then refactor the family of
clone_repo*()functions in our test suite's shell library so they all operate identically and always create a log file with the output of thegit clonecommand, regardless of whether that command succeeds or fails. This will allow us to write tests which intentionally perform agit clonecommand that fails, and then check the log file for particular error messages.Regarding the issue with libcurl on macOS 10.12, the underlying issue was fixed by commit curl/curl@7c9b9ad in libcurl version 7.52.0, and so we no longer need to work around any macOS-specific exceptions raised by the use of TLS/SSL client certificates in our tests.
As for the legacy
APPVEYOR_REPO_COMMIT_MESSAGEenvironment variable set by our test suite, this was introduced in PR #1825 to demonstrate that a problem exposed by a commit in that PR had been addressed. TheEnvrion()function in ourlfspackage, which is used to produce the output of thegit lfs envcommand, currently ignores any environment variables whose names do not begin withGIT_. However, at the time of PR #1825 it would instead include in the map it returns any variables whose names or values merely contained the stringGIT_. This issue was revealed when the AppVeyor CI service then in use populated one of its own environment variables with the message from commit 0951c69 that just happened to contain the stringGIT_. To confirm that the bug was fixed, a copy of the problematic variable was simply inserted into our test suite's environment, keeping both the name and value. (For more discussion of this history, see the discussion in #5615 (comment).) The variable's name makes its purpose unclear, so we replace it with dedicatedTEST_GIT_EXAMPLEenvironment variables in the specific test scripts that check the output of thegit lfs envcommand.This PR will be most easily reviewed on a commit-by-commit basis.