-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Add --fixup option to migrate info command #4501
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We adjust the text of the error message output in the case of an unrecognized value for the --pointers option to the "git lfs migrate info" command, in order to make the text and calling convention align with other similar option parsing and error handling.
We refactor the tests of various invalid options to the "git lfs migrate info" command to use the same checks as those found in the t/t-migrate-fixup.sh tests, among others. This allows us to confirm both the error message text and that the command fails as expected.
We split out the existing tests of various values of the "git lfs migrate info" --pointers option, added in PR git-lfs#4436, in order to treat each value in a separate test. This has the small advantage that if support for a particular option value should be broken unexpectedly in the future, it should result in a specific test failure instead of a more generic one.
We add a --fixup option to the "git lfs migrate info" command which has similar properties as the --fixup option to the "git lfs migrate import" command, namely that it looks for files in the repository which should be Git LFS pointers (according to the filter settings in .gitattributes) but which are not. In the case of the "migrate info" command this new option only reports the sum totals for any such files, and does not make any changes to the repository. When the --fixup option is supplied, the "migrate info" command will not output a summary of any other files, including .gitattributes files, which are otherwise normally included in the command's final output. The new --fixup option is taken to imply the --pointers=ignore option, and is incompatible with any other settings of that option.
bk2204
approved these changes
May 18, 2021
Member
bk2204
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great!
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Jun 1, 2023
The "migrate info (--fixup, no .gitattributes)" test was added to the t/t-migrate-info.sh test suite in commit 4800c5e of PR git-lfs#4501, when we added --fixup option support to the "git lfs migrate info" command. As we expect to introduce additional, similar tests in subsequent commits in this PR, we rename the test to clarify that no fixup action is expected in this case, and refactor the test's invocation of the "git lfs migrate info --fixup" command so that it does not run inside a pipeline. This latter change ensures that if the command fails, the test also fails due to the "set -e" shell option. While this change to avoid the use of a pipeline is not critical for this test, it will be important for the similar tests we add in subsequent commits, because they will need to validate that the "git lfs migrate info --fixup" command does not simply crash and produce no output on stdout, which, if it occured in a pipeline, would not be detected. This is particularly true in these tests because we expect the command to produce no output on stdout, which it would not do both on success and if it panics, so we can not distinguish these cases by examining the number of lines of output. We then make a copy of this test in the t/t-migrate-fixup.sh test suite as well, except using the "git lfs migrate import --fixup" command, as we also expect to add similar tests to this suite in subsequent commits. Note that for this test it is critical that we supply the --yes option to the command, to avoid it waiting forever for input because it has detected the log file as a new file in the Git working tree and therefore has prompted for confirmation to proceed. Finally, we remove one unneeded "git add" command from the setup_single_local_branch_tracked_corrupt() test helper function, which we refactor further in subsequent commits.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Jun 1, 2023
Support for Git macro attributes was added in a series of commits in PR git-lfs#3391, including commit 9d3e52d, where the Line structure of the "git/gitattr" package was updated to include a Macro element which would be set non-nil for macro definition lines in a .gitattributes file file, while the existing Pattern element would be set non-nil for all other lines. The "git lfs track" command, among others, was then adjusted to create a MacroProcessor structure (from the same "git/gitattr" package) and call its ProcessLines() method to resolve any macro references and thus convert the "raw" parsed Line structures into a set for which the Pattern element was always non-nil, and no Macro elements appeared. Later, the "git lfs fsck" command gained the ability to process macro definitions in .gitattributes files, in PR git-lfs#4525. However, the "git lfs migrate import" command was not adjusted, specifically in the implementation of its "--fixup" option, which initializes a Tree structure (also of the "git/gitattr" package) for the root tree of each commit in a repository's history using the package's New() function. This function traverses all the trees in the hierarchy and finds and parses all the .gitattributes files in them. Then, when the command visits each file within the commit's tree using the Rewrite() method of the Rewriter structure in the "git/githistory" package, it calls the (*Tree).Applied() method to match the file's path against any applicable Git attributes, to see if the file should be treated as a Git LFS object. This lack of support for macro attributes in the "git lfs migrate import --fixup" command was then propagated to the "git lfs migrate info --fixup" command in commit 4800c5e of PR git-lfs#4501, when the "git lfs migrate info" command was updated to respect the --fixup option. As a result, both of these commands (when used with the --fixup option) would panic if they encountered a .gitattributes file with any macro definition, as they would call the (*Tree).Applied() method and it would attempt to access the nil Pattern element of the lines with non-nil Macro elements. (Prior to the changes in commit c374d1f of PR git-lfs#5375 the "git lfs migrate import --fixup" command would then stall indefinitely, but it now also exits after the panic condition.) These problems were reported in issue git-lfs#5332. To resolve this problem and avoid similar ones in the future, we refactor the Line structure into a Line interface, which only provides a Attrs() method to retrieve a slice of Attr attributes, and no other methods. We then also define two additional interfaces, each of which embeds the Line interface, PatternLine and MacroLine, with corresponding getter methods for their respective elements. The ParseLine() function of the "git/gitattr" package now returns a slice of generic Line types, each of which is either a PatternLine or a MacroLine, but never both. Callers like the Applied() method of the Tree structure therefore need to perform type assertions or switches to determine which type of Line they are handling, which ensures they always access the line's data through safe methods. We then update the Go tests for the "git/gitattr" package as appropriate, and also add two tests each to the t/t-migrate-fixup.sh and t/t-migrate-import.sh test suites. All four of these new shell tests fail without the changes in this commit. In particular, several of these tests make sure to run the "git lfs migrate" commands outside of any shell pipeline so the test will fail if the command panics and produces no output, even if no output is the expected condition for a successful execution of the command.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Jun 1, 2023
Support for Git macro attributes was added in a series of commits in PR git-lfs#3391, including commit 9d3e52d, where the Line structure of the "git/gitattr" package was updated to include a Macro element which would be set non-nil for macro definition lines in a .gitattributes file file, while the existing Pattern element would be set non-nil for all other lines. The "git lfs track" command, among others, was then adjusted to create a MacroProcessor structure (from the same "git/gitattr" package) and call its ProcessLines() method to resolve any macro references and thus convert the "raw" parsed Line structures into a set for which the Pattern element was always non-nil, and no Macro elements appeared. Later, the "git lfs fsck" command gained the ability to process macro definitions in .gitattributes files, in PR git-lfs#4525. However, the "git lfs migrate import" command was not adjusted, specifically in the implementation of its "--fixup" option, which initializes a Tree structure (also of the "git/gitattr" package) for the root tree of each commit in a repository's history using the package's New() function. This function traverses all the trees in the hierarchy and finds and parses all the .gitattributes files in them. Then, when the command visits each file within the commit's tree using the Rewrite() method of the Rewriter structure in the "git/githistory" package, it calls the (*Tree).Applied() method to match the file's path against any applicable Git attributes, to see if the file should be treated as a Git LFS object. This lack of support for macro attributes in the "git lfs migrate import --fixup" command was then propagated to the "git lfs migrate info --fixup" command in commit 4800c5e of PR git-lfs#4501, when the "git lfs migrate info" command was updated to respect the --fixup option. As a result, both of these commands (when used with the --fixup option) panic if they encounter a .gitattributes file with any macro definitions, as they call the (*Tree).Applied() method and it attempts to access the nil Pattern element of the lines with non-nil Macro elements. (Prior to the changes in commit c374d1f of PR git-lfs#5375 the "git lfs migrate import --fixup" command would then stall indefinitely, but it now also exits after the panic condition.) These problems were reported in issue git-lfs#5332. To resolve this problem and avoid similar ones in the future, we refactor the Line structure into a Line interface, which only provides an Attrs() method to retrieve a slice of Attr attributes, and no other methods. We then also define two additional interfaces, each of which embeds the Line interface, PatternLine and MacroLine, with corresponding getter methods for their respective elements. The ParseLine() function of the "git/gitattr" package now returns a slice of generic Line types, each of which is either a PatternLine or a MacroLine, but never both. Callers like the Applied() method of the Tree structure therefore need to perform type assertions or switches to determine which type of Line they are handling, which ensures they always access the line's data through safe methods. We then update the Go tests for the "git/gitattr" package as appropriate, and also add two tests each to the t/t-migrate-fixup.sh and t/t-migrate-import.sh test suites. All four of these new shell tests fail without the changes in this commit. In particular, several of these tests make sure to run the "git lfs migrate" commands outside of any shell pipeline so the test will fail if the command panics and produces no output, even if no output is the expected condition for a successful execution of the command.
This was referenced Jun 1, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We add a
--fixupoption to thegit lfs migrate infocommand which has similar properties to the--fixupoption to thegit lfs migrate importcommand, namely that it looks for files in the repository which should be Git LFS pointers (according to the filter settings in.gitattributesfiles) but which are not.In the case of the
migrate infocommand this new option only reports the sum totals for any such files, and does not make any changes to the repository. When the--fixupoption is supplied, themigrate infocommand will not output a summary of any other files, including.gitattributesfiles, which are otherwise normally included in the command's final output.The new
--fixupoption is taken to imply the--pointers=ignoreoption setting, and is incompatible with any other settings of that option.We add tests of the new option, including invalid option combinations, and also make some related improvements to the existing tests in the
t/t-migrate-info.shscript, notably by breaking up "omnibus" tests added in #4436 into separate tests, and by refactoring all the tests of invalid options so they align with other scripts' and check that the command exits with an expected failure code. We also adjust the text of one such invalid option error message (again from #4436) to align better with the other existing and new error messages for such cases.Fixes #4419.
/cc @kid1412621 as reporter.