Multiple hash support #4971
Merged
Multiple hash support #4971
+93
−8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Right now, we provide signed SHA-256 hashes for our releases. This is fine and sufficient, and also cryptographically secure. However, many distributors use other algorithms, and it would be convenient if we could provide easy access to those hashes as well. For example, NetBSD uses SHA-512 and BLAKE2s. Let's add a script to hash files with various algorithms and output them in the BSD format. The advantage of the BSD format over the traditional GNU format is that it includes the hash algorithm, which allows us to distinguish between hashes of the same length, such as SHA-256, SHA-512/256, and SHA3-256. It is generated by shasum, sha*sum, sha3sum, and b2sum with the --tag format, and all of these programs accept it for verification with no problems. Using the BSD format means that we need only provide one additional file with all the additional algorithms. There is therefore no need to add multiple new files, and if we desire to add additional algorithms in the future, that's easily done without modification. For aesthetics, we sort first by hash name and then by filename in the output. Unlike sorting with `sort`, this keeps the SHA-2 and SHA-3 algorithms separate instead of interspersing them, which aids in reading. Add some comments because the algorithm, while logical, is somewhat subtle.
Right now, we provide signed SHA-256 hashes for our releases. This is fine and sufficient, and also cryptographically secure. However, many distributors use other algorithms, and it would be convenient if we could provide easy access to those hashes as well. For example, NetBSD uses SHA-512 and BLAKE2s. Let's add an additional file, hashes.asc, which contains a general set of hashes in the BSD format. The advantage of the BSD format over the traditional GNU format is that it includes the hash algorithm, which allows us to distinguish between hashes of the same length, such as SHA-256, SHA-512/256, and SHA3-256. It is generated by shasum, sha*sum, sha3sum, and b2sum with the --tag format, and all of these programs accept it for verification with no problems. Using the BSD format means that we need only provide one additional file with all the additional algorithms. There is therefore no need to add multiple new files, and if we desire to add additional algorithms in the future, that's easily done without modification. If the user has sha3sum (which comes from Perl's Digest::SHA3) or b2sum (part of GNU coreutils), then we use them to verify our hashes. There are no known commands available on a typical Linux system to verify BLAKE2s, but we assume that if OpenSSL and our Ruby script correctly generated the SHA-2 entries, then it will also have properly generated the other hashes as well. Since we must now run programs inside the repository, we need to know where that file is located, and therefore we use git to find the root of the repository and now must run within the repository. Since this script is only run by Git LFS core team members or the CI system when doing releases, this is not expected to be an issue.
Indicate that there are also additional hashes in another file in the README.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Right now, we provide signed SHA-256 hashes for our releases. This is fine and sufficient, and also cryptographically secure. However, many distributors use other algorithms, and it would be convenient if we could provide easy access to those hashes as well. For example, NetBSD uses SHA-512 and BLAKE2s.
Let's add an additional file, hashes.asc, which contains a general set of hashes in the BSD format. The advantage of the BSD format over the traditional GNU format is that it includes the hash algorithm, which allows us to distinguish between hashes of the same length, such as SHA-256, SHA-512/256, and SHA3-256. It is generated by shasum, sha*sum, sha3sum, and b2sum with the --tag format, and all of these programs accept it for verification with no problems.
Using the BSD format means that we need only provide one additional file with all the additional algorithms. There is therefore no need to add multiple new files, and if we desire to add additional algorithms in the future, that's easily done without modification.
I don't love the name
hashesand would prefer something different, but I am, as usual, bad at naming things, so suggestions are welcome.Fixes #4937