-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Update release process documentation and scripts #5920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update release process documentation and scripts #5920
Conversation
Before updating the documentation of our release process, we first adjust the indentation of several paragraphs to match the preceding ones.
Before updating the documentation of our release process, we first correct a minor misspelling.
In commit d61b3ea of PR git-lfs#5607 we added support in our Makefile and scripts to build and upload a version of our Linux release assets for the LoongArch64 platform, and at the same time we added an entry for this platform to the end of the list of Linux assets in our release process documentation. Before updating the documentation of our release process, we first reorder this list to keep it in alphanumeric order, so as to match how assets are listed in our public release notes.
In PR git-lfs#5438 we added support in our Makefile and scripts to build and upload a version of our Linux release assets for the RISC-V 64 platform. As the documentation of our release process includes a list of the Linux assets we build, we now add an entry for the RISC-V 64 platform so as to match the assets listed in our public release notes.
Based on the recent experience of releasing the 3.6.0 version of Git LFS, we update several steps in our documentation to more accurately reflect our current release process. First, we add a reminder that we should include in our release notes any changes from the previous Git LFS release in terms of platform requirements or the set of operating system versions we support. As well, we more fully document the dependencies of our script/finalize script, and note that a PR to update Homebrew should now be generated automatically within a few hours of our making a new release public.
Before we release a new version of Git LFS, we generally want to upgrade the version of Go used in the Docker container images generated by our git-lfs/build-dockers repository, so we add notes regarding this step to our release process documentation. Although Markdown does not require us to renumber the items in an ordered list, we do so anyway as this makes our documentation more readable when viewed as a plain text file.
Our script/hash-files script is executed by our script/upload script to generate hashes of the asset files built by our release process, so that we can then sign the resulting "hashes" file with a GPG signing key and publish the signed file in our release announcement. This script makes use of the OpenSSL::Digest Ruby gem to perform hashing with a variety of algorithms, including BLAKE2b, BLAKE2s, SHA-512, and others. On macOS, a number of these digests require the Ruby 3.x interpreter and "openssl" gem available via Homebrew rather than the Ruby 2.x versions provided by current versions of macOS; otherwise, the script returns "first num too large" errors from OpenSSL. We therefore add a simple check to our script which exits with a non-zero code and an error message if the Ruby 3.x interpreter is not in use.
The verify_assets() function in our script/upload script performs a final verification of the contents of the GPG-signed "sha25sum.asc" and "hashes.asc" files we include in the assets attached to our public release notes. To perform these checks, it runs the shasum(1) utility for all the SHA-2 hashes, the sha3sum(1) utility for the SHA-3 hashes, and the b2sum(1) utility for the BLAKE2b hashes, passing each utility the appropriate lines from the "hashes.asc" file. These lines have the format output by our script/hash-files script, namely: <hash-name> (<file-name>) = <hash-value> While this format is accepted and parsed successfully by all of the utilities on most Linux systems, the sha3sum(1) program available on macOS via Homebrew does not permit this input format with its -c option. Instead, it requires as input exactly the same format as it outputs when hashing a set of files, i.e.: <hash-value> <file-name> This disparity is due to the fact that Homebrew packages as the "sha3sum" utility a different implementation than is packaged for Debian Linux. Specifically, the Homebrew package is built from the following source project: https://codeberg.org/maandree/sha3sum https://formulae.brew.sh/formula/sha3sum On Debian, however, the sha3sum(1p) utility is a Perl script provided with the Digest::SHA3 Perl extension: https://metacpan.org/dist/Digest-SHA3/view/sha3sum https://sources.debian.org/src/libdigest-sha3-perl/ https://manpages.debian.org/unstable/libdigest-sha3-perl/sha3sum.1p.en.html This Perl script accepts a wider range of input formats than the program packaged by Homebrew, including the one created by our script/hash-files script. It is also able to deduce the SHA-3 algorithm to use based on the hash name in each line, which the sha3sum(1) utility on macOS is unable to do. As we sometimes want to run our scripts on macOS when generating a new Git LFS release, we adjust our script/upload script so it rewrites the lines from the "hashes.asc" file into the format accepted by the Homebrew version of sha3sum(1), and also so it invokes that utility separately for each hash algorithm. As this also works with the Perl script version of sha3sum(1p), our script/upload script should now succeed on both macOS and Linux in the future.
| and then use that branch as the base for the PATCH release. | ||
|
|
||
| 1. If the `release-M.N` branch does not already exist, create it from | ||
| 1. Upgrade or downgrade the Go version used in the `git-lfs/build-dockers` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When would we downgrade?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The most obvious situation occurs when Go releases a new minor (or major) version which drops support for older platforms, and we've moved our development branches ahead to keep pace with that.
However, we normally want our patch releases to support the same set of platform versions as all the other Git LFS releases we've made in that series, so we generally don't want to upgrade to a new minor (or major) version of Go for a patch release, and might have to downgrade Go temporarily as a result.
This PR updates our "Releasing Git LFS" documentation page to reflect the current state of our release process, and also adjusts two of the scripts used in that process so they function successfully on macOS systems (or provide more useful error messages if the necessary dependencies are not available).
In particular, our
script/uploadscript expects that thesha3sum(1)utility is available and that it accepts input in the format generated by ourscript/hash-filesscript, i.e.:However, the version of the
sha1sum(1)program packaged and installed by Homebrew is different from the one packaged by Debian, which is a Perl script that accepts a wider range of input formats, including the format output by ourscript/hash-filesscript. The program provided by Homebrew'ssha3sumformula requires input in the following format, and is not able to determine the appropriate algorithm to use without it being specified by an-acommand-line option.Therefore we adjust our
script/uploadscript so it is able to function with either implementation of thesha1sum(1)utility, since the Perl script also accepts the same format as the Homebrew version.This PR will be most easily reviewed on a commit-by-commit basis, with whitespace changes ignored.