-
Notifications
You must be signed in to change notification settings - Fork 126
Howto Release
Current model:
- only 1 released branch (master)
- all releases are tags on master (no branch for, say, 1.8.x releases)
- changing the model requires "mergers" to be informed (eventually to be discussed to the next meeting)
- the current process should take about 2h if everything goes well.
- The person in charge of the release should have accounts on all services involved in the automatic verification of versions and pull requests:
-
github.com:math-comp/math-compandmath-comp/docker-mathcomp, -
gitlab.inria.fr: (math-comp/math-comp(not-yet-created) and)math-comp/docker-mathcomp, -
docker.com:mathcomp.
-
- In addition, this person should be registered as a member of the
math-compcore team on all these sites.
-
(optional but recommended) set up commit/tag GPG signing following instructions at https://help.github.com/en/articles/managing-commit-signature-verification
-
milestone faithful: look for issues and PR with no milestone that should belong to the current release, and give it the right milestone.
-
milestone full: check if all issues/PR in https://github.com/math-comp/math-comp/milestones are fixed/merged (eventually change the milestone of not-fixed/not-merged items)
-
send email to [email protected] telling your colleagues to please don't merge until further notice. Proposed text for this email:
Subject: [IMPORTANT] Release of MathComp v${version} in progress Dear all, We are now starting the release process for mathcomp v${version}. All pull requests targeted to this release have been merged. This message grabs the lock on the master branch. **Please refrain from adding anything to the master branch until the lock is released.** Best wishes, -- ${names of the release managers} -
send the same contents to a new topic
v${version} releasein the stream#math-comp-devs -
edit files
CHANGELOG.mdandCHANGELOG_UNRELEASED.md(the latter need to be generated withdoc/changelog/generate-release-changelog.sh)- update
CHANGELOG_UNRELEASED.mdto regroup entries about the same file together (sort entries by some topological order of compilation). - move the contents of
CHANGELOG_UNRELEASED.mdat the head ofCHANGELOG.mdand update dates and releases - remove all files in
doc/changelog/*that are now inCHANGELOG_UNRELEASED.md - remove
CHANGELOG_UNRELEASED.md - update the links at the top of
CHANGELOG.mdto point-jump to the last releases - check that supported Coq versions are listed
- check whether version numbers in
INSTALL.mdmust be updated - compute and add the list of contributors with the following command (update the names of the tags, here we list contributors from 1.12.0 to now, fixme: apply mailmap also to coauthors):
git log mathcomp-1.12.0.. --use-mailmap | egrep -i "(co-authored-by|author):.*" | cut -d : -f 2 | cut -d '<' -f 1 | sort -u | sed -e 's/ *$//' | tr -s '\n' ','
- update
-
make sure the
*.opamfiles at the root of the repository are accurate, in particular check the dependency in the versions of Coq. -
close milestone (button
Closein the list of milestones). -
CI green
- look for green badge "pipeline" in https://github.com/math-comp/math-comp/
- check if all supported Coq versions are checked by CI
-
Publish the new documentation.
-
Start with setting the version name
V="x_yy_z"(replacex_yy_zby the current release number in all the following) -
Work in a fresh directory, named
$FRESH_DIRECTORYin these notes, e.g.FRESH_DIRECTORY=$(mktemp -d)echo $FRESH_DIRECTORY
-
clone the math-comp web site (math-comp.github.io):
git clone --depth 1 [email protected]:math-comp/math-comp.github.io.git $FRESH_DIRECTORY/math-comp.github.io -
clone the math-comp repository
git clone --depth 1 [email protected]:math-comp/math-comp.git $FRESH_DIRECTORY/math-comp -
create a new branch
cd $FRESH_DIRECTORY/math-comp.github.io && git checkout -b doc_$V -
compile the doc. There are two ways, a quick one using nix and its cache and a slower one which does rebuild mathcomp on your computer.
- if you opt for traditional compilation:
-
cd $FRESH_DIRECTORY/math-comp && make -j 10 doc(it rebuilds the documentation inside directorymathcomp/_build_doc)- building the doc, in particular the browsable graph, requires the lua scripting language see https://github.com/math-comp/math-comp/blob/master/etc/buildlibgraph
- building the doc, in particular putting the comments in coqdoc style, requires the gnu sed
utility (
brew package gnu-sedon OSX), see https://github.com/math-comp/math-comp/blob/master/etc/utils/builddoc_lib.sh
- copy the generated files
cp -r $FRESH_DIRECTORY/math-comp/_build_doc/htmldoc $FRESH_DIRECTORY/math-comp.github.io/htmldoc_$V - copy the common material
cd $FRESH_DIRECTORY/math-comp.github.io && cp -r htmldoc_template/* htmldoc_$V/
-
- if you opt for nix:
- run
cd $FRESH_DIRECTORY/math-comp; rm -f result; nix-build --argstr job mathcomp-doc, it create a symlinkresultpointing to the build result - copy the generated files
cd $FRESH_DIRECTORY/math-comp; cp --no-preserve=mode,ownership -r result/share/coq/*/htmldoc/ $FRESH_DIRECTORY/math-comp.github.io/htmldoc_$V/
- run
- if you opt for traditional compilation:
-
update the symbolic link
htmldocto point to the last version of the documentation:cd $FRESH_DIRECTORY/math-comp.github.io && rm htmldoc && ln -s htmldoc_$V htmldoc -
link the resulting doc in the file
emacs $FRESH_DIRECTORY/math-comp.github.io/index.orgby copy-pasting the line from a previous version andC-c C-lto edit the link -
Run
"emacs" index.org --batch -f org-html-export-to-html --kill(or within emacs typeC-c C-e h h) to generate theindex.htmlfile -
commit changes in math-comp.github.io/htmldoc with
cd $FRESH_DIRECTORY/math-comp.github.io && git add index.org index.html htmldoc htmldoc_$V && git commit -m "htmldoc regenerated for release $V" -
push the branch
git push origin doc_$V -
open a pull-request for branch
doc_$V
-
- Go to github.com/math-comp/math-comp/releases/new in order to draft a new release
- write in the
Tag versiontext entry the version to be released prefixed with "mathcomp-", eg "mathcomp-1.8.0" and leave @ target "master" - title is "The Mathematical Components Library X.Y.Z"
- In the description message list:
- compatible Coq versions
- main changes if any
- convenience link to the
CHANGELOG.mdfile, eg[CHANGELOG.md](https://github.com/math-comp/math-comp/blob/master/CHANGELOG.md)
- if we release a beta, mark as a pre-release, otherwise as a release.
- save the draft release
- check it looks OK in https://github.com/math-comp/math-comp/releases
- edit it one more and this time click "Publish the release"
- Release the lock on math-comp/master by sending a mail to [email protected]
Subject: [IMPORTANT] Release of MathComp v${version} completed Dear all, The release has been tagged. We are releasing the lock on math-comp master. Best wishes, -- ${names of the release managers} - send the same contents to the topic
v${version} releasein the stream #math-comp-devs
To update nix packages do (only the first time):
- On github, make a fork of
https://github.com/NixOS/nixpkgs/ - Clone
git clone --depth=1 [email protected]:your_user_name/nixpkgs.git - Update remotes
git remote add upstream https://github.com/NixOS/nixpkgs/
Then:
- Create a branch for the pull request
git fetch --depth=1 update upstream; git checkout -b update-mathcomp-$V upstream/master - Edit
emacs pkgs/development/coq-modules/mathcomp/default.nix- In the
releaserecord a line like"1.14.0".sha256 = "";(yes, the hash is empty) - In the
defaultVersionrecord set{ case = isGe "8.14"; out = "1.14.0"; }
- In the
- Do a local build
nix-build -A coqPackages_8_14.mathcomp- The build will fail with a message like
hash mismatch in fixed-output derivation '/nix/store/3rz47a815jf97k3fdixfjgm3ryz1llap-source': wanted: sha256:0000000000000000000000000000000000000000000000000000 got: sha256:07yamlp1c0g5nahkd2gpfhammcca74ga2s6qr7a3wm6y6j5pivk9 - Update the sha256 in
default.nix - Rerun the local build (shall not fail this time)
- The build will fail with a message like
- Push
git push origin update-mathcomp-$V - Click on the link printed to open the pull request
- add a comment
FYI @vbgl @Zimmi48 @CohenCyril - if you have the rights add a comment saying
@GrahamcOfBorg build coqPackages_8_14
- add a comment
- test the PR on coq-nix-toolbox following instructions on https://github.com/coq-community/coq-nix-toolbox
- Start with setting the version name
VOP="x.yy.z" - in your math-comp git directory, run
git pull --tags upstreamto synchronize the information about the freshly made release - run
bash etc/utils/packager ${VOP}from the root of the mathcomp repository that generates./opam/released/- note that this tool takes the opam files from the release by default (e.g. in order to get the right versions of Coq at all times).
- if the opam files were changed after the release, you can run
bash etc/utils/packagerwith an extra argument (or use option--help). E.g. you can runbash etc/utils/packager ${VOP} masterif the updated opam files are in your master branch.
- make a "Fork" of https://github.com/rocq-prover/opam in your github account
cd $FRESH_DIRECTORY && git clone [email protected]:rocq-prover/opam.git && cd opamgit remote add ${USERNAME} [email protected]:${USERNAME}/opam.git-
git checkout master && git pull origin master(if you use an old existing copy of the repository instead of the previous line) git checkout -b mathcomp-${VOP}cp -r ~/work/math-comp/opam/released/packages released/git add released && git commit -m 'update mathcomp packages'-
git push ${USERNAME} mathcomp-${VOP}(then follow the link printed by git in order to open the pull request) - in the pull request description, one can use
ci-skip: coq-mathcomp-ssreflect.${VOP} coq-mathcomp-fingroup.${VOP} coq-mathcomp-algebra.1.19.0 coq-mathcomp-solvable.${VOP} coq-mathcomp-field.${VOP}to avoid having the CI compile each package multiple times (thanks Karl Palmskog for the trick, be careful to update all the version numbers, it won't work if the version numbers are not right first time)
- in your math-comp git directory, run
git fetch --tagsto synchronize the information about the freshly made release - write to the coq club and ssreflect mailing lists (currently
[email protected]and[email protected]) a message following this template
Subject: MathComp X.Y.Z released
We are proud to announce the immediate availability of the Mathematical Components library version X.Y.Z.
The webpage, and documentation, are available at https://math-comp.github.io/.
This release is compatible with Coq 8.Y1, 8.Y2 and X.Y
The main changes are [... (if any) ...]
The contributors to this version are: ...
We also wish to thank all the reviewers of the various contributions.
See https://github.com/math-comp/math-comp/releases/tag/mathcomp-X.Y.Z to download or see the `CHANGELOG.md`.
Packages for opam, nix, and docker are in preparation.
Best regards,
--
The Mathematical Components team as well as in the Zulip Coq and mathcomp channels and Coq's discourse (a mail to coq+announcements <[email protected]> would do).
Work on this section should be done only after the pull-request for opam packages has been duly merged. The construction of docker images (see below) cannot happen before.
To prepare the configuration and automatically build the images:
- Clone https://github.com/math-comp/docker-mathcomp.git (
git clone [email protected]:math-comp/docker-mathcomp.git && cd docker-mathcomp) or update your existing copy of this repository (git checkout master && git pull). - Create a branch, e.g.,
git checkout -b mathcomp-1.8.0. - Modify the
images.ymlspecification file by adding one item (or two items, if the new release supportscoq-dev) at the beginning of theimagesYAML list. - Ensure the item(s) you add also contain the following snippet (to enable automatic cross-repo image rebuild using a GitLab CI trigger from Docker-Coq and docker-keeper):
build:
keywords:
- '{matrix[coq]}'
- Remove the
- tag: 'latest-coq-{matrix[coq]}'lines from the previous release in this YAML file, provided the new release is a stable release, not a beta.- (As by convention with Docker tags, the
latestkeyword always denotes a stable release.)
- (As by convention with Docker tags, the
- Commit your changes by doing
git commit -a -m 'New release' - Run
git push -u origin mathcomp-1.8.0(then follow the link printed by git in order to open the pull request). - Wait for the https://gitlab.inria.fr/math-comp/docker-mathcomp/-/pipelines to terminate.
- Several linting/consistency-tests are performed by GitLab CI.
- Review the computed list of tags from the
images.ymlspec by browsing the pipeline jobdebriefand opening theREADMEartifact URL. - FTR, another standard way to retrieve the generated artifacts amounts to browse list of all pipelines, and click on the right-hand-side icon (see screenshot below)
- NB: Rendering in your web browser might slightly differ
- As soon as the new release package is available in opam-coq-archive, Merge the docker-mathcomp PR in master, which will generate a new pipeline with an additional (deploy) step to https://hub.docker.com/r/mathcomp/mathcomp
- (Access to docker.com might be slow in general)
- When the pipeline associated with the PR terminates:
- click on the "notify / debrief" job (the last one of the pipeline)
- to reach for the textual output of the run of the virtual machine
- follow the steps to remove potential obsolete tags
- (E.g.,
latest-coq-8.7if compatibility with Coq 8.7 has been dropped by the new release.) - Concretely, this means: select the tags to be removed and choose the "remove" action using the webpage interface of docker.com.
- (E.g.,
- upload the generated README.md to https://hub.docker.com/repository/docker/mathcomp/mathcomp/general
- by clicking on
Editat the bottom of the docker.com webpage. ⚠️ NB: this documentation step is important, otherwise end users may believe that the mathcomp release is not available on Docker Hub.
- by clicking on
- click on the "notify / debrief" job (the last one of the pipeline)