Skip to content

Fix packaging workflows on master#3086

Merged
jdtournier merged 9 commits intoMRtrix3:masterfrom
daljit46:fix_master_packaging
Apr 25, 2025
Merged

Fix packaging workflows on master#3086
jdtournier merged 9 commits intoMRtrix3:masterfrom
daljit46:fix_master_packaging

Conversation

@daljit46
Copy link
Member

@daljit46 daljit46 commented Apr 11, 2025

This PR fixes all of our packaging workflows in preparation for the 3.0.5 release. Main changes:

  • Windows workflow now uses the same Github Action used on dev to set up an MSYS2 environment (MINGW64 instead of UCRT64 for compatibility reasons). Additionally, an explicit addition of -lopengl32 linker flag has been made (otherwise the build was failing).
  • The MacOS conda workflow now uses the conda-incubator/setup-miniconda to set up a miniconda environment.
  • Qt 5.15.2 is used for the MacOS native build (we may want to discuss this further).
  • Extensions of release artefacts have been changed to match the new standards used by package managers (conda and pacman).

Since GitHub Actions are painful to debug, I tested these on a separate fork of MRtrix3. The workflows seem to produce the expected results with the exception of the MacOS native build script, which is configured to directly clone and build the MRtrix3 repo (the release tag that triggers the workflow must be present in the MRtrix3 official repo). We should probably do a test release to check that all of them work as intended.

Closes #3007.

- Use Github Action to setup miniconda
- Hardode repo name for packaging script
- Install MSYS2 environment using Github Action
- Remove qtbinpatcher step
New standard for pacman now uses zstd rather than .tar.xz
@daljit46 daljit46 requested a review from a team April 11, 2025 19:00
@daljit46 daljit46 modified the milestone: 3.0.5 updates Apr 11, 2025
@daljit46 daljit46 self-assigned this Apr 11, 2025
@Lestropie Lestropie added this to the 3.0.5 updates milestone Apr 15, 2025
Lestropie
Lestropie previously approved these changes Apr 17, 2025
Copy link
Member

@Lestropie Lestropie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very difficult one to review. Technically could look at the results you obtained on your fork, but I'm trusting that you've verified everything as well as can be. It's probably a matter of re-verifying as we generate the tag and correcting anything that comes up before announcing that the version is available. So if you're content yourself, I'm content to merge whenever you see fit.

@daljit46
Copy link
Member Author

daljit46 commented Apr 17, 2025

Further testing of the MacOS native workflow (the only one I couldn't verify because of the reason mentioned above) seems to show some problems:

  • In order to minimise build conflicts, the build script strips PATH to: /bin:/sbin:/usr/bin:/usr/sbin. This causes a failure because the configure/build scripts can't find Python. A (hacky but workable) solution is to symlink Python in a local directory and add that to PATH when running the build script.
  • There seems to be an error in linking Qt.

So will need to solve this before merging, will convert this PR to draft until these issues are solved.

@daljit46 daljit46 marked this pull request as draft April 17, 2025 13:44
@bjeurissen
Copy link
Member

  • In order to minimise build conflicts, the build script strips PATH to: /bin:/sbin:/usr/bin:/usr/sbin. This causes a failure because the configure/build scripts can't find Python. A (hacky but workable) solution is to symlink Python in a local directory and add that to PATH when running the build script.

The macOS bundled python version should be in the default PATH, no? Is this not a python2 vs python3 issue? As far as I know python3 should be in the default PATH.

The main reason I reset the PATH to the bare macOS path is to make sure we only rely on OS-bundled libs and do not accidentally use third party libs (from homebrew, cmake installations, custom python installs, etc), which can’t be relied on for a binary installer.

@daljit46
Copy link
Member Author

daljit46 commented Apr 17, 2025

Is this not a python2 vs python3 issue?

Sorry, yes that's correct. However, we can't create a symlink in /usr/bin so either we have to call python3 ./configure (similarly for build) inside the script or we can use the solution I proposed above.

We symlink /usr/bin/python3 in $PWD/python_dir and pass this folder
as an extra path to the build script. This allows MRtrix3's build and
configure script to use Python3 to build the project.
To build universal binaries with MacOS we need at least Qt 5.15.9
@daljit46
Copy link
Member Author

I have now updated the MacOS build script to support the building of universal binaries. To do this, I have upgraded the Qt version to 5.15.16 (the support for building universal binaries was introduced in version 5.15.9). I have tested the script locally and also using a test workflow on GitHub Actions. It seems to produce the expected results.
Additionally, to obviate the Python issue as outlined above, the workflow now creates a local symlink to /usr/bin/python3 and passes this to the build script.

I think this can now be merged.

@daljit46 daljit46 marked this pull request as ready for review April 24, 2025 19:02
@jdtournier jdtournier added this pull request to the merge queue Apr 25, 2025
Merged via the queue into MRtrix3:master with commit bb36e36 Apr 25, 2025
5 checks passed
@tclose tclose mentioned this pull request Jun 22, 2025
@Lestropie Lestropie mentioned this pull request Sep 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix CI release workflows for master branch

4 participants