Firstly simplify the script please:

script = raw"""
cd $WORKSPACE/srcdir/spglib
args=""
If [[ ! -z "${CMAKE_TARGET_TOOLCHAIN}" ]]; then
  args="${args} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN"
fi
cmake -B ./build \
      -DCMAKE_INSTALL_PREFIX=${prefix} \
      ${args}
cmake --build ./build -j${nproc}
cmake --install ./build
"""

Secondly it is generally encouraged to run the test-suites before doing an install (if it's not cross-compiled). So add:

• a ctest --test-dir ./build between build and install instruction, guarded by an if [[ -z "${CMAKE_TARGET_TOOLCHAIN}" ]] to check that it is not cross-compiled
• add a dependency on the gtest library (devel package, not just runtime)

If there really is a policy against testing (whichit shouldn't) than disable the tests with SPGLIB_WITH_TESTS. At least add the disable to the if statement shown in the script.

Thirdly I think it's supposed to be CMAKE_TOOLCHAIN_FILE.

Finally do you use shared or static linking in julia? That one is controlled SPGLIB_SHARED_LIBS.

Also note, openmp support is disabled by default and there might be plans to drop it

Secondly it is generally encouraged to run the test-suites before doing an install (if it's not cross-compiled). So add:

In Yggdrasil scripts, we basically always are cross compiling the libraries/executables, and there is always a CMake toolchain file present. We usually disable the tests for any binary package built, and it is up to the Julia package using the library to effectively test the libraries through their wrappers, since the packages should have CI on the various supported platforms.

Hmm, that is rather unfortunate. Ideally there would be some infrastructure to trigger a build against upstream main, PR etc. so that upstream can be made aware if there are any packaging issues before submitting a release. But this indirectness complicates that workflow.

I tried to install many different versions of GoogleTest (v1.8, v1.11, v1.12.1, v1.14), but failed. I couldn't even build GoogleTest. I copied the build script of GoogleTest from here. But it did not work (both on my computer and the CI system). Could you please help me?

The errors can be found here: https://buildkite.com/julialang/yggdrasil/builds/5693#018af464-074f-4fe4-8357-c3e167022c71 with the current commit.

Thanks that quite instructive. Indeed the patch worked, it's just stuck at another section which I think I know how to fix.

Anyway from what @imciner2 said, we can't do the tests anyway unless there is some emulation. So you can add -DSPGLIB_WITH_TESTS=OFF in the if statement. Put it there for now just to confirm that it is always cross-compiling.

@singularitti @imciner2 In the failing CI, what compiler is used? _Thread_local should be available from C11, but it seems it is not picked up.
https://github.com/spglib/spglib/blob/a96d72a432f2cccb1c6e8f4d4352fb5ba80b15e3/src/spglib.c#L64-L74

In the failing CI, what compiler is used?

The default is GCC 4.8.5.

_Thread_local should be available from C11, but it seems it is not picked up.

Don't trust compiler support for this necessarily. We have actually just been looking at this for a different piece of software, and even new toolchains for macOS don't support this attribute. You are more likely to get support for __thread than _Thread_local, probably.

As for support for that attribute directly in GCC, according to https://gcc.gnu.org/wiki/C11Status, it requires GCC 4.9+. You can request a compiler newer than that for the build by adding the preferred_gcc_version = v"5" to the build_tarballls command.

@imciner2 Also, is LLVMOpenMP_jll needed? According to @LecrisUT:

openmp support is disabled by default and there might be plans to drop it

Honestly GCC 4.x support should be dropped. @singularitti could you make it use at least a GCC10, or at least any GCC version that the current LTS OS are using?

Honestly GCC 4.x support should be dropped. @singularitti could you make it use at least a GCC10, or at least any GCC version that the current LTS OS are using?

The general policy in the Yggdrasil recipes is that the oldest working GCC version should be used, because that will ensure that the compiled libraries have the best compatibility across all the varying installs of Julia. In this case, since the restriction is the need for C11's _Thread_local support, GCC 5 is appropriate to use instead of the default.

@singularitti

Also, is LLVMOpenMP_jll needed?

If you don't want to enable the OpenMP support, then you don't need the dependencies that provide the libraries for it (I believe you should be able to remove both LLVMOpenMP_jll and CompilerSupportLibraries_jll from the dependencies then). We don't really have an opinion on what features are used for what libraries usually, since that is best left to the people who maintain their Julia-facing components. So the question of using OpenMP or not is up to you/the Julia community using this tool. (I will note that I don't see any upstream changelog mention that OpenMP was disabled in the new release, and tracing it back through the git commits shows it was done in a commit labelled as a docs update with no further explanation, and the docs seem to confuse OpenMP with MPI in their description of the CMake option).

The general policy in the Yggdrasil recipes is that the oldest working GCC version should be used, because that will ensure that the compiled libraries have the best compatibility across all the varying installs of Julia.

Every downstream have their own standards. But beware that these versions are not maintained and there can be bug, security flaws etc. even if the code compiles. Following the LTS distros at least guarantees that there are people maintaining these.

Also it is not guaranteed that if you build with an old gcc thqt the binary will still run on a system with more recent glibc, especially for releases that are more than 10 year old. If there are users that require such old compilers they generally have countless other issues, and since the packages are not properly tested, building with those old compilers has very little benefit.

Also it is not guaranteed that if you build with an old gcc thqt the binary will still run on a system with more recent glibc

That's why we build with an old glibc.

@LecrisUT I've updated the script using the latest GCC I can use on the platform (12.1.0), and it builds on every platform on my computer. Let's see if it works on the CI.

@LecrisUT, Regarding @imciner2's comment:

I will note that I don't see any upstream changelog mention that OpenMP was disabled in the new release, and tracing it back through the git commits shows it was done in a commit labelled as a docs update with no further explanation, and the docs seem to confuse OpenMP with MPI in their description of the CMake option.

I too did not find a notice about this. Could the Spglib team add a comment in the docs or somewhere else?

I too did not find a notice about this. Could the Spglib team add a comment in the docs or somewhere else?

It's an internal plan. The issue is that it is not tested, and there is very little to gain for the user out of parallelizing with openmp. To reduce complexity and other unexpected parallelization overhead for downstream we encourage to phase it out.

There are other issues that need to be addressed, refactoring the test-suite, thread sanitization (e.g. when run in mpi), modernizing python tools and improving the C/C++ interference for better memory and error management. Maybe after that openmp implementation can be revisited.

and the docs seem to confuse OpenMP with MPI in their description of the CMake option.

Oh did not catch that typo. It is not a confusion with mpi, it is referring to openmp. Checked for other mpi typos and thankfully that was the only one.

All checks have passed, good!

Is this PR ready to be merged?

Great, thank you all for your help!