I'm just playing around here with the 2nd commit for now, the ldc-build-runtime one - will be extracted to a follow-up PR if we wanna go that route. We should probably have an explicit CMake var for the .conf base name (or do the rename in the ldc-build-runtime tool, but then the CMake-install might overwrite an existing 50-target-default.conf...), and have the tool do the triple regex patching, by having the user specify it in the ldc-build-runtime cmdline.

Rough sketch for a future ldc-add-target tool (or so), extending a local LDC installation by some cross-compilation target based on the prebuilt official packages, without building anything yourself (that's related ldc-build-runtime):

Tasks for a ldc-add-target windows-arm64 example (<target name> being windows-arm64):

1. Download the according official release package (matching the LDC version, like ldc-build-runtime optionally downloading the associated source package) containing the libs for the desired target: ldc2-1.41.0-windows-multilib.7z
2. Unpack it and move the desired libs subdir to the local LDC installation, renaming to lib-<target name>.
3. Move the matching 5X config file from extracted etc/ldc2.conf/ to the local LDC installation's etc/ldc2.conf/, renaming to 55-target-<target name>.conf and patching the triple regex. Edit: Oh, and also patching the lib-dirs - just the new name, as they are all portable already with the %%ldcbinarypath%%/../ parent.
4. Optionally allow appending extra switches provided in the command-line to the config file, such as a -gcc=aarch64-linux-gnu-gcc for the linux-aarch64 target.

Would need a mapping for each supported target, such as:

windows-arm64:
    artifact_suffix: "windows-multilib.7z"
    artifact_lib_dir: "lib-windows-arm64"  # would be "lib64" for the windows-x86_64 target
    artifact_conf_file: "55-target-windows-arm64.conf"  # would be "50-target-default.conf" for the windows-x86_64 target
    conf_triple_regex: "(aarch|arm)64-.*-windows-msvc"  # only needed for 50-target-default.conf's `default`, so trivial to patch

Then all a user would additionally need is the according C cross-compiler for non-Apple Posix targets (apt install gcc-aarch64-linux-gnu).

Optionally allow appending extra switches provided in the command-line to the config file, such as a -gcc=aarch64-linux-gnu-gcc for the linux-aarch64 target.

That would work with gcc, but a user may have aarch64-unknown-linux-gnu-gcc installed in a non-PATH location, that's one edge-case we need to consider. This also wouldn't work with clang, as it would require -gcc=clang and -Xcc=-target==aarch64-unknown-linux-gnu. However, adding -target in the config file would now break cross setups using gcc as it doesn't understand -target.

This is another point that I want to make. When cross-compiling, you're building things for another system. The issue here is that you can build for multiple other systems, and those systems may require different configs. Maybe on one system you need to use clang -target aarch64-linux-gnu as CC and for another build you want to use aarch64-linux-gnu-gcc. This is an argument for not configuring CC in the config files, as they can't easily be altered on a per-build basis.

Another, albeit less CC-focused argument but still relevant to the config files, is rpath. In its %%ldcbinarypath%%/../lib-arm64 form it's almost always wrong. The correct value depends on the system you're installing for, which, again, may not be singular and is not known at ldc-runtime build time. In my case at least, as part of a portage build, I would need the config files to reference:

lib-dirs = [ "/usr/aarch64-unknown-linux-gnu/usr/lib/ldc2/1.41/lib64" ]
rpath = "/usr/lib/ldc2/1.41/lib64"

Should this be something that ldc-build-runtime or ldc-add-runtime need to support?

Optionally allow appending extra switches provided in the command-line to the config file, such as a -gcc=aarch64-linux-gnu-gcc for the linux-aarch64 target.

There's also an important precedent here. The config file can, technically, contain anything, but people will assume that only some flags may be present. If we make the decision to embed -Xcc and -gcc in the config file how would a build system cope with them? Should it always assume that ldc may not respect CC, unless a -gcc= is given, and that it may pass unsupported flags to the CC the build systems knows about? I really don't want this to end up like dmd respects DFLAGS, but not the ones you set, it only respects the DFLAGS that are internally set by the config file, well, unless there's no config file. It's misleading for anyone who only reads the documentation, and doesn't check the actual code.

Now, I think I also understand your points. Those being that most of my above concerns are very niche edge cases, and, so long as it is technically possible to support them (like by passing -gcc=), it's worth doing something that works for the majority of people. For example a aarch64-linux-gnu-gcc as CC system with -link-defaultlib-shared=false on a linux host. My issue with this is that it may be better not to give users this simple out of the box configuration and, instead, require them to use a build system or, emulate one themselves. This would be more complex at first but users would have better documentation to follow and they would be more likely to understand what they actually need for their specific application's cross build. This may be better than giving users a basic setup that works for a hello-world program but which fails with importC or an external library dependency.

However https://wiki.dlang.org/Cross-compiling_with_LDC does document some of the things I'm arguing against. Anyway, I haven't drawn a conclusion yet, I'm still trying to understand the consequences of each choice.

I've asked around a meson dev what he think of the changes and the short summary is: it would be great if this sort of CC detection didn't overwrite the CC from the environment. Suggestions to achieve this included trying to look for <triple>-gcc instead of just cc when CC is unset, or adding another key to the config file. I think the former sounds like a better idea.

Yeah in essence I think the only controversial point here is the -gcc, and whether/where to add it to the config. As this remains a manual step, not done as part of ldc-build-runtime, people are free to do as they please. My perspective here is solely on the official packages (and so affecting ldc-add-target/runtime), where the -gcc would only come in for the Android x86[_64] cross-targets [which didn't have any config sections so far; I was too lazy, and noone complained] - and yeah, the FreeBSD package also sets -gcc as you've seen (edit: but independent from the target), but only to make sure that the same-LLVM-versioned clang is used (IIRC, that was required for compiler-rt support for CI at one point, but might not be needed anymore after your improvements there).

And my stance there wrt. -gcc is to provide a sane default, so that -mtriple=… alone suffices for cross-linking. So while I obviously have no idea where the guys installed their Android NDK, I think it's good that we still default to the expected clang wrapper (same name on Windows and Linux) at least. Otherwise I guess on Windows you'd get an error saying that it cannot find gcc for linking; aarch64-linux-android30-clang is much better in that regard - might be in PATH, yay, or otherwise provides much better context for what's required (and could be fixed once by the user specifying the full path in the config). And IIRC, those Android clang wrappers are just bash/cmd.exe scripts, no need to provide a -target … switch for some generic clang.

So similarly, when invoking ldc-add-target linux-aarch64 to add that cross-compilation target, I wanna default to an aarch64-linux-gnu-gcc, not a generic gcc. But that's just an ldc-add-target detail (not in the official native Linux-aarch64 package's conf).

LDC package maintainers for distros such as you are free to do what they want, tweaking the config files to their system, and maybe even patching the ldc-add-target tool accordingly.

The only problematic aspect of defaulting to a -gcc is, as you mentioned, that it overrides a CC set up by some build system. Which can be avoided by passing -gcc= in the cmdline, something the build system could do. Adding something inbetween, like a -defaultCC, which would NOT override CC, isn't really something I'd like, just for this rather theoretical problem - the main 'build systems' for D (dub and reggae) don't set CC. Edit: Oh and I'm pretty sure reggae doesn't support any cross-compiles - for dub, it's dub --target=<triple>, passing along that triple to LDC, and depending on the LDC config to set up the rest.

Another, albeit less CC-focused argument but still relevant to the config files, is rpath. In its %%ldcbinarypath%%/../lib-arm64 form it's almost always wrong. The correct value depends on the system you're installing for, which, again, may not be singular and is not known at ldc-runtime build time. In my case at least, as part of a portage build, I would need the config files to reference:

lib-dirs = [ "/usr/aarch64-unknown-linux-gnu/usr/lib/ldc2/1.41/lib64" ]
rpath = "/usr/lib/ldc2/1.41/lib64"

Should this be something that ldc-build-runtime or ldc-add-runtime need to support?

IIRC, ldc-build-runtime produces absolute paths to the LDC installation, as CMAKE_INSTALL_PREFIX is set to the LDC's install prefix, which would most likely be /usr/aarch64-unknown-linux-gnu/usr/lib/ldc2/1.41 in your example. So if the rpath somehow needs to be different, we could add an according CMake var (RPATH_INSTALL_PREFIX or so), but I wouldn't support it directly in the tools, because I think that's rather exotic, and at least for ldc-build-runtime, setting any CMake var in the cmdline isn't rocket science.

Edit: But you could patch ldc-build-runtime to automatically set up the rpath correctly for Arch Linux.

So if the rpath somehow needs to be different, we could add an according CMake var (RPATH_INSTALL_PREFIX or so)

I wouldn't want that because, in my case, I mostly care about cross runtime builds in the sense that they would be installed on a cross-system. When they are initially built, the files do end up in a /usr/aarch64-unknown-linux-gnu/usr/lib/ldc2/1.41/lib64 directory on the build machine but the rpath in the config files is right to point to /usr/lib/ldc2/1.41/lib64 because that's where the libraries will be on the host, at runtime.

I do use this form of cross builds and I would like the cmake build to able to create an exact copy of what a native build on the target system would have produced, and whatever ldc-build-runtime / ldc-add-runtime or other internal cmake logic decide do to, would be done of top of an already completed build. This matches what you suggested for ldc-add-runtime, to extract and generate a config from a release archive for another target, my only suggestion is being able to produce those "another target" archives, both natively and on a different build machine (we can only support the runtime at first, since that has a simpler build)

The only problematic aspect of defaulting to a -gcc is, as you mentioned, that it overrides a CC set up by some build system. Which can be avoided by passing -gcc= in the cmdline, something the build system could do. Adding something inbetween, like a -defaultCC, which would NOT override CC, isn't really something I'd like, just for this rather theoretical problem - the main 'build systems' for D (dub and reggae) don't set CC. Edit: Oh and I'm pretty sure reggae doesn't support any cross-compiles - for dub, it's dub --target=, passing along that triple to LDC, and depending on the LDC config to set up the rest.

I very much dislike the idea of making ldc not respect CC and I much more prefer something akin to (untested and not 100% what is desired):

diff --git i/driver/tool.cpp w/driver/tool.cpp
index 6e25c4ce8c..929364a600 100644
--- i/driver/tool.cpp
+++ w/driver/tool.cpp
@@ -86,7 +86,12 @@ std::string getGcc(std::vector<std::string> &additional_args,
   // Posix: in case $CC contains spaces split it into a command and arguments
   std::string cc = env::get("CC");
   if (cc.empty())
+  {
+    auto crossGcc = global.params.targetTriple->getTriple() + "-gcc";
+    if (llvm::sys::findProgramByName(crossGcc))
+      return getProgram(crossGcc.c_str(), &gcc);
     return getProgram(fallback, &gcc);
+  }
 
   // $CC is set so fallback doesn't matter anymore.
   if (cc.find(' ') == cc.npos)

I very much dislike the idea of making ldc not respect CC and I much more prefer something akin to

Okay, making the compiler choose a somewhat sane default cross-CC when explicitly providing an -mtriple option is definitely on the table. We'd have to come up with some rules, like doing that for non-Apple/Windows targets only, and maybe stripping the vendor automatically (or simply assuming the user won't specify it for those targets). Edit: And probably including some gcc vs. clang logic, so that we use clang for Android (and FreeBSD?).

I very much dislike the idea of making ldc not respect CC and I much more prefer something akin to

Okay, making the compiler choose a somewhat sane default cross-CC when explicitly providing an -mtriple option is definitely on the table. We'd have to come up with some rules, like doing that for non-Apple/Windows targets only, and maybe stripping the vendor automatically (or simply assuming the user won't specify it for those targets). Edit: And probably including some gcc vs. clang logic, so that we use clang for Android (and FreeBSD?).

Alright then, how do we want the list of lookups to be like?

• Arch uses aarch64-linux-gnu-gcc: https://archlinux.org/packages/extra/x86_64/aarch64-linux-gnu-gcc/files/
• Debian uses aarch64-linux-gnu-gcc: https://packages.debian.org/bullseye/amd64/gcc-aarch64-linux-gnu/filelist
• Gentoo uses aarch64-unknown-linux-gnu-gcc

clang here doesn't come with a aarch64-unknown-linux-gnu-clang script and I can't find anything on debian and arch as well so I assume it similarly doesn't exist there.

The other OSes I don't know about.

Thx for checking. - Oof, so Gentoo does include the nonsensical vendor. I'd go with NOT tampering with the vendor then, so that it works on Gentoo when using -mtriple=aarch64-unknown-linux-gnu, and otherwise using the -mtriple=aarch64-linux-gnu.

And wrt. clang vs. gcc, I'd only use clang where we know it's the only or default supported compiler. I.e., Android for recent NDKs, and maybe FreeBSD if it comes with these wrappers; no idea where else clang is the default by now (=> hope for PRs from users if they care about it).

Okay Andrei, ready now from my side. Please let me know if you can live with this, or what you'd like changed.

Okay Andrei, ready now from my side. Please let me know if you can live with this, or what you'd like changed.

Everything looks great. The only nit that I can struggle to find is that we started installing a .conf file, which comes with the section triple as default, which may mess up setups in which users were creating the config manually, and now their settings would be overwritten. But you did mention this in the Changelog so, if you don't consider this too much of a downside, I won't comment on it either.

The only nit that I can struggle to find is that we started installing a .conf file, which comes with the section triple as default, which may mess up setups in which users were creating the config manually, and now their settings would be overwritten.

Thx for the review! How do you propose we improve on that nit? We could bail out in ldc-build-runtime, if a non-empty install suffix was specified without also specifying RT_CONF_TRIPLE_REGEX; as I can't think of why anyone would want to install a 55-target-<suffix> with a default triple.

The only nit that I can struggle to find is that we started installing a .conf file, which comes with the section triple as default, which may mess up setups in which users were creating the config manually, and now their settings would be overwritten.

Thx for the review! How do you propose we improve on that nit? We could bail out in ldc-build-runtime, if a non-empty install suffix was specified without also specifying RT_CONF_TRIPLE_REGEX; as I can't think of why anyone would want to install a 55-target-<suffix> with a default triple.

Yes, something like that. If the users wants to install the runtime but doesn't pass a RT_CONF_TRIPLE_REGEX (or a --conf-triple-regex options if we decide to promote the switch) we can just fail early. This way, even if some setups may start failing, users will be made aware of the change and be able to decide how to fix it: either let the 55-target-<suffix> file be installed (which is probably what most want) by passing the config triple; or specifically disable installing the config file with -DRT_CONF_DIR=. So, just don't silently allow the possibly incomplete config file to be installed, either keep its installation opt in, or install it by default but fail early if the triple is not specified.

And, if someone really wants the default triple they can still pass -DRT_CONF_TRIPLE_REGEX=default.

Yeah okay then I'll add that bail-out soon-ish, makes sense to me too; the expanded help text is probably too buried by itself.