Skip to content

cling: 1.0 -> 1.2#439741

Merged
emilazy merged 2 commits intoNixOS:masterfrom
codedownio:cling-1.2-final
Sep 4, 2025
Merged

cling: 1.0 -> 1.2#439741
emilazy merged 2 commits intoNixOS:masterfrom
codedownio:cling-1.2-final

Conversation

@thomasjm
Copy link
Contributor

@thomasjm thomasjm commented Sep 2, 2025

This actually makes the derivation cleaner because upstream improved the build process somewhat.

Also fixes an issue on Darwin where the Jupyter kernel wasn't being copied to the output.

Putting up for review; it's not ready to merge yet because updating LLVM breaks xeus-cling. I'm working on patching it now.

CC @emilazy

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

@thomasjm
Copy link
Contributor Author

thomasjm commented Sep 2, 2025

Okay, I just pushed another commit to patch xeus-cling to use LLVM 18. I haven't fully tested it yet. Once I do I'll open a PR upstream, but upstream tends to be slow about accepting changes.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab labels Sep 3, 2025
@thomasjm thomasjm requested a review from emilazy September 3, 2025 00:19
@thomasjm
Copy link
Contributor Author

thomasjm commented Sep 3, 2025

Okay yeah, the tests I'm running against xeus-cling are all passing so I think this is good!

Copy link
Member

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good to me and seems to work fine on Darwin! Happy to merge when you’re satisfied with the xeus-cling changes, as I have not tested that.

I hope that some of the stuff in #439740 can be of use in future to potentially simplify the derivation here and make it more robust to changes in our wrappers (e.g. we’re moving to using the system libc++ on Darwin, which might require adjustments here). But this meets my primary goal of getting the old LLVM versions out of Nixpkgs, so I am very happy with it, thank you! :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should probably use stdenv.cc.libcxx, as the libc++ libraries are backwards‐compatible with older headers but mixing the library versions is a bad idea. (Not a blocker, as it’s not a new problem.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean llvmPackages_18.libcxx -> stdenv.cc.libcxx? That doesn't seem like a good idea considering these are library flags. And also that we deliberately use clangStdenv above.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clangStdenv doesn’t imply the use of libstdc++ vs. libc++, only the C compiler; it keeps the default C++ library for the platform. (libcxxStdenv explicitly picks the combination of Clang and libc++.)

The C++ standard library is ABI‐sensitive, so mixing different versions of libc++, or libstdc++ with libc++, can cause compatibility issues. However, libc++’s headers only work with a specific range of Clang versions, but can be used to target later versions of the library. So it is correct to use a specific include directory here, but you generally want to ensure the use of the same version to actually link with.

That’s why it probably doesn’t make sense to have a separate useLLVMLibcxx here, as opposed to conditioning on the platform (and hence why I dropped it in my PR). So, in that case, stdenv.cc.libcxx would be non‐null exactly when the platform uses libc++, and you’d want to link with that version while using the headers from Cling’s version of LLVM.

Anyway, this is all stuff you don’t have to think about if you reuse our existing wrapper logic, ideally :) And in this case hopefully the things Cling builds don’t link with much, so the scope for problems is limited. But generally linking against a specific C++ standard library isn’t too reliable (and indeed problems with mixing them in obscure cases are what is driving us to switch to linking against the system‐provided library from the SDK on Darwin).

@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Sep 3, 2025
@thomasjm
Copy link
Contributor Author

thomasjm commented Sep 3, 2025

I hope that some of the stuff in #439740 can be of use in future to potentially simplify the derivation here and make it more robust to changes in our wrappers (e.g. we’re moving to using the system libc++ on Darwin, which might require adjustments here).

Huh. I think that if clangStdenv remains a reasonable clang/llvm toolchain, the current version will still work?

I wasn't familiar with mkLLVMPackages before, that looks interesting. This package is a tricky one to iterate on though, as constantly rebuilding Clang can take a long time.

I'm not sure I understand what this part is doing, it looks like it's trying to pretend Cling is a compiler?

https://github.com/emilazy/nixpkgs/blob/9b370d7a3d2bdf28067bc258b50260052e242573/pkgs/by-name/cl/cling/package.nix#L87-L89

@thomasjm
Copy link
Contributor Author

thomasjm commented Sep 3, 2025

I just re-ran my xeus-cling tests again and it's all green, so happy to merge!

@eihqnh eihqnh mentioned this pull request Sep 3, 2025
1 task
@thomasjm
Copy link
Contributor Author

thomasjm commented Sep 3, 2025

This closes #338370 btw.

@emilazy
Copy link
Member

emilazy commented Sep 4, 2025

Huh. I think that if clangStdenv remains a reasonable clang/llvm toolchain, the current version will still work?

I wasn't familiar with mkLLVMPackages before, that looks interesting. This package is a tricky one to iterate on though, as constantly rebuilding Clang can take a long time.

Yeah, it’s a pain to rebuild Clang. OTOH, splitting out the LLVM build reduces that to an extent – any changes that only affect the Cling package don’t rebuild LLVM/Clang.

The issue isn’t with clangStdenv per se, but with the wrapper flags. Those may need to change when we stop using the source‐based libc++ from LLVM on Darwin, for example. Using the wrappers would help avoid that.

I'm not sure I understand what this part is doing, it looks like it's trying to pretend Cling is a compiler?

https://github.com/emilazy/nixpkgs/blob/9b370d7a3d2bdf28067bc258b50260052e242573/pkgs/by-name/cl/cling/package.nix#L87-L89

Well, it basically is a compiler – it reuses Clang’s frontend, and the flags passed along to it are very similar to the ones we pass along to Clang to get the required include directories and libraries set up. So the idea is that you can ideally use the same wrapper to get the flags you need, and remove the platform‐specific logic and complexity that currently has to live in the Cling derivation. (The rename to clang++ when passing it to the wrapper is because it somewhat unfortunately assumes that anything which isClang has its executables named that way, and I didn’t want to block on changing the wrapper in ways that would cause a mass rebuild.)

Unfortunately… Cling’s insistence on trying to find header include paths however it can, and weird Cling behaviour that seems buggy to me, got in the way of making this work totally smoothly before seeing that you were also working on the update :) It would also require some adaptation to work with xeus-cling, I presume. But it would generally be a better path if it can be made to work, I think.

All that said, this PR currently works and mine certainly doesn’t, and I’m very happy to merge in the current state :) Thanks for working on this!

@emilazy emilazy merged commit d72864f into NixOS:master Sep 4, 2025
29 of 31 checks passed
@thomasjm
Copy link
Contributor Author

thomasjm commented Sep 4, 2025

No problem, thanks for explaining! I'll look into using these ideas next time Cling needs work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: jupyter Interactive computing tooling: kernels, notebook, jupyterlab 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants