cmake: fix strictDeps#318226
cmake: fix strictDeps#318226emilazy merged 3 commits intoNixOS:stagingfrom paparodeo:cmake-cc-full-path
Conversation
ofborg
bot
added
10.rebuild-darwin-stdenv
|
cc cmake maintainers: @ttuegel @LnL7 @AndersonTorres |
paparodeo
changed the title
|
after a little matrix discussion it seems that we shouldn't be using CMAKE_PREFIX_PATH to add the buildInputs path as that ends up adding the diff --git a/pkgs/by-name/cm/cmake/setup-hook.sh b/pkgs/by-name/cm/cmake/setup-hook.sh
index b28ed42b6896..63f42a68f02a 100755
--- a/pkgs/by-name/cm/cmake/setup-hook.sh
+++ b/pkgs/by-name/cm/cmake/setup-hook.sh
@@ -1,5 +1,6 @@
addCMakeParams() {
- addToSearchPath CMAKE_PREFIX_PATH $1
+ addToSearchPath CMAKE_INCLUDE_PATH "$1/include"
+ addToSearchPath CMAKE_LIBRARY_PATH "$1/lib"
}
fixCmakeFiles() { |
|
so it used to just use |
This is clearly KDE-specific. Should it be "generalized" outside KDE? |
|
with a modified cmake setup-hooks only specifying CMAKE_LIBRARY_PATH and CMAKE_INCLUDE PATH i was getting failures with and after a bit of trial and error i found that after adding addCMakeParams() {
addToSearchPath CMAKE_INCLUDE_PATH "$1/include"
# [edit 4]
addToSearchPath CMAKE_INCLUDE_PATH "$1"
addToSearchPath CMAKE_LIBRARY_PATH "$1/lib"
# [edit 4]
addToSearchPath CMAKE_LIBRARY_PATH "$1"
addToSearchPath CMAKE_PREFIX_PATH "$1/lib/cmake"
# [edit 1]
addToSearchPath CMAKE_PREFIX_PATH "$1/share/cmake"
# [edit 3] -- have not hit but looks like cmake would search here given $1
addToSearchPath CMAKE_PREFIX_PATH "$1/lib"
# [edit 2]
addToSearchPath CMAKE_PREFIX_PATH "$1/share"
} this configuration also worked for zig / castxml when built with clangStdenv and strictDeps enabled -- actually for all 3 packages only having will try to build [edit] needed [edit2] need just [edit3] CMAKE_PREFIX_PATH should have $1/lib/cmake, $1/lib/share, $1/lib $1/share [edit 4] saw failures that required CMAKE_{LIBRARY,INCLUDE}_PATH to also just be set to $1 |
paparodeo
changed the title
paparodeo
changed the title
|
was not finding pkgconfig in some cases (plasma5Packages.libksysguard) so switched strategy and just patched out using [edit] can fix plasma5Packages.libksysguard with the old (non c++) patch. pkgconfig worked fine but had its own custom FindNL.cmake code which searches for headers / libraries using |
|
last change removed the cmake patch in favor of modifying |
This is mainly used for |
A long time ago it was recommended to me "avoid Is there anything documenting these things in a more practical way? |
|
Well it was just my opinion 😅 I found it very difficult to reason about dependencies when I was trying to fix |
Not exactly on-topic, but pythonXPackages now use pyproject.toml-style |
|
We're pretty sure that this breaks nix-ros-overlay: lopsided98/nix-ros-overlay#491 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
This had been resolved in NixOS#318226.
strictDeps works with Clang after NixOS#318226.
|
is there a reason this had to define a custom nix envvar, instead of just using the generic |
|
@sielicki I think these two messages reflect the knowledge we've had at the moment: We sought how other projects (cmake, vcpkg, connan) approach this problem but cut the search short and went for paparodeo's patch as interim solution to cross issues. I don't think we were aware of As you bring this up, I'll start listing some side considerations again: Details
|
Totally agree on this overall idea. Recent cmake releases have a lot of new features to make this easier.
I can try to put that together -- just having trouble understanding exactly how these variables are used, and what stands to break. |
yes, because they don't work. IIRC the ignore_xxx_path will also ignore things in |
Is there an upstream issue? It's mentioned elsewhere in the cmake docs, but not on these specific doc pages, that these variables are semicolon separated and not colon-separated like normal search paths. So you need to use Here's a patch I have locally, but I need to test it more to see what breaks: 9f3110649578b2f7c8bff356570e97138c3ffc10
Author: Nicholas Sielicki <[email protected]>
AuthorDate: Sun Apr 20 23:16:39 2025 -0700
Commit: Nicholas Sielicki <[email protected]>
CommitDate: Sun Apr 20 23:18:35 2025 -0700
remove a cmake patch
5 files changed, 22 insertions(+), 57 deletions(-)
.../cm/cmake/000-nixpkgs-cmake-prefix-path.diff | 28 ------------------
pkgs/by-name/cm/cmake/package.nix | 3 --
pkgs/by-name/cm/cmake/setup-hook.sh | 33 ++++++++++++++--------
.../me/meson/000-nixpkgs-cmake-prefix-path.patch | 12 --------
pkgs/by-name/me/meson/package.nix | 3 --
deleted pkgs/by-name/cm/cmake/000-nixpkgs-cmake-prefix-path.diff
@@ -1,28 +0,0 @@
-diff --git a/Source/cmFindBase.cxx b/Source/cmFindBase.cxx
-index 8840cdcb..c34b7ee9 100644
---- a/Source/cmFindBase.cxx
-+++ b/Source/cmFindBase.cxx
-@@ -280,6 +280,11 @@ void cmFindBase::FillCMakeEnvironmentPath()
- // Add CMAKE_*_PATH environment variables
- std::string var = cmStrCat("CMAKE_", this->CMakePathName, "_PATH");
- paths.AddEnvPrefixPath("CMAKE_PREFIX_PATH");
-+ if (this->CMakePathName != "PROGRAM") {
-+ // Like CMAKE_PREFIX_PATH except when searching for programs. Programs need
-+ // to be located via PATH
-+ paths.AddEnvPrefixPath("NIXPKGS_CMAKE_PREFIX_PATH");
-+ }
- paths.AddEnvPath(var);
-
- if (this->CMakePathName == "PROGRAM") {
-diff --git a/Source/cmFindPackageCommand.cxx b/Source/cmFindPackageCommand.cxx
-index 9b51b1ad..6acc676c 100644
---- a/Source/cmFindPackageCommand.cxx
-+++ b/Source/cmFindPackageCommand.cxx
-@@ -2039,6 +2039,7 @@ void cmFindPackageCommand::FillPrefixesCMakeEnvironment()
-
- // And now the general CMake environment variables
- paths.AddEnvPath("CMAKE_PREFIX_PATH");
-+ paths.AddEnvPath("NIXPKGS_CMAKE_PREFIX_PATH");
- if (this->DebugMode) {
- debugBuffer = cmStrCat(debugBuffer,
- "CMAKE_PREFIX_PATH env variable "
modified pkgs/by-name/cm/cmake/package.nix
@@ -59,9 +59,6 @@ stdenv.mkDerivation (finalAttrs: {
patches =
[
- # Add NIXPKGS_CMAKE_PREFIX_PATH to cmake which is like CMAKE_PREFIX_PATH
- # except it is not searched for programs
- ./000-nixpkgs-cmake-prefix-path.diff
# Don't search in non-Nix locations such as /usr, but do search in our libc.
./001-search-path.diff
]
modified pkgs/by-name/cm/cmake/setup-hook.sh
@@ -1,8 +1,11 @@
-addCMakeParams() {
- # NIXPKGS_CMAKE_PREFIX_PATH is like CMAKE_PREFIX_PATH except cmake
- # will not search it for programs
- addToSearchPath NIXPKGS_CMAKE_PREFIX_PATH $1
-}
+addCmakeSystemAppbundlePath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_APPBUNDLE_PATH "$@" ; }
+addCmakeSystemFrameworkPath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_FRAMEWORK_PATH "$@" ; }
+addCmakeSystemIgnorePath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_IGNORE_PATH "$@" ; }
+addCmakeSystemIgnorePrefixPath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_IGNORE_PREFIX_PATH "$@" ; }
+addCmakeSystemIncludePath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_INCLUDE_PATH "$@" ; }
+addCmakeSystemLibraryPath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_LIBRARY_PATH "$@" ; }
+addCmakeSystemPrefixPath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_PREFIX_PATH "$@" ; }
+addCmakeSystemProgramPath() { addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_PROGRAM_PATH "$@" ; }
fixCmakeFiles() {
# Replace occurences of /usr and /opt by /var/empty.
@@ -147,7 +150,15 @@ if [ -z "${dontUseCmakeConfigure-}" -a -z "${configurePhase-}" ]; then
configurePhase=cmakeConfigurePhase
fi
-addEnvHooks "$targetOffset" addCMakeParams
+addEnvHooks "$targetOffset" \
+ addCmakeSystemAppbundlePath \
+ addCmakeSystemFrameworkPath \
+ addCmakeSystemIgnorePath \
+ addCmakeSystemIgnorePrefixPath \
+ addCmakeSystemIncludePath \
+ addCmakeSystemLibraryPath \
+ addCmakeSystemPrefixPath \
+ addCmakeSystemProgramPath
makeCmakeFindLibs() {
isystem_seen=
@@ -155,22 +166,22 @@ makeCmakeFindLibs() {
for flag in ${NIX_CFLAGS_COMPILE-} ${NIX_LDFLAGS-}; do
if test -n "$isystem_seen" && test -d "$flag"; then
isystem_seen=
- addToSearchPath CMAKE_INCLUDE_PATH "${flag}"
+ addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_INCLUDE_PATH "${flag}"
elif test -n "$iframework_seen" && test -d "$flag"; then
iframework_seen=
- addToSearchPath CMAKE_FRAMEWORK_PATH "${flag}"
+ addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_FRAMEWORK_PATH "${flag}"
else
isystem_seen=
iframework_seen=
case $flag in
-I*)
- addToSearchPath CMAKE_INCLUDE_PATH "${flag:2}"
+ addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_INCLUDE_PATH "${flag:2}"
;;
-L*)
- addToSearchPath CMAKE_LIBRARY_PATH "${flag:2}"
+ addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_LIBRARY_PATH "${flag:2}"
;;
-F*)
- addToSearchPath CMAKE_FRAMEWORK_PATH "${flag:2}"
+ addToSearchPathWithCustomDelimiter ";" CMAKE_SYSTEM_FRAMEWORK_PATH "${flag:2}"
;;
-isystem)
isystem_seen=1
deleted pkgs/by-name/me/meson/000-nixpkgs-cmake-prefix-path.patch
@@ -1,12 +0,0 @@
-diff --git a/mesonbuild/dependencies/data/CMakePathInfo.txt b/mesonbuild/dependencies/data/CMakePathInfo.txt
-index 662ec58..4d5f4e4 100644
---- a/mesonbuild/dependencies/data/CMakePathInfo.txt
-+++ b/mesonbuild/dependencies/data/CMakePathInfo.txt
-@@ -5,6 +5,7 @@ list(APPEND TMP_PATHS_LIST ${CMAKE_PREFIX_PATH})
- list(APPEND TMP_PATHS_LIST ${CMAKE_FRAMEWORK_PATH})
- list(APPEND TMP_PATHS_LIST ${CMAKE_APPBUNDLE_PATH})
- list(APPEND TMP_PATHS_LIST $ENV{CMAKE_PREFIX_PATH})
-+list(APPEND TMP_PATHS_LIST $ENV{NIXPKGS_CMAKE_PREFIX_PATH})
- list(APPEND TMP_PATHS_LIST $ENV{CMAKE_FRAMEWORK_PATH})
- list(APPEND TMP_PATHS_LIST $ENV{CMAKE_APPBUNDLE_PATH})
- list(APPEND TMP_PATHS_LIST ${CMAKE_SYSTEM_PREFIX_PATH})
modified pkgs/by-name/me/meson/package.nix
@@ -25,9 +25,6 @@ python3.pkgs.buildPythonApplication rec {
};
patches = [
- # Nixpkgs cmake uses NIXPKGS_CMAKE_PREFIX_PATH for the search path
- ./000-nixpkgs-cmake-prefix-path.patch
-
# In typical distributions, RPATH is only needed for internal libraries so
# meson removes everything else. With Nix, the locations of libraries
# are not as predictable, therefore we need to keep them in the RPATH.
|
I apologize -- I have no idea what you're trying to do. please test your change and open a new PR. the 80 some comments in this PR log the various methods I tried and why we ended up with the cmake patch. To test the various changes I looked at all the packages which had looking over your patch I am pretty sure it doesn't work given these are cmake variables you're modifying and not env vars and you're not passing them via I'd ask if you even tried your patch to see if anything works but I don't think that a response belongs in this PR comments and you need to open up a new PR. |
8 participants
Description of changes
patch cmake (and meson) to use a new
NIXPKGS_CMAKE_PREFIX_PATHvariable which is likeCMAKE_PREFIX_PATHbut doesn't provide a search path tofind_programcmake uses
CMAKE_PREFIX_PATHto search for libraries, header files, binaries, cmake modules, etc. This is mostly fine except for binaries. We setCMAKE_PREFIX_PATHto the derivation output locations, which results in cmake finding and using binaries which are not set in PATH. when strictDeps is true, nativeBuildInputs is not added toCMAKE_PREFIX_PATHbut buildInputs so binaries are first searched for in the buildInputs packages before falling back to PATH.This change attempts to fix the cmake search path such that cmake will only locate binaries accessible via PATH.
the meson patch is required when it falls back on cmake to find modules, it needs to know about the
NIXPKGS_CMAKE_PREFIX_PATHenv variable.the following PRs fix various packages that enable
strictDepsbut fail to build due to packages missing fromnativeBuildInputstesting
tests were usually run on master with the cmake modifications contained in a new cmake2 package to skip unnecessary rebuilds. packages that required cmake were then built using an override to the modified cmake or manually edited/hacked when overriding was not supported, eg:
packages that require cmake were found using a variation of the following script run from the root of the source tree
example script to get a json list of packages using cmake but not strict deps
then iterate over the packages (or a sample of) passing in the modified
cmake2Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.