python311: enable windows cross-compilation, make win-dll-hook link to distant dependancies#227818
Conversation
marius851000
requested review from
Ericson2314,
FRidh,
jonringer and
matthewbauer
as code owners
github-actions
bot
added
the
6.topic: python
ofborg
bot
added
the
6.topic: cross-compilation
marius851000
force-pushed
the
python_win
branch
2 times, most recently
from
43c4fe4 to
96b698d
Compare
ofborg
bot
added
10.rebuild-linux-stdenv
|
Seems that both failed builds are due to timeout. |
|
Additionnally, this should fix #38451 and partially replace #38632 . specific mention to @Ericson2314 , which seems interested by that. |
|
Seems that I messed the part that try to not copy dll already loaded by python into the dynload folder. I’ll fix that later today. |
marius851000
force-pushed
the
python_win
branch
from
96b698d to
14f3b85
Compare
|
Non python non hook changes split to #251828 |
|
Rebased and tweaked, now only one commit remains! 😄 |
ofborg
bot
added
the
10.rebuild-darwin-stdenv
|
@Artturin Do you want to hack this up to make it a non-mass-rebuild, and then fix up the code (to be like it is now) on staging afterwards? |
Artturin
force-pushed
the
python_win
branch
3 times, most recently
from
6a4ef75 to
6740ec4
Compare
Should be like that now |
ofborg
bot
added
10.rebuild-darwin: 0
There was a problem hiding this comment.
There was a problem hiding this comment.
for readline
diff --git a/pkgs/development/libraries/readline/8.2.nix b/pkgs/development/libraries/readline/8.2.nix
index 1c53da3cdfa4..eca19897e73e 100644
--- a/pkgs/development/libraries/readline/8.2.nix
+++ b/pkgs/development/libraries/readline/8.2.nix
@@ -1,4 +1,4 @@
-{ fetchurl, stdenv, lib, ncurses
+{ fetchurl, stdenv, lib, ncurses, fetchgit
}:
stdenv.mkDerivation rec {
@@ -33,6 +33,23 @@ stdenv.mkDerivation rec {
]
++ upstreamPatches;
+ postPatch = if (stdenv.hostPlatform.isMinGW) then (
+ let
+ # https://src.fedoraproject.org/rpms/mingw-gdbm
+ readline-patch = fetchgit {
+ name = "mingw-readline-patches";
+ url = "https://src.fedoraproject.org/rpms/mingw-readline.git";
+ rev = "c15e77591106244a9f1c318226167d5bcea7bf04"; # patches for readline 8.2
+ sha256 = "sha256-QcMJ/nq2T4QlLkw0tAutMohAB0zWa1mRUYuN7D/Kvh0=";
+ };
+ in
+ ''
+ for p in ${readline-patch}/*.patch; do
+ patch -p1 -d . -i $p
+ done
+ ''
+ ) else null;
+
meta = with lib; {
description = "Library for interactive line editing";
@@ -57,7 +74,7 @@ stdenv.mkDerivation rec {
maintainers = with maintainers; [ dtzWill ];
- platforms = platforms.unix;
+ platforms = platforms.unix ++ platforms.windows;
branch = "8.2";
};
}readline-x86_64-w64-mingw32> /nix/store/8gfd58vf0arh22p7bc8da0qidrydfi4b-x86_64-w64-mingw32-binutils-2.40/bin/x86_64-w64-mingw32-ld: cannot
find -lncurses: No such file or directory
readline-x86_64-w64-mingw32> /nix/store/8gfd58vf0arh22p7bc8da0qidrydfi4b-x86_64-w64-mingw32-binutils-2.40/bin/x86_64-w64-mingw32-ld: cannot
find -ltermcap: No such file or directory
fedora has mingw-termcap https://src.fedoraproject.org/rpms/mingw-termcap/blob/rawhide/f/mingw-termcap.spec
which has a comment
# Note: Termcap was deprecated and removed from Fedora after F-8. It
# has been replaced by ncurses. However ncurses cannot be compiled on
# Windows so we have to supply termcap.
but pkgsCross.mingwW64.ncurses builds for us and is already in propagatedBuildInputs but does not work.
There was a problem hiding this comment.
the gdbm patch is for 1.11 and doesn't apply to our 1.23
There was a problem hiding this comment.
Removed the in-tree patch and did this instead.
Artturin
force-pushed
the
python_win
branch
2 times, most recently
from
acac81b to
ff131bc
Compare
There was a problem hiding this comment.
I updated the rev to python 3.11.5 even though the patch files haven't changed.
Artturin
left a comment
Artturin
left a comment
There was a problem hiding this comment.
Merge after eval is done and 0 rebuilds
|
Thanks, and sorry for not having fixed that myselr. Kept pushing that back to tomorrow since I'm back from vacation. For now, my mind is rather away to Nixpkgs and into other projects (maybe NixCon's stream will change that, maybe not). Have a nice day. |
4 participants
Description of changes
A minimal set of change to enable cross-compilation of the python 3.11 interpreter to windows (pkgsCross.mingwW64.python311), thanks to patch used by fedora.
It also change win-dll-hook to not only link necessary dlls from all the outputs in the binary folder, but also those in distant package, based on a new environment variable for Windows cross-build,
LINK_DLL_FOLDERS, that list folders containing DLLs that may be copied to the binary folder.(for exemple, it will move
libgcc_s_seh-1.dll,libwinpthread-1.dll,libpython3.11.dllandmcfgthread-12.dllin the bin folder of python311, the python fixup phase will do the same for compiled python dependancies, that are stored in a different folder)It also cause a world rebuild due to the change in the gcc build script and the generic CC wrapper.
Note that this is only sufficient for compiling the interpreter itself, but not dependancies. I’ll make another PR improving the makeBinaryWrapper, that should also replace shebang with an executable for Windows before trying to take a deeper look at compiling dependancies.
ps: Only tested with Wine on my machine. I haven’t tested on Windows, and I’m a bit worried in regard to the Nix store. Ideally, everything use a relative path, so I can just copy a folder containing all run-time derivation and move it around.
Things done
I tried to make particularly sure that change that are not behind an isWindows or similar doesn’t have side-effect for other packages, althought I didn’t tried to re-compile everything.
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/) (NOTE: only tested executable files. Windows can’t run shebang, but I’ll make another independant patch for a generic solution to this problem involving makeBinaryWrapper)superseed #84815