Allow postgresql to cross compile#44083
Merged
Ericson2314 merged 15 commits intoNixOS:masterfrom Aug 3, 2018
Merged
Conversation
GrahamcOfBorg
added
10.rebuild-darwin: 101-500
dezgeg
reviewed
Jul 25, 2018
| patches = [ ./shtool.patch ]; | ||
|
|
||
| preConfigure = '' | ||
| export ac_cv_va_copy=yes |
Contributor
There was a problem hiding this comment.
I think conventionally in Nixpkgs, this sort of stuff is not exported but added to configureFlags instead.
Contributor
Author
There was a problem hiding this comment.
Thats the problem. This configure script does not allow you to fix the test results on va_copy. Please advise a better approach, if any.
Member
There was a problem hiding this comment.
@dezgeg means passing the foo=var directly to configure, which does work for autoconf-based configure scripts.
dezgeg
reviewed
Jul 25, 2018
| outputs = [ "out" "lib" "doc" "man" ]; | ||
| setOutputFlags = false; # $out retains configureFlags :-/ | ||
|
|
||
| combinedLibXML2 = symlinkJoin { |
Contributor
There was a problem hiding this comment.
Hmm, why is this needed? Why doesn`t
nativeBuildInputs = [ libxml2 ];
work?
Contributor
Author
There was a problem hiding this comment.
Possibly, but I just realised that the upstream broke cross-compilation capability, possibly due to #44068 by @Ericson2314. I will close this PR for now.
Contributor
Author
There was a problem hiding this comment.
@dezgeg I tried. However, the configure of postgresql expects the libxml2 path to contain both its headers and its static libraries.
We would either patch the configure, or just supply the combined libxml2.
Member
There was a problem hiding this comment.
If you pass it as a buildInput, its dev outputs will be passed with the headers, and the -L to the libs added by cc-wrapper will probably leave the configure script unable to notice the difference. If you make it just a nativeBuildInput you will not get the -L.
Contributor
Author
There was a problem hiding this comment.
@Ericson2314 I just confirmed that if libxml2 is added to buildInputs, the static library is visible to the cross compilers. However, the headers are not picked up by the configure script.
Sorry that I am new to this cross compiling business. Should we modify CFLAGS to point to the headers?
Contributor
Author
There was a problem hiding this comment.
Currently libxml2 export headers in the include/libxml2 folder in the Nix store, but the CFLAGS is pointing to include only. When configure is resolving #include <libxml/parser.h>, it is looking at the include folder only and this fails the configure checks.
The build works if I point the headers to include/libxml2. The question now is whether libxml2 Nix expression evaluating correctly.
Contributor
Author
|
Cross compilation is broken. Close for now and will come back later. |
Member
|
Cross compilation is fixed again. |
| makeFlags = [ "world" ]; | ||
|
|
||
| preConfigure = '' | ||
| export CFLAGS="$CFLAGS -I${libxml2.dev}/include/libxml2" |
Member
There was a problem hiding this comment.
I think we usually prefer to do this like:
NIX_CFLAGS_COMPILE = "-I${libxml2.dev}/include/libxml2";
Mostly this is because not all build systems recognize CFLAGS.
| ++ lib.optional (!stdenv.isDarwin) "--with-ossp-uuid"; | ||
| ++ lib.optional stdenv.isDarwin "--with-uuid=e2fs" | ||
| ++ lib.optional (!stdenv.isDarwin) "--with-ossp-uuid" | ||
| ++ lib.optional stdenv.isCross "--with-system-tzdata=${tzdata}"; |
Member
There was a problem hiding this comment.
This one you can probably do unconditionally.
Mic92
reviewed
Jul 30, 2018
| makeFlags = [ "world" ]; | ||
|
|
||
| preConfigure = '' | ||
| export NIX_CFLAGS_COMPILE="$NIX_CFLAGS_COMPILE -I${libxml2.dev}/include/libxml2" |
Member
There was a problem hiding this comment.
Does postgresql also picks this up, when pkgconfig is included in nativeBuildInputs?
Member
There was a problem hiding this comment.
Now I see why this fails:
It needs xml2-config compiled for the build platform.
Member
There was a problem hiding this comment.
Instead of preConfigure, this can be exported in nix instead:
# needed for cross-compiling, since xml2-config is not compiled for the build platform
NIX_CFLAGS_COMPILE = [ "-I${libxml2.dev}/include/libxml2" ];
Member
There was a problem hiding this comment.
The long-term solution would be pkg-config: However this would been an upstream fix: libarchive/libarchive#407
Contributor
Author
There was a problem hiding this comment.
I see. I will try to raise this issue in postgresql mailing list.
Contributor
There was a problem hiding this comment.
Actually xml2-config is built correctly but it doesn't get into the PATH since we don't add binaries from dev output to PATH. This is something we should do in stdenv since many other packages will benefit from that as well.
Contributor
Author
There was a problem hiding this comment.
Should I do it in a separate pull request? Or this one?
Member
There was a problem hiding this comment.
I think for now this is fine.
Contributor
There was a problem hiding this comment.
Yes I agree it can be done later.
Let's however conditionalize it on cross, so that someone doesn't break it by noticing it native-compiles fine with it.
Member
|
@GrahamcOfBorg build pkgsCross.raspberryPi.postgresql |
|
Success on x86_64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Failure on x86_64-darwin (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
No attempt on aarch64-linux (full log) The following builds were skipped because they don't evaluate on aarch64-linux: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
No attempt on x86_64-linux (full log) The following builds were skipped because they don't evaluate on x86_64-linux: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
matthewbauer
reviewed
Aug 2, 2018
Ericson2314
force-pushed
the
cross-compiling-postgresql
branch
from
ccc2357 to
f539c30
Compare
Member
|
@GrahamcOfBorg build pkgsCross.raspberryPi.postgresql |
Member
|
retroactively testing oops (I missed that above, but I also figured at least native builds would continue to work). |
|
Success on x86_64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql The following builds were skipped because they don't evaluate on x86_64-linux: retroactively... Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
Member
|
Cool :) |
|
Failure on x86_64-darwin (full log) Attempted: pkgsCross.raspberryPi.postgresql The following builds were skipped because they don't evaluate on x86_64-darwin: retroactively... Partial log (click to expand)
|
|
Failure on x86_64-darwin (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql The following builds were skipped because they don't evaluate on aarch64-linux: retroactively... Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: pkgsCross.raspberryPi.postgresql Partial log (click to expand)
|
Member
|
@Ericson2314 this was a mass-rebuild and should have gone to staging. |
Member
|
Sorry @FRidh. I missed that completely. |
9 tasks
Janik-Haag
added
the
12.first-time contribution
wolfgangwalther
added a commit
to wolfgangwalther/nixpkgs
that referenced
this pull request
Apr 4, 2024
This was introduced in NixOS#44083 to fix cross building, where xml2-config wouldn't run on the host platform. This was fixed upstream two years later [1], so that from v13 on pkg-config is used before xml2-config is. Once v12 is EOL, we can remove this entirely. [1]: postgres/postgres@0bc8ceb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation for this change
Currently,
PostgreSQLdoes not cross compile due to not listinglibXML2headers as a native build dependency properly, andlibossp-uuidas a dependency ofPostgreSQLdoes not cross compile because it does not respectstripcommand setting made byconfigure.Things done
sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)nix path-info -Sbefore and after)