emscripten: 2.0.10 -> 2.0.26 by yu-re-ka · Pull Request #133217 · NixOS/nixpkgs

yu-re-ka · 2021-08-09T04:38:15Z

Motivation for this change

This uses some of the fixes from #130706

Things done

yu-re-ka · 2021-08-09T04:50:27Z

@ofborg build emscriptenPackages.zlib

pkgs/development/compilers/emscripten/default.nix

yu-re-ka · 2021-08-09T18:51:45Z

@ofborg build emscriptenPackages.zlib emscriptenPackages.json-c emscriptenPackages.libxml2

yu-re-ka · 2021-08-09T18:56:37Z

@ofborg build emscriptenPackages.json_c

happysalada · 2021-08-10T05:36:16Z

I am running the build on darwin and it fails with the following

❯ nix log /nix/store/1zpa0vj4xwhd33h5rqdgkd8nkh5iawap-libcxx-13.0.0-rc1.drv
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/b6wzns2zx2jzgdg62752nk58gvjvba18-source
source root is source/libcxx
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
applying patch /nix/store/jrxfq5l2fxs17rhx8w2glh6r8g9bw7jc-gnu-install-dirs.patch
patching file CMakeLists.txt
patching file cmake/Modules/HandleLibCXXABI.cmake
patching file src/CMakeLists.txt
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EX
-- The CXX compiler identification is Clang 7.1.0
-- The C compiler identification is Clang 7.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/jx31ah14p6pf488i7wwd88y2izmxi0wj-clang-wrapper-7.1.0/bin/cla
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/jx31ah14p6pf488i7wwd88y2izmxi0wj-clang-wrapper-7.1.0/bin/clang
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring for standalone build.
-- Linker detection: ld64
-- Found Python3: /nix/store/ad19ya4siaxhz63w0m8xh97067iahks0-python3-3.9.6/bin/python3.9 (found version "
...skipping...
/tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/src/support/runtime/exception_pointer_unimplemented.i
#  warning exception_ptr not yet implemented
   ^
/tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/src/support/runtime/exception_pointer_unimplemented.i
#  warning exception_ptr not yet implemented
   ^
/tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/src/support/runtime/exception_pointer_unimplemented.i
#  warning exception_ptr not yet implemented
   ^
7 warnings generated.
[ 83%] Building CXX object src/CMakeFiles/cxx_static.dir/functional.cpp.o
[ 83%] Building CXX object src/CMakeFiles/cxx_static.dir/future.cpp.o
[ 83%] Building CXX object src/CMakeFiles/cxx_static.dir/hash.cpp.o
[ 83%] Building CXX object src/CMakeFiles/cxx_static.dir/memory.cpp.o
[ 84%] Building CXX object src/CMakeFiles/cxx_static.dir/mutex.cpp.o
[ 84%] Building CXX object src/CMakeFiles/cxx_static.dir/mutex_destructor.cpp.o
[ 84%] Building CXX object src/CMakeFiles/cxx_static.dir/new.cpp.o
[ 84%] Building CXX object src/CMakeFiles/cxx_static.dir/optional.cpp.o
In file included from /tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/src/optional.cpp:9:
/tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/build/include/c++/v1/optional:224:5: error: destructo
    _LIBCPP_CONSTEXPR_AFTER_CXX17 ~__optional_destruct_base()
    ^
/tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/build/include/c++/v1/__config:1054:41: note: expanded
#  define _LIBCPP_CONSTEXPR_AFTER_CXX17 constexpr
                                        ^
1 error generated.
make[2]: *** [src/CMakeFiles/cxx_static.dir/build.make:303: src/CMakeFiles/cxx_static.dir/optional.cpp.o]
make[1]: *** [CMakeFiles/Makefile2:431: src/CMakeFiles/cxx_static.dir/all] Error 2
make: *** [Makefile:149: all] Error 2

how do you feel about staying on 12 ?

yu-re-ka · 2021-08-10T06:16:08Z

Have you tried if using llvmPackages_12 fixes darwin? In that case I would use LLVM 12 on darwin and LLVM 13 on linux. I don't have a Mac to debug it, so there's not much I can do.

happysalada · 2021-08-10T06:19:00Z

sure, let me try and update you.

happysalada · 2021-08-10T06:30:26Z

From the changelog, it looks like the last version change mentions updating to llvm 12
https://github.com/emscripten-core/emscripten/blob/main/ChangeLog.md#2023---05262021

However building it with version 12 fails with the following

emscripten> /nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c:35:1: error: static_assert failed due to requirement '(__alignof(max_align_t)) == 8' "max_align_t must be 8"
emscripten> _Static_assert(MALLOC_ALIGNMENT == 8, "max_align_t must be 8");
emscripten> ^              ~~~~~~~~~~~~~~~~~~~~~
emscripten> 1 error generated.
emscripten> emcc: error: '/nix/store/iby5mgyx2m3kgzavavlkp18x1njrxsm6-emscripten-llvm-2.0.26/bin/clang -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=26 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/cache/sysroot -resource-dir=/nix/store/iby5mgyx2m3kgzavavlkp18x1njrxsm6-emscripten-llvm-2.0.26/lib/clang/12.0.1/ -idirafter/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/compat -Werror -fno-unroll-loops -O2 -fno-builtin -g -flto=full -DNDEBUG -c /nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c -o /nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/cache/build/libdlmalloc/dlmalloc.o' failed (returned 1)
emscripten> Traceback (most recent call last):
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/emcc.py", line 3774, in <module>
emscripten>     sys.exit(main(sys.argv))
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/emcc.py", line 3767, in main
emscripten>     ret = run(args)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/emcc.py", line 1102, in run
emscripten>     phase_calculate_system_libraries(state, linker_arguments, linker_inputs, newargs)
emscripten>   File "/nix/store/ad19ya4siaxhz63w0m8xh97067iahks0-python3-3.9.6/lib/python3.9/contextlib.py", line 79, in inner
emscripten>     return func(*args, **kwds)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/emcc.py", line 2468, in phase_calculate_system_libraries
emscripten>     extra_files_to_link += system_libs.calculate([f for _, f in sorted(linker_inputs)] + extra_files_to_link, forced=state.forced_stdlibs)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1583, in calculate
emscripten>     add_library('libmalloc')
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1535, in add_library
emscripten>     libs_to_link.append((lib.get_link_flag(), need_whole_archive))
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 316, in get_link_flag
emscripten>     fullpath = self.get_path()
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 308, in get_path
emscripten>     return shared.Cache.get_lib(self.get_filename(), self.build)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/cache.py", line 137, in get_lib
emscripten>     return self.get(name, *args, **kwargs)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/cache.py", line 165, in get
emscripten>     creator(cachename)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 388, in build
emscripten>     create_lib(out_filename, self.build_objects(build_dir))
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 381, in build_objects
emscripten>     run_build_commands(commands)
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 89, in run_build_commands
emscripten>     shared.run_multiple_processes(commands, env=clean_env())
emscripten>   File "/nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/tools/shared.py", line 204, in run_multiple_processes
emscripten>     raise Exception('Subprocess %d/%d failed (%s)! (cmdline: %s)' % (idx + 1, len(commands), returncode_to_str(finished_process.returncode), shlex_join(commands[idx])))
emscripten> Exception: Subprocess 1/2 failed (returned 1)! (cmdline: /nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/bin/emcc -Werror -fno-unroll-loops -O2 -fno-builtin -g -flto=full -DNDEBUG -c /nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c -o /nix/store/p4hlijd61nj0l405vjvcjwrixcycz3gs-emscripten-2.0.26/share/emscripten/cache/build/libdlmalloc/dlmalloc.o)

feel free to mark it as broken.

yu-re-ka · 2021-08-10T06:46:17Z

I marked it as broken on darwin, ~~but out of curiousity: Did the emscripten 2.0.10 packaging (what is currently on master) work on darwin?~~ Ah yes, I can see in Hydra that it worked before.

yu-re-ka · 2021-08-10T07:13:08Z

@happysalada please try what I have pushed now 🙂

happysalada · 2021-08-10T08:52:11Z

I've got the same failure

error: builder for '/nix/store/1zpa0vj4xwhd33h5rqdgkd8nkh5iawap-libcxx-13.0.0-rc1.drv' failed with exit code 2;
       last 10 log lines:
       > /tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/build/include/c++/v1/optional:224:5: error: destructor cannot be marked constexpr
       >     _LIBCPP_CONSTEXPR_AFTER_CXX17 ~__optional_destruct_base()
       >     ^
       > /tmp/nix-build-libcxx-13.0.0-rc1.drv-0/source/libcxx/build/include/c++/v1/__config:1054:41: note: expanded from macro '_LIBCPP_CONSTEXPR_AFTER_CXX17'
       > #  define _LIBCPP_CONSTEXPR_AFTER_CXX17 constexpr
       >                                         ^
       > 1 error generated.
       > make[2]: *** [src/CMakeFiles/cxx_static.dir/build.make:303: src/CMakeFiles/cxx_static.dir/optional.cpp.o] Error 1
       > make[1]: *** [CMakeFiles/Makefile2:431: src/CMakeFiles/cxx_static.dir/all] Error 2
       > make: *** [Makefile:149: all] Error 2
       For full logs, run 'nix log /nix/store/1zpa0vj4xwhd33h5rqdgkd8nkh5iawap-libcxx-13.0.0-rc1.drv'.
error: 1 dependencies of derivation '/nix/store/pj1zg0wfyys1p1wfillzdzf3ayjpss4r-libcxxabi-13.0.0-rc1.drv' failed to build

I'm not sure why it's trying to biuld that.
The build is trying to continue after that, I wonder why that libcxx-13 is being built.

yu-re-ka · 2021-08-10T08:54:41Z

Yes, it should not attempt to build that on darwin with the current state of my branch. How did you start the build, from a local checkout or maybe the tar was cached?

happysalada · 2021-08-10T08:57:01Z

I am running nixpkgs-review, it looks right because it's trying to use clang-12 at the beginning.
Somehow after that it tries to build that libcxx

yu-re-ka · 2021-08-10T09:53:50Z

Ah yes of course: nixpkgs-review will build all packages that were changed by the PR. Please try this instead:

$ nix build -f https://github.com/yu-re-ka/nixpkgs/archive/feature/emscripten-2-0-26.tar.gz emscriptenPackages.zlib

yu-re-ka · 2021-08-10T17:43:03Z

Okay, went back to marking darwin as broken

yu-re-ka · 2021-08-10T17:43:59Z

@primeos it would be awesome if you could give some feedback on the LLVM changes

primeos · 2021-08-10T18:30:22Z

The LLVM changes look ok to me. I'm not sure if it's a good idea to pass trough release_version that way (I guess it could potentially cause problems with nix-env and/or Hydra due to recurseIntoAttrs), AFAIK we usually use passthru for that but I never looked into the details / actual implementation. Access to release_version sounds like a good idea tough. And thanks for adding comments to the LLVM patches. I get an HTTP 404 Not Found error for https://reviews.llvm.org/file/data/ohyszloh6mmx4p2klifz/PHID-FILE-kfnsgzppcgouonbonyrv/D107020.diff so we probably need to find a stable URL to fetch Phabricator patches. I guess we can simply use https://github.com/llvm/llvm-project/commit/5060224d9eed8b8359ed5090bb7c577b8575e9e7.patch now. Otherwise the patches seem fine.

yu-re-ka · 2021-08-10T19:07:05Z

I'm not sure if it's a good idea to pass trough release_version that way

Do you know who might be able to tell us how to implement it properly? I could expose it as llvmPackages_*.passthru.release_version or as llvmPackages_*.clang.passthru.release_version.

I get an HTTP 404 Not Found error for [...]

Changed it to the GitHub link, so it should work for you now

happysalada · 2021-08-11T03:40:17Z

Result of nixpkgs-review pr 133217 run on x86_64-darwin 1

6 packages marked as broken and skipped:

fast-cli
pdal
puppeteer-cli
python38Packages.tiledb
python39Packages.tiledb
zig

11 packages failed to build:

emscripten
lld_13 (llvmPackages_13.lld)
lldb_13 (llvmPackages_13.lldb)
llvmPackages_13.clang (llvmPackages_13.libcxxClang)
llvmPackages_13.clangNoLibc
llvmPackages_13.clangNoLibcxx
llvmPackages_13.clangUseLLVM
llvmPackages_13.compiler-rt (llvmPackages_13.compiler-rt-libc ,llvmPackages_13.compiler-rt-no-libc)
llvmPackages_13.stdenv (llvmPackages_13.libcxxStdenv)
llvmPackages_13.libstdcxxClang
llvmPackages_13.libunwind

27 packages built:

bat-extras.prettybat
binaryen
ccls
clang-analyzer
clang-tools
clang_12 (llvmPackages_12.clang ,llvmPackages_12.libcxxClang ,llvmPackages_latest.clang ,llvmPackages_latest.libcxxClang)
include-what-you-use
lldb (lldb_12 ,llvmPackages_12.lldb ,llvmPackages_latest.lldb)
llvmPackages_12.clang-manpages (llvmPackages_latest.clang-manpages)
llvmPackages_12.libclang (llvmPackages_12.clang-unwrapped ,llvmPackages_latest.clang-unwrapped ,llvmPackages_latest.libclang)
llvmPackages_12.clangNoCompilerRt (llvmPackages_latest.clangNoCompilerRt)
llvmPackages_12.clangNoCompilerRtWithLibc (llvmPackages_latest.clangNoCompilerRtWithLibc)
llvmPackages_12.clangNoLibc (llvmPackages_latest.clangNoLibc)
llvmPackages_12.clangNoLibcxx (llvmPackages_latest.clangNoLibcxx)
llvmPackages_12.clangUseLLVM (llvmPackages_latest.clangUseLLVM)
llvmPackages_12.stdenv (llvmPackages_12.libcxxStdenv ,llvmPackages_latest.libcxxStdenv ,llvmPackages_latest.stdenv)
llvmPackages_12.libstdcxxClang (llvmPackages_latest.libstdcxxClang)
llvmPackages_13.clang-manpages
llvmPackages_13.libclang (llvmPackages_13.clang-unwrapped)
llvmPackages_13.clangNoCompilerRt
llvmPackages_13.clangNoCompilerRtWithLibc
llvm_13 (llvmPackages_13.llvm)
llvmPackages_13.llvm-manpages
llvmPackages_13.openmp
rust-bindgen
tiledb
vscode-extensions.vadimcn.vscode-lldb

primeos · 2021-08-13T14:51:20Z

Do you know who might be able to tell us how to implement it properly?

Not sure, sorry :o
I usually try to find similar examples in Nixpkgs but they can be difficult to find due to the size and there aren't always any.

I could expose it as llvmPackages_.passthru.release_version or as llvmPackages_.clang.passthru.release_version.

I would've probably exposed it as llvmPackages_*.llvm.passthru.release_version because that seems like the main attribute but I don't have a strong opinion/preference on this. llvmPackages_*.passthru.release_version might be nice as well but I'm not sure how to implement that (would require having a look at how stdenv.mkDerivation handles passthru but I assume that part is only relevant for derivations as well - since your current implementation already exposes release_version as llvmPackages_*.release_version and it's just a string instead of an derivation it might already be fine).

Anyway, I hope my comment didn't seem like a request to change that part, it was just meant as a FYI that this might potentially cause problems that would need to be addressed later.

Actually, I just took a brief look at pkgs/top-level/python-packages.nix and it also contains a lot of special attributes, e.g.:

  inherit (python.passthru) isPy27 isPy35 isPy36 isPy37 isPy38 isPy39 isPy3k isPyPy pythonAtLeast pythonOlder;
  inherit python bootstrapped-pip buildPythonPackage buildPythonApplication;
  inherit fetchPypi;
  inherit hasPythonModule requiredPythonModules makePythonPath disabled disabledIf;
  inherit toPythonModule toPythonApplication;
  inherit buildSetupcfg;

So the current approach might be fine (bundling the exposed attributes in an attribute set like passthru.release_version might be nice to make them more discoverable and avoid potential name collisions but the name passthru might be confusing in this case and I'm not sure what else to use as e.g. meta also already has another meaning).

And sorry for the delay, I was a bit busy and lost track of this.

yu-re-ka · 2021-08-13T17:08:07Z

Okay! I did understand it as a request to change the way release_version is exposed, but if that is not the case thanks for the hint and I will look out for breakage after the merge 🙂

RaitoBezarius · 2021-08-20T14:49:00Z

How much effort would be to go directly to 2.0.27 as it is released now? :P
I am trying to test this PR in my local Nix User Repository but I'm running into issues, maybe I didn't apply the patches correctly.

RaitoBezarius · 2021-08-20T15:18:22Z

In particular, with LLVM12 on Linux (NixOS):

/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c:35:1: error: static_assert failed due to requirement '(__alignof(max_align_t)) == 8' "max_align_t must be 8"
_Static_assert(MALLOC_ALIGNMENT == 8, "max_align_t must be 8");
^              ~~~~~~~~~~~~~~~~~~~~~
1 error generated.
emcc: error: '/nix/store/c1h2187xf8lh7qdl23ngydv8n54l5z3f-emscripten-llvm-2.0.26/bin/clang -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=26 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/cache/sysroot -resource-dir=/nix/store/c1h2187xf8lh7qdl23ngydv8n54l5z3f-emscripten-llvm-2.0.26/lib/clang/12/ -idirafter/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/cache/sysroot/include -iwithsysroot/include/c++/v1 -Xclang -iwithsysroot/include/compat -Werror -fno-unroll-loops -O2 -fno-builtin -g -flto=full -DNDEBUG -c /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c -o /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/cache/build/libdlmalloc/dlmalloc.o' failed (returned 1)
Traceback (most recent call last):
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/emcc.py", line 3774, in <module>
    sys.exit(main(sys.argv))
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/emcc.py", line 3767, in main
    ret = run(args)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/emcc.py", line 1102, in run
    phase_calculate_system_libraries(state, linker_arguments, linker_inputs, newargs)
  File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/emcc.py", line 2468, in phase_calculate_system_libraries
    extra_files_to_link += system_libs.calculate([f for _, f in sorted(linker_inputs)] + extra_files_to_link, forced=state.forced_stdlibs)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1583, in calculate
    add_library('libmalloc')
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1535, in add_library
    libs_to_link.append((lib.get_link_flag(), need_whole_archive))
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 316, in get_link_flag
    fullpath = self.get_path()
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 308, in get_path
    return shared.Cache.get_lib(self.get_filename(), self.build)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/cache.py", line 137, in get_lib
    return self.get(name, *args, **kwargs)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/cache.py", line 165, in get
    creator(cachename)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 388, in build
    create_lib(out_filename, self.build_objects(build_dir))
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 381, in build_objects
    run_build_commands(commands)
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 89, in run_build_commands
    shared.run_multiple_processes(commands, env=clean_env())
  File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/shared.py", line 204, in run_multiple_processes
    raise Exception('Subprocess %d/%d failed (%s)! (cmdline: %s)' % (idx + 1, len(commands), returncode_to_str(finished_process.returncode), shlex_join(commands[idx])))
Exception: Subprocess 1/2 failed (returned 1)! (cmdline: /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/bin/emcc -Werror -fno-unroll-loops -O2 -fno-builtin -g -flto=full -DNDEBUG -c /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c -o /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/cache/build/libdlmalloc/dlmalloc.o)
builder for '/nix/store/ax3w8qc7rip89gfwdrcfddqf803np7gg-emscripten-2.0.26.drv' failed with exit code 1
error: build of '/nix/store/ax3w8qc7rip89gfwdrcfddqf803np7gg-emscripten-2.0.26.drv' on 'ssh://dc1-playground-builder' failed: builder for '/nix/store/ax3w8qc7rip89gfwdrcfddqf803np7gg-emscripten-2.0.26.drv' failed with exit code 1
error: builder for '/nix/store/ax3w8qc7rip89gfwdrcfddqf803np7gg-emscripten-2.0.26.drv' failed with exit code 1;
       last 10 log lines:
       >   File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 388, in build
       >     create_lib(out_filename, self.build_objects(build_dir))
       >   File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 381, in build_objects
       >     run_build_commands(commands)
       >   File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 89, in run_build_commands
       >     shared.run_multiple_processes(commands, env=clean_env())
       >   File "/nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/tools/shared.py", line 204, in run_multiple_processes
       >     raise Exception('Subprocess %d/%d failed (%s)! (cmdline: %s)' % (idx + 1, len(commands), returncode_to_str(finished_process.returncode), shlex_join(commands[idx])))
       > Exception: Subprocess 1/2 failed (returned 1)! (cmdline: /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/bin/emcc -Werror -fno-unroll-loops -O2 -fno-builtin -g -flto=full -DNDEBUG -c /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/system/lib/dlmalloc.c -o /nix/store/09mqz0l3p66abz3hkjx38xy9kyah76ms-emscripten-2.0.26/share/emscripten/cache/build/libdlmalloc/dlmalloc.o)
       > builder for '/nix/store/ax3w8qc7rip89gfwdrcfddqf803np7gg-emscripten-2.0.26.drv' failed with exit code 1
       For full logs, run 'nix log /nix/store/ax3w8qc7rip89gfwdrcfddqf803np7gg-emscripten-2.0.26.drv'.

yu-re-ka · 2021-08-20T18:18:10Z

In particular, with LLVM12 on Linux (NixOS):
[...]

Emscripten requires LLVM 13 release candidates:
emscripten-core/emscripten#10072 (comment)

RaitoBezarius · 2021-08-20T19:41:37Z

In particular, with LLVM12 on Linux (NixOS):
[...]

Emscripten requires LLVM 13 release candidates:
emscripten-core/emscripten#10072 (comment)

~~I'm getting other errors with LLVM13rc1 I think~~

RaitoBezarius · 2021-08-20T20:16:22Z

Nevermind, sorry, it was due to me, but I'm getting errors like this with a simple project (GMP):

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by GNU MP configure 6.1.2, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure CC_FOR_BUILD=gcc CCAS=gcc -c CFLAGS=-m32 -DPIC -Oz -O3 --host=x86_64-pc-linux-gnu --build=i686-pc-linux-gnu --disable-assembly --prefix=/home/raito/dev/projects/Lean/lean/build/gmp-root

## --------- ##
## Platform. ##
## --------- ##

hostname = Thors
uname -m = x86_64
uname -r = 5.10.52
uname -s = Linux
uname -v = #1-NixOS SMP Tue Jul 20 14:05:59 UTC 2021

/usr/bin/uname -p = unknown
/bin/uname -X     = unknown

/bin/arch              = unknown
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /tmp/tmp.xBQTZu9JgZ/sysroot/bin
PATH: /tmp/tmp.xBQTZu9JgZ/sysroot/bin
PATH: /nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/bin
PATH: /nix/store/pyv3351wlsa5r8sa8bbjpici3ksbvcqc-gnumake-4.3/bin
PATH: /nix/store/p1lk3bb1wzyn5v3nw8nyxrc44iy60vwk-cmake-3.19.7/bin
PATH: /nix/store/m3irdnp0f6ra49fgyws231rycvkzfk0s-gnum4-1.4.18/bin
PATH: /nix/store/lxf3l92rrgwqmrmqlhgvkj62ayzr8x8b-clang-wrapper-13.0.0-rc1/bin
PATH: /nix/store/ps21y5rwwh8r1blfgc6q62givqakakp8-kitty-0.21.2/bin
PATH: /nix/store/3hxmczsja3i15v9138syxpk9a8lmjy50-imagemagick-7.1.0-4/bin
PATH: /nix/store/i05nf24h2p3kmfr74icyin9ln3ks9xh3-xsel-unstable-2020-05-27/bin
PATH: /nix/store/1lr5wgik95n8apy8lbq54v6s4jk94686-ncurses-6.2-dev/bin
PATH: /run/wrappers/bin
PATH: /home/raito/.nix-profile/bin
PATH: /etc/profiles/per-user/raito/bin
PATH: /nix/var/nix/profiles/default/bin
PATH: /run/current-system/sw/bin


## ----------- ##
## Core tests. ##
## ----------- ##

configure:3055: checking build system type
configure:3069: result: i686-pc-linux-gnu
configure:3089: checking host system type
configure:3102: result: x86_64-pc-linux-gnu
configure:3139: checking for a BSD-compatible install
configure:3207: result: /run/current-system/sw/bin/install -c
configure:3218: checking whether build environment is sane
configure:3273: result: yes
configure:3332: checking for x86_64-pc-linux-gnu-strip
configure:3362: result: no
configure:3372: checking for strip
configure:3402: result: no
configure:3424: checking for a thread-safe mkdir -p
configure:3463: result: /run/current-system/sw/bin/mkdir -p
configure:3470: checking for gawk
configure:3486: found /run/current-system/sw/bin/gawk
configure:3497: result: gawk
configure:3508: checking whether make sets $(MAKE)
configure:3530: result: yes
configure:3559: checking whether make supports nested variables
configure:3576: result: yes
configure:3705: checking whether to enable maintainer-specific portions of Makefiles
configure:3714: result: no
User:
ABI=
CC=/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/bin/emcc
CFLAGS=-m32 -DPIC -Oz -O3
CPPFLAGS=(unset)
MPN_PATH=
GMP:
abilist=64 x32 32
cclist=gcc icc cc
configure:5779: /nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/bin/emcc -c conftest.c >&5
cache:INFO: generating system headers: sysroot_install.stamp... (this will be cached in "/tmp/tmp.xBQTZu9JgZ/sysroot_install.stamp" for subsequent builds)
Traceback (most recent call last):
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/emcc.py", line 3774, in <module>
    sys.exit(main(sys.argv))
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/emcc.py", line 3767, in main
    ret = run(args)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/emcc.py", line 1074, in run
    linker_inputs = phase_compile_inputs(options, state, newargs, input_files)
  File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/emcc.py", line 2316, in phase_compile_inputs
    system_libs.ensure_sysroot()
  File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 2009, in ensure_sysroot
    shared.Cache.get('sysroot_install.stamp', install_system_headers, what='system headers')
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/tools/cache.py", line 165, in get
    creator(cachename)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1988, in install_system_headers
    copytree_exist_ok(src, dest)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1964, in copytree_exist_ok
    copytree_exist_ok(srcname, dstname)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/tools/system_libs.py", line 1966, in copytree_exist_ok
    shared.safe_copy(srcname, dstname)
  File "/nix/store/yw06kcgg0akbmpk32mqc8763s9j4q5zw-emscripten-2.0.26/share/emscripten/tools/shared.py", line 765, in safe_copy
    shutil.copy(src, dst)
  File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/tmp/tmp.xBQTZu9JgZ/sysroot/include/bits/errno.h'

I tried to change the cache using EM_CACHE as it was trying to write in the read-only cache of the Nix store.

yu-re-ka · 2021-08-20T20:22:01Z

Is this problem introduced by the changes in this PR?

RaitoBezarius · 2021-08-20T20:26:46Z

Is this problem introduced by the changes in this PR?

I'm not getting it using emscripten 2.0.10 (but other errors related to the mismatch with the LLVM version… invalid symbols JSON is generated by wasm-finalize).

EDIT: to be precise, I suppose there must be a knob to say to emscripten, store these cached libs in some mutable path.

https://bugs.llvm.org/show_bug.cgi?id=50611

Co-authored-by: Alexander Khovansky <[email protected]>

yu-re-ka · 2021-08-23T07:58:02Z

@RaitoBezarius can you provide the full expression that builds on nixpkgs master, but not on this branch?

RaitoBezarius · 2021-08-25T09:36:48Z

@RaitoBezarius can you provide the full expression that builds on nixpkgs master, but not on this branch?

On master, I don't know. On nixpkgs unstable (more or less recent versions of it), https://github.com/RaitoBezarius/nixexprs/blob/master/pkgs/science/lean/emscripten.nix shows two very different errors with the emscripten 2.0.10 and the emscripten on this branch.

The first one fails due to LLVM version mismatches (very late), the second one fails due to being unable to write in the default cache directory of emscripten which is a nix store path (very early).

To reproduce this, having a local checkout of my repository and doing nix-build -E 'import ./default.nix {}; with pkgs.lean."3.32.1"; [ emscripten (emscripten.override { emscripten = <provide the emscripten of this branch>; }) ] should exhibit the behaviors mentioned above.

All in all, I do not want this to be a blocking point of this pull request, if people are okay with it, I prefer to move on and see what can be done to fix it.

7c6f434c · 2021-08-26T19:33:29Z

@happysalada hmm, it looks like you reported new Darwin failures in straight clang (13)? Are those new?

yu-re-ka · 2021-08-26T22:44:45Z

Our packaging of LLVM 13 release candidates never worked on darwin. Also see here.

yu-re-ka requested a review from matthewbauer as a code owner August 9, 2021 04:38

yu-re-ka requested review from gebner and primeos August 9, 2021 04:51

ofborg bot requested review from 7c6f434c, asppsa, dtzWill, lovek323 and qknight August 9, 2021 04:51

ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Aug 9, 2021

SuperSandro2000 reviewed Aug 9, 2021

View reviewed changes

pkgs/development/compilers/emscripten/default.nix Outdated Show resolved Hide resolved

yu-re-ka requested a review from SuperSandro2000 August 9, 2021 10:13

happysalada reviewed Aug 9, 2021

View reviewed changes

pkgs/development/compilers/emscripten/default.nix Outdated Show resolved Hide resolved

yu-re-ka requested a review from happysalada August 9, 2021 16:41

yu-re-ka mentioned this pull request Aug 10, 2021

malloc results are not aligned to alignof(max_align_t) emscripten-core/emscripten#10072

Closed

ofborg bot added 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. and removed 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Aug 10, 2021

ofborg bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Aug 10, 2021

yuyuyureka and others added 8 commits August 20, 2021 23:07

llvm_13: workaround for llvm bug 50611

91b15b6

https://bugs.llvm.org/show_bug.cgi?id=50611

llvm_13: fix tests on non-x86 platforms

48d1e39

llvmPackages_*: expose release_version

b0f27ee

llvmPackages_13.libcxx: mark as broken on darwin

f8230bb

binaryen: 99 -> 101

d6869f3

emscripten: 2.0.10 -> 2.0.27

691464d

emscriptenPackages.zlib: fix checks

0d8f369

Co-authored-by: Alexander Khovansky <[email protected]>

emscriptenPackages.json_c: fix build

d7ac3b4

Co-authored-by: Alexander Khovansky <[email protected]>

7c6f434c merged commit 7c3be09 into NixOS:master Aug 26, 2021

midchildan mentioned this pull request Nov 19, 2021

Downgrade llvmPackages_latest to 12 on Darwin #146517

Closed

12 tasks

Uh oh!

Conversation

yu-re-ka commented Aug 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Things done

Uh oh!

yu-re-ka commented Aug 9, 2021

Uh oh!

Uh oh!

Uh oh!

yu-re-ka commented Aug 9, 2021

Uh oh!

yu-re-ka commented Aug 9, 2021

Uh oh!

happysalada commented Aug 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

happysalada commented Aug 10, 2021

Uh oh!

happysalada commented Aug 10, 2021

Uh oh!

yu-re-ka commented Aug 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

happysalada commented Aug 10, 2021

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

happysalada commented Aug 10, 2021

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

primeos commented Aug 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yu-re-ka commented Aug 10, 2021

Uh oh!

happysalada commented Aug 11, 2021

Uh oh!

primeos commented Aug 13, 2021

Uh oh!

yu-re-ka commented Aug 13, 2021

Uh oh!

RaitoBezarius commented Aug 20, 2021

Uh oh!

RaitoBezarius commented Aug 20, 2021

Uh oh!

yu-re-ka commented Aug 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RaitoBezarius commented Aug 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RaitoBezarius commented Aug 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yu-re-ka commented Aug 20, 2021

Uh oh!

RaitoBezarius commented Aug 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yu-re-ka commented Aug 23, 2021

Uh oh!

RaitoBezarius commented Aug 25, 2021

Uh oh!

7c6f434c commented Aug 26, 2021

Uh oh!

yu-re-ka commented Aug 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

yu-re-ka commented Aug 9, 2021 •

edited

Loading

happysalada commented Aug 10, 2021 •

edited

Loading

yu-re-ka commented Aug 10, 2021 •

edited

Loading

primeos commented Aug 10, 2021 •

edited

Loading

yu-re-ka commented Aug 20, 2021 •

edited

Loading

RaitoBezarius commented Aug 20, 2021 •

edited

Loading

RaitoBezarius commented Aug 20, 2021 •

edited

Loading

RaitoBezarius commented Aug 20, 2021 •

edited

Loading

yu-re-ka commented Aug 26, 2021 •

edited

Loading