qscintilla: fix build on darwin by willcohen · Pull Request #150595 · NixOS/nixpkgs

willcohen · 2021-12-13T20:44:07Z

Motivation for this change

Per #71398, qscintilla is currently broken on darwin.

Things done

Mechanism of change

@lsix @peterhoeg

Reorganize qscintilla packages to make qt5 the default and move the legacy version to qscintilla-qt4:

Moves current qscintilla to qscintilla-qt4 and drops its support for both qt5 and darwin.
Makes current qscintilla-qt5 the default for qscintilla and migrates to libsForQt5.
Moves current qscintilla python package to qscintilla-qt4.

Upgrading qscintilla version:

Update qscintilla to 2.13.1
Convert qscintilla-qt5 to use pyproject.toml and ensure build on darwin.
Adjust derivations using qscintilla and qt4 to specify qscintilla-qt4

Previous issues before patch was ready for review

~~Trying to build the current version leads to issues the following build problem on Darwin:~~

$ nix-build $NIXPKGS -A python3Packages.qscintilla-qt5
                               
this derivation will be built:
  /nix/store/g3rv06mv76dg46gcfk80gnr89d111jhr-python3.9-qscintilla-2.11.6.drv
building '/nix/store/g3rv06mv76dg46gcfk80gnr89d111jhr-python3.9-qscintilla-2.11.6.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
qtPreHook
unpacking sources
unpacking source archive /nix/store/v8297rqiv4wd1zyizwykwz74qbqd3dkz-QScintilla-2.11.6.tar.gz
source root is QScintilla-2.11.6
setting SOURCE_DATE_EPOCH to timestamp 1606139171 of file QScintilla-2.11.6/doc/html-Qt4Qt5/splitbar.png
patching sources
configuring
Configuring QScintilla 2.11.6...
QScintilla 2.11.6 is being used.
The QScintilla .sip files will be installed in
/nix/store/sl09w0ciwcsk74b89h1a13d5ni8dmpvz-python3.9-qscintilla-2.11.6/share/sip/PyQt5.
QScintilla will be installed in
/nix/store/sl09w0ciwcsk74b89h1a13d5ni8dmpvz-python3.9-qscintilla-2.11.6/lib/python3.9/site-packages/PyQt5.
PyQt 5.15.4 is being used.
Qt 5.12.10 is being used.
sip 4.19.25 is being used.
The sip executable is
/nix/store/n5l38hjfqhly3xzzvx83yyypfzx6lm90-python3.9-sip-4.19.25/bin/sip.
QScintilla is being built with 'protected' redefined as 'public'.
The PEP 484 stubs will be installed in
/nix/store/sl09w0ciwcsk74b89h1a13d5ni8dmpvz-python3.9-qscintilla-2.11.6/lib/python3.9/site-packages/PyQt5.
The QScintilla API file will be installed in
/nix/store/sl09w0ciwcsk74b89h1a13d5ni8dmpvz-python3.9-qscintilla-2.11.6/api/python3.9/api/python.
Generating the C++ source for the Qsci module...
Generating the .pro file for the Qsci module...
Generating the QScintilla API file...
Generating the top-level .pro file...
Generating the Makefiles...
Error: /nix/store/8mkvcnwm55ahy2lra5krhc1zxbicx6wx-qtbase-5.12.10-dev/bin/qmake
failed to create a Makefile from QScintilla.pro.
error: builder for '/nix/store/g3rv06mv76dg46gcfk80gnr89d111jhr-python3.9-qscintilla-2.11.6.drv' failed with exit code 1;
       last 10 log lines:
       > /nix/store/sl09w0ciwcsk74b89h1a13d5ni8dmpvz-python3.9-qscintilla-2.11.6/lib/python3.9/site-packages/PyQt5.
       > The QScintilla API file will be installed in
       > /nix/store/sl09w0ciwcsk74b89h1a13d5ni8dmpvz-python3.9-qscintilla-2.11.6/api/python3.9/api/python.
       > Generating the C++ source for the Qsci module...
       > Generating the .pro file for the Qsci module...
       > Generating the QScintilla API file...
       > Generating the top-level .pro file...
       > Generating the Makefiles...
       > Error: /nix/store/8mkvcnwm55ahy2lra5krhc1zxbicx6wx-qtbase-5.12.10-dev/bin/qmake
       > failed to create a Makefile from QScintilla.pro.
       For full logs, run 'nix log /nix/store/g3rv06mv76dg46gcfk80gnr89d111jhr-python3.9-qscintilla-2.11.6.drv'.

The newest version of QScintilla drops support for Qt4. Unsure how to get past this failed to create a Makefile from QScintilla.pro error, I have tried updating to latest qscintilla. Accordingly, this draft PR performs a few tasks, none of them successfully.

The C portion of qscintilla still builds successfully on darwin. Unfortunately, in the move to pyproject, I am having trouble getting sip configured correctly. I've tried to patch in additional PyQt5 includes into the toml file, but the build still fails with the error AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel', an issue I am not sure how to resolve:

$ nix-build $NIXPKGS -A python3Packages.qscintilla-qt5
                               
this derivation will be built:
  /nix/store/61fpap2s2lq2x567awvf7sgpsf3n5kx0-python3.9-qscintilla-2.13.1.drv
building '/nix/store/61fpap2s2lq2x567awvf7sgpsf3n5kx0-python3.9-qscintilla-2.13.1.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing pip-build-hook
Using pipBuildPhase
Using pipShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
qtPreHook
unpacking sources
unpacking source archive /nix/store/ljszkfby50a4vzdqfhi7h4vwybjqmhzd-QScintilla_src-2.13.1.tar.gz
source root is QScintilla_src-2.13.1
setting SOURCE_DATE_EPOCH to timestamp 1634205963 of file QScintilla_src-2.13.1/doc/html/splitbar.png
patching sources
configuring
no configure script, doing nothing
building
Executing pipBuildPhase
Creating a wheel...
WARNING: The directory '/homeless-shelter/Library/Caches/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Ignoring indexes: https://pypi.org/simple
Created temporary directory: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-ephem-wheel-cache-b0v6s6hl
Created temporary directory: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv
Initialized build tracking at /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv
Created build tracker: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv
Entered build tracker: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv
Created temporary directory: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-wheel-j3qxbh8n
Processing /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/QScintilla_src-2.13.1/Python
  Created temporary directory: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-build-a51b2ng9
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
  Added file:///private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/QScintilla_src-2.13.1/Python to build tracker '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv'
    Created temporary directory: /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-modern-metadata-vxclppiu
    Running command /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmporbo4upf
    Traceback (most recent call last):
      File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 126, in prepare_metadata_for_build_wheel
        hook = backend.prepare_metadata_for_build_wheel
    AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in <module>
        main()
      File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in prepare_metadata_for_build_wheel
        return _get_wheel_metadata_from_wheel(backend, metadata_directory,
      File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 159, in _get_wheel_metadata_from_wheel
        whl_basename = backend.build_wheel(metadata_directory, config_settings)
      File "/nix/store/drghjg1bsrxn6jc4m7n9xggk94jgb2qa-python3.9-sip-6.1.1/lib/python3.9/site-packages/sipbuild/api.py", line 51, in build_wheel
        project = AbstractProject.bootstrap('pep517')
      File "/nix/store/drghjg1bsrxn6jc4m7n9xggk94jgb2qa-python3.9-sip-6.1.1/lib/python3.9/site-packages/sipbuild/abstract_project.py", line 83, in bootstrap
        project.setup(pyproject, tool, tool_description)
      File "/nix/store/drghjg1bsrxn6jc4m7n9xggk94jgb2qa-python3.9-sip-6.1.1/lib/python3.9/site-packages/sipbuild/project.py", line 484, in setup
        self.apply_user_defaults(tool)
      File "project.py", line 47, in apply_user_defaults
        super().apply_user_defaults(tool)
      File "/nix/store/4gznpqar782bxly71y2dvqd4sd5a25rh-python3.9-pyqt-builder-1.10.1/lib/python3.9/site-packages/pyqtbuild/project.py", line 70, in apply_user_defaults
        super().apply_user_defaults(tool)
      File "/nix/store/drghjg1bsrxn6jc4m7n9xggk94jgb2qa-python3.9-sip-6.1.1/lib/python3.9/site-packages/sipbuild/project.py", line 230, in apply_user_defaults
        self.builder.apply_user_defaults(tool)
      File "/nix/store/4gznpqar782bxly71y2dvqd4sd5a25rh-python3.9-pyqt-builder-1.10.1/lib/python3.9/site-packages/pyqtbuild/builder.py", line 66, in apply_user_defaults
        raise PyProjectOptionException('qmake',
    sipbuild.pyproject.PyProjectOptionException
    Preparing wheel metadata ... error
WARNING: Discarding file:///private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/QScintilla_src-2.13.1/Python. Command errored out with exit status 1: /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmporbo4upf Check the logs for full command output.
ERROR: Command errored out with exit status 1: /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmporbo4upf Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
    status = self.run(options, args)
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/commands/wheel.py", line 142, in run
    requirement_set = resolver.resolve(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 103, in resolve
    r = self.factory.make_requirement_from_install_req(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 444, in make_requirement_from_install_req
    raise self._build_failures[ireq.link]
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 200, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 306, in __init__
    super().__init__(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 151, in __init__
    self.dist = self._prepare()
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 234, in _prepare
    dist = self._prepare_distribution()
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 317, in _prepare_distribution
    return self._factory.preparer.prepare_linked_requirement(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 508, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 570, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 60, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 36, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 549, in prepare_metadata
    self.metadata_directory = self._generate_metadata()
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 534, in _generate_metadata
    return generate_metadata(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/operations/build/metadata.py", line 31, in generate_metadata
    distinfo_dir = backend.prepare_metadata_for_build_wheel(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 184, in prepare_metadata_for_build_wheel
    return self._call_hook('prepare_metadata_for_build_wheel', {
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
    self._subprocess_runner(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 274, in runner
    call_subprocess(
  File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 244, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmporbo4upf Check the logs for full command output.
Removed file:///private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/QScintilla_src-2.13.1/Python from build tracker '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv'
Removed build tracker: '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv'
error: builder for '/nix/store/61fpap2s2lq2x567awvf7sgpsf3n5kx0-python3.9-qscintilla-2.13.1.drv' failed with exit code 1;
       last 10 log lines:
       >     return self._call_hook('prepare_metadata_for_build_wheel', {
       >   File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
       >     self._subprocess_runner(
       >   File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 274, in runner
       >     call_subprocess(
       >   File "/nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 244, in call_subprocess
       >     raise InstallationSubprocessError(proc.returncode, command_desc)
       > pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmporbo4upf Check the logs for full command output.
       > Removed file:///private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/QScintilla_src-2.13.1/Python from build tracker '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv'
       > Removed build tracker: '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/pip-req-tracker-xqcpiapv'
       For full logs, run 'nix log /nix/store/61fpap2s2lq2x567awvf7sgpsf3n5kx0-python3.9-qscintilla-2.13.1.drv'.

Partially resolves: #71398

willcohen · 2021-12-13T23:39:34Z

Additional googling suggests that this metadata issue means that sip-install needs additional flags or configuration to run successfully. Will keep looking into it. The same issue applies even when falling back to build 2.11.5 with the toml, so it’s something with my config and not with the new version.

pkgs/development/python-modules/qscintilla-qt5/default.nix

willcohen · 2021-12-14T17:08:53Z

sip now works and is able to start the build, which now fails with:

    Compiling the project...
    make
    _in_process.py: 'make' failed returning 2
    cd Qsci/ && ( test -e Makefile || /nix/store/8mkvcnwm55ahy2lra5krhc1zxbicx6wx-qtbase-5.12.10-dev/bin/qmake -o Makefile /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmpcvxmua7e/Qsci/Qsci.pro ) && make -f Makefile
    make[1]: Entering directory '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmpcvxmua7e/Qsci'
    clang++ -c -pipe -stdlib=libc++ -O2 -fno-exceptions -Wall -W -fPIC -DPy_LIMITED_API=0x03040000 -DSIP_PROTECTED_IS_PUBLIC -Dprotected=public -DQT_NO_EXCEPTIONS -DQT_NO_DEBUG -DQT_PLUGIN -DQT_GUI_LIB -DQT_CORE_LIB -I. -I. -I.. -isystem /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/include/python3.9 -isystem /nix/store/8mkvcnwm55ahy2lra5krhc1zxbicx6wx-qtbase-5.12.10-dev/include/QtGui -isystem /nix/store/8mkvcnwm55ahy2lra5krhc1zxbicx6wx-qtbase-5.12.10-dev/include/QtCore -I. -I/nix/store/jhj8q4vq5x3y943d8r9iyai4flsmmxch-apple-framework-OpenGL/Library/Frameworks/OpenGL.framework/Headers -I/nix/store/jwb156jli8d0236anmjlrxyflpz4ln7a-apple-framework-AGL/Library/Frameworks/AGL.framework/Headers -I/nix/store/8mkvcnwm55ahy2lra5krhc1zxbicx6wx-qtbase-5.12.10-dev/mkspecs/macx-clang -o sipQscicmodule.o sipQscicmodule.cpp
    /nix/store/imnrnq7sm16zxgxsmi9r0iqlj2jqq9c5-python3.9-PyQt5-5.15.4/lib/python3.9/site-packages/PyQt5/bindings/QtWidgets/qwidget.sip:28:10: fatal error: 'qwidget.h' file not found
    #include <qwidget.h>
             ^~~~~~~~~~~
    1 error generated.
    make[1]: *** [Makefile:553: sipQscicmodule.o] Error 1
    make[1]: Leaving directory '/private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmpcvxmua7e/Qsci'
    make: *** [Makefile:50: sub-Qsci-make_first-ordered] Error 2
    Preparing wheel metadata ... error
WARNING: Discarding file:///private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/QScintilla_src-2.13.1/Python. Command errored out with exit status 1: /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmp_hg34w_z Check the logs for full command output.
ERROR: Command errored out with exit status 1: /nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/bin/python3.9 /nix/store/h5j6wymds31fgj0pdhfrdh27qhlw6d1v-python3.9-pip-21.1.3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/nix-build-python3.9-qscintilla-2.13.1.drv-0/tmp_hg34w_z Check the logs for full command output.

@lsix this issue with qwidget.h seems very similar to #72302, with the new issue that configure.py can't be patched anymore. Based on the sip-install help, it would seem that in addition to the settings being set to provide the locations for the various qscintilla files, that adding --qmake-setting 'QT += widgets' --qmake-setting 'QT += printsupport' to the build command be an equivalent solution to what was done before, but as seen on this branch the build still fails.

$ nix-build $NIXPKGS --run-env -A python3Packages.qscintilla-qt5
[...]
$ sip-install --help
usage: sip-install [-h] [-V] [--quiet] [--verbose] [--build-dir DIR] [--no-distinfo] [--scripts-dir DIR] [--target-dir DIR] [--api-dir DIR] [--android-abi ABI]
                   [--link-full-dll] [--qml-debug] [--jobs N] [--qmake FILE] [--qmake-setting 'NAME += VALUE'] [--spec SPEC] [--concatenate N] [--debug] [--no-docstrings]
                   [--pep484-pyi] [--protected-is-public] [--no-protected-is-public] [--tracing] [--qsci-features-dir DIR] [--qsci-include-dir DIR] [--qsci-library-dir DIR]

Build and install the project.

optional arguments:
[...]
  --qmake-setting 'NAME += VALUE'
                        add the 'NAME += VALUE' setting to any .pro file
[...]
  --qsci-features-dir DIR
                        the qscintilla2.prf features file is in DIR
  --qsci-include-dir DIR
                        the Qsci include file directory is in DIR
  --qsci-library-dir DIR
                        the QScintilla library is in DIR

Additionally, I went back and tried to clean up the commit messages to get this closer to a workable PR.

willcohen · 2021-12-14T18:31:36Z

With 6706e4c (and a slightly brittle substituteInPlace inspired by #72249), PR now completes the make step on darwin for qscintilla-qt5, though it still fails the python imports check:

...
Executing pythonImportsCheckPhase
Check whether the following modules can be imported: PyQt5.Qsci
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/nix/store/w8lqcsrwlwrvmchj46ph9880w9h4m50a-python3-3.9.6/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 565, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1173, in create_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ImportError: dlopen(/nix/store/wrc59by9az0lglzcm9lm6lbawsjgxy65-python3.9-qscintilla-2.13.1/lib/python3.9/site-packages/PyQt5/Qsci.abi3.so, 0x0002): symbol not found in flat namespace '__ZN10QsciLexerD16staticMetaObjectE'

willcohen · 2021-12-14T21:51:36Z

With 4362b38 this now builds, installs, and links successfully on darwin. Because qscintilla now specifies qt5 as a minimum (I figure this makes sense since qt6 is already being worked on as an additional package for nix), I tried to find all uses of qscintilla that still are on qt4, which turned up miniaudicle and sqliteman.

willcohen · 2021-12-14T22:47:06Z

Result of nixpkgs-review pr 150595 run on x86_64-darwin 1

1 package marked as broken and skipped:

qscintilla-qt4

1 package failed to build:

openscad

6 packages built:

qscintilla (libsForQt5.qscintilla)
libsForQt514.qscintilla
libsForQt515.qscintilla (plasma5Packages.qscintilla)
octaveFull
python38Packages.qscintilla (python38Packages.qscintilla-qt5)
python39Packages.qscintilla (python39Packages.qscintilla-qt5)

willcohen · 2021-12-14T23:44:16Z

Result of nixpkgs-review pr 150595 run on x86_64-linux 1

2 packages failed to build:

qgis
qgis-unwrapped

11 packages built:

qscintilla (libsForQt5.qscintilla)
libsForQt512.qscintilla
libsForQt514.qscintilla
octaveFull
openscad
ovito
python38Packages.qscintilla (python38Packages.qscintilla-qt5)
python39Packages.qscintilla (python39Packages.qscintilla-qt5)
qscintilla-qt4
tora
tortoisehg

willcohen · 2021-12-15T00:09:55Z

On linux, qgis-unwrapped currently fails with:

error: builder for '/nix/store/gk8mdaw5h55k4yagcjp8arrb5q95xbrj-qgis-unwrapped-3.16.14.drv' failed with exit code 1;
       last 10 log lines:
       > [2780/5436] Generating gui/sip_guipart0.cpp, gui/sip_guipart1.cpp, gui/sip_guipart2.cpp, gui/sip_guipart3.cpp, gui/sip_guipart4.cpp, gui/sip_guipart5.cpp, gui/sip_guipart6.cpp, gui/sip_guipart7.cpp, gui/sip_guipart8.cpp, gui/sip_guipart9.cpp, gui/sip_guipart10.cpp/sip_corepart10.cppip_analysispart8.cpp, analysis/sip_analysispart9.cpp, analysis/sip_analysispart10.cpp
       > FAILED: python/gui/sip_guipart0.cpp python/gui/sip_guipart1.cpp python/gui/sip_guipart2.cpp python/gui/sip_guipart3.cpp python/gui/sip_guipart4.cpp python/gui/sip_guipart5.cpp python/gui/sip_guipart6.cpp python/gui/sip_guipart7.cpp python/gui/sip_guipart8.cpp python/gui/sip_guipart9.cpp python/gui/sip_guipart10.cpp /build/source/build/python/gui/sip_guipart0.cpp /build/source/build/python/gui/sip_guipart1.cpp /build/source/build/python/gui/sip_guipart2.cpp /build/source/build/python/gui/sip_guipart3.cpp /build/source/build/python/gui/sip_guipart4.cpp /build/source/build/python/gui/sip_guipart5.cpp /build/source/build/python/gui/sip_guipart6.cpp /build/source/build/python/gui/sip_guipart7.cpp /build/source/build/python/gui/sip_guipart8.cpp /build/source/build/python/gui/sip_guipart9.cpp /build/source/build/python/gui/sip_guipart10.cpp
       > cd /build/source/build/python && /nix/store/q2lj6pxd6sn75fa9pjx9x3jpgxnr6zld-cmake-3.21.2/bin/cmake -E echo && /nix/store/q2lj6pxd6sn75fa9pjx9x3jpgxnr6zld-cmake-3.21.2/bin/cmake -E touch /build/source/build/python/gui/sip_guipart0.cpp /build/source/build/python/gui/sip_guipart1.cpp /build/source/build/python/gui/sip_guipart2.cpp /build/source/build/python/gui/sip_guipart3.cpp /build/source/build/python/gui/sip_guipart4.cpp /build/source/build/python/gui/sip_guipart5.cpp /build/source/build/python/gui/sip_guipart6.cpp /build/source/build/python/gui/sip_guipart7.cpp /build/source/build/python/gui/sip_guipart8.cpp /build/source/build/python/gui/sip_guipart9.cpp /build/source/build/python/gui/sip_guipart10.cpp && /nix/store/kp9dpdpgf1gd3r9z33r7zfy49qjdxwmw-python3.9-sip-4.19.25/bin/sip -w -e -x ANDROID -x ARM -x MOBILITY_LOCATION -x VECTOR_MAPPED_TYPE -n PyQt5.sip -t Qt_5_15_0 -t WS_X11 -g -o -a /build/source/build/python/qgis.gui.api -n PyQt5.sip -y /build/source/build/output/python/qgis/_gui.pyi -I /nix/store/1dz6hmnlx15qwcqd4bvwa71qzalagzxg-python3.9-qscintilla-2.13.1/share/sip/PyQt5 -j 11 -c /build/source/build/python/gui -I /build/source/build/python/gui -I /nix/store/knynibwmvj63w0ak6w307dwd4nhcd1d7-python3.9-PyQt5-5.15.4/lib/python3.9/site-packages/PyQt5/bindings -I /build/source/python /build/source/build/python/gui/gui.sip
       >
       > sip: Unable to find file "Qsci/qscimod5.sip"
       > [2782/5436] Generating analysis/sip_analysispart0.cpp, analysis/sip_analysispart1.cpp, analysis/sip_analysispart2.cpp, analysis/sip_analysispart3.cpp, analysis/sip_analysispart4.cpp, analysis/sip_analysispart5.cpp, analysis/sip_analysispart6.cpp, analysis/sip_analysispart7.cpp, analysis/sip_analysispart8.cpp, analysis/sip_analysispart9.cpp, analysis/sip_analysispart10.cpp
       >
       > [2783/5436] Generating core/sip_corepart0.cpp, core/sip_corepart1.cpp, core/sip_corepart2.cpp, core/sip_corepart3.cpp, core/sip_corepart4.cpp, core/sip_corepart5.cpp, core/sip_corepart6.cpp, core/sip_corepart7.cpp, core/sip_corepart8.cpp, core/sip_corepart9.cpp, core/sip_corepart10.cpp
       >
       > ninja: build stopped: subcommand failed.
       For full logs, run 'nix log /nix/store/gk8mdaw5h55k4yagcjp8arrb5q95xbrj-qgis-unwrapped-3.16.14.drv'.
error: 1 dependencies of derivation '/nix/store/9z141mkxg1wdi84arxdd2yxmz474phf7-qgis-3.16.14.drv' failed to build

Revisions to address the QGIS package should probably be done alongside #150286

pkgs/top-level/all-packages.nix

pkgs/development/libraries/qscintilla-qt4/default.nix

pkgs/development/python-modules/qscintilla-qt4/default.nix

willcohen · 2022-01-06T00:04:42Z

@prusnak I made all the changes accordingly (and managed to pull in a million other upstream changes when rebasing -- apologies to all for the noise.)

willcohen · 2022-01-06T01:13:24Z

@prusnak the changes from your review are now just in the commit 8dfa516, and ofborg still passes all the builds. Please let me know if you see anything else!

prusnak

Left couple more cleanup nits. Otherwise good.

pkgs/top-level/qt5-packages.nix

pkgs/top-level/all-packages.nix

pkgs/development/python-modules/qscintilla-qt5/default.nix

pkgs/development/libraries/qscintilla/default.nix

willcohen · 2022-01-06T14:48:03Z

@prusnak Thanks! Comments applied with b8504e9 and a98370a.

prusnak · 2022-01-06T16:04:01Z

@GrahamcOfBorg build miniaudicle qgis sqliteman python3Packages.qscintilla-qt4 python3Packages.qscintilla-qt5

siraben · 2022-01-06T16:25:07Z

Result of nixpkgs-review pr 150595 run on x86_64-darwin 1

1 package built:

qscintilla

prusnak

LGTM!

prusnak · 2022-01-06T17:03:11Z

Small follow-up PR: #153744

prusnak · 2022-01-06T17:23:25Z

pkgs/development/python-modules/qscintilla-qt4/default.nix

+, pyqt4
+}:
+
+disabledIf (isPy3k || isPyPy)


Btw, where does this limitation come from? I don't see it in the previous code

It was just a 100% copy of what was in https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/qscintilla/default.nix -- I didn't do any of the refactoring you're looking into now!

ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Dec 13, 2021

github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Dec 13, 2021

ofborg bot added the 8.has: package (new) This PR adds a new package label Dec 13, 2021

ofborg bot requested review from lsix and peterhoeg December 13, 2021 20:56

ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Dec 13, 2021

veprbl reviewed Dec 14, 2021

View reviewed changes

pkgs/development/python-modules/qscintilla-qt5/default.nix Outdated Show resolved Hide resolved

willcohen changed the title ~~Initial work for qscintilla on darwin~~ WIP: qscintilla: fix build on darwin Dec 14, 2021

willcohen force-pushed the qscintilla branch 2 times, most recently from e99d793 to d8c95d6 Compare December 14, 2021 16:57

willcohen force-pushed the qscintilla branch from 2a7b1e7 to 716aca8 Compare December 14, 2021 21:44

willcohen changed the title ~~WIP: qscintilla: fix build on darwin~~ qscintilla: fix build on darwin Dec 14, 2021

willcohen force-pushed the qscintilla branch from 716aca8 to f77d6c0 Compare December 14, 2021 21:49

willcohen marked this pull request as ready for review December 14, 2021 21:51

willcohen requested review from FRidh and jonringer as code owners December 14, 2021 21:51

willcohen mentioned this pull request Dec 15, 2021

Improvements to qgis and grass packaging, introduce qgis-ltr derivation #150286

Closed

13 tasks

ofborg bot requested review from erictapen and sikmir December 15, 2021 03:35

veprbl reviewed Dec 15, 2021

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

pkgs/development/libraries/qscintilla-qt4/default.nix Outdated Show resolved Hide resolved

pkgs/development/python-modules/qscintilla-qt4/default.nix Outdated Show resolved Hide resolved

willcohen force-pushed the qscintilla branch from ffa3326 to c059c9f Compare December 15, 2021 17:02

ofborg bot requested a review from eikek December 15, 2021 17:28

github-actions bot removed the 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. label Jan 6, 2022

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 6, 2022

ofborg bot requested review from erictapen and fgaz and removed request for aanderse, erictapen, fgaz and jonringer January 6, 2022 00:15

erictapen removed their request for review January 6, 2022 08:58

prusnak requested changes Jan 6, 2022

View reviewed changes

willcohen added 2 commits January 6, 2022 08:30

qscintilla: cleanup of derivations

b8504e9

qt5: remove stdenv from qt5-packages

a98370a

ofborg bot requested review from erictapen and fgaz and removed request for erictapen January 6, 2022 13:50

prusnak approved these changes Jan 6, 2022

View reviewed changes

siraben merged commit 88d7e0f into NixOS:master Jan 6, 2022

willcohen deleted the qscintilla branch January 6, 2022 16:52

prusnak mentioned this pull request Jan 6, 2022

qscintilla: cleanup #153744

Merged

13 tasks

prusnak reviewed Jan 6, 2022

View reviewed changes

willcohen mentioned this pull request Jan 6, 2022

ovito: mark as broken on darwin #153753

Merged

13 tasks

willcohen mentioned this pull request Feb 2, 2022

qgis, qgis-ltr: fix dependencies on darwin #157862

Open

13 tasks

Janik-Haag added the 12.first-time contribution This PR is the author's first one; please be gentle! label Jun 12, 2023

Uh oh!

Conversation

willcohen commented Dec 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Things done

Mechanism of change

Previous issues before patch was ready for review

Uh oh!

willcohen commented Dec 13, 2021

Uh oh!

Uh oh!

willcohen commented Dec 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

willcohen commented Dec 14, 2021

Uh oh!

willcohen commented Dec 14, 2021

Uh oh!

willcohen commented Dec 14, 2021

Uh oh!

willcohen commented Dec 14, 2021

Uh oh!

willcohen commented Dec 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

willcohen commented Jan 6, 2022

Uh oh!

willcohen commented Jan 6, 2022

Uh oh!

prusnak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

willcohen commented Jan 6, 2022

Uh oh!

prusnak commented Jan 6, 2022

Uh oh!

siraben commented Jan 6, 2022

Uh oh!

prusnak left a comment

Choose a reason for hiding this comment

Uh oh!

prusnak commented Jan 6, 2022

Uh oh!

prusnak Jan 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willcohen Jan 6, 2022

Choose a reason for hiding this comment

Uh oh!

prusnak Jan 6, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

willcohen commented Dec 13, 2021 •

edited

Loading

willcohen commented Dec 14, 2021 •

edited

Loading

willcohen commented Dec 15, 2021 •

edited

Loading

prusnak Jan 6, 2022 •

edited

Loading