Skip to content

python3Packages.wxPython_4_0, python3Packages.wxPython_4_1: add numpy…#127768

Merged
SuperSandro2000 merged 1 commit intoNixOS:masterfrom
deliciouslytyped:wxpython
Jun 22, 2021
Merged

python3Packages.wxPython_4_0, python3Packages.wxPython_4_1: add numpy…#127768
SuperSandro2000 merged 1 commit intoNixOS:masterfrom
deliciouslytyped:wxpython

Conversation

@deliciouslytyped
Copy link
Contributor

… and pillow to propagatedBuildInputs as required by upstream's dependency settings

See In light of #94108 (comment) and #94108 (comment) (two consecutive comments) for additional thoughts, quoted here:

Hm. https://discuss.wxpython.org/t/does-wxpython-4-0-6-must-also-install-numpy/32969/2
Of course it's never simple.. (that's also a pretty old post, IDK if things have changed)

I also don't quite understand why no one has run into the issue of buildPythonApplication trying to require numpy and pillow due to wxPython yet.

pip doesn't seem to provide any sane way of excluding deps pypa/pip#3090 , so I probably need to use some sort of indirect method if I deem them unnecessary.

Well, Robin (discuss.wxpython.org link above) does list --no-deps, but that seems rather shotgun.

The concrete example of these deps attempting to be pulled is this: f33d2a5#diff-9e9c51bfcf7e9562777c9c8d1b946f5a61995d0947955d188c7f1fc5b9a4c5c0R27

When run you get:

Executing pipInstallPhase
/build/source/dist /build/source
Processing ./WikidPad-2.4a1-py3-none-any.whl
Requirement already satisfied: wxpython>=4.0 in /nix/store/qy1ya8pcgjcjfhqvkm43pzb71w80122r-python3.7-wxPython-4.0.7.post2/lib/python3.7/site-packages (from WikidPad==2.4a1) (4.0.7.post2)
INFO: pip is looking at multiple versions of wikidpad to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement pillow (from wxpython)
ERROR: No matching distribution found for pillow

I'm entirely unsure what the best thing to do here is.
Apparently wxPython has numpy and pillow them as runtime dependencies because they're needed relatively often, but they're actually kind of optional?
Additionally there are "other packages" that are used less commonly, which are also runtime dependencies, but they aren't listed in the requirements because they're needed less often?

(Where does one find a list of these optional dependencies?)

related: wxWidgets/Phoenix#1932

Hi RobinD42 , can you maybe clarify the dependency situation with numpy and pillow in wxPython?
Is there anything significant to pay attention to across versions?
Is there a list of other more optional packages somewhere? (see above post)

Specifically I would appreciate clarification on

Apparently wxPython has numpy and pillow them as runtime dependencies because they're needed relatively often, but they're actually kind of optional?
Additionally there are "other packages" that are used less commonly, which are also runtime dependencies, but they aren't listed in the requirements because they're needed less often?

in relation to https://discuss.wxpython.org/t/does-wxpython-4-0-6-must-also-install-numpy/32969/2 .

wxWidgets/Phoenix#1932 is already asking to make these dependencies optional, and they seem to have done a little work looking at what's needed where.

- and if I have your attention anyway, would you consider packaging wxPython with https://python-poetry.org/ ? I hear good things about it. (I could open an issue on the Phoenix issue tracker I guess). (It's not obvious whether the specific optional dependency handling is any better though at the time of https://stackoverflow.com/questions/60971502/python-poetry-how-to-install-optional-dependencies )

From a practical point of view, adding dependencies seems to be easier than removing them. - which contravariantly, means that the requirement should be removed from upstream.

… and pillow to `propagatedBuildInputs` as required by upstream's dependency settings
@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jun 22, 2021
@SuperSandro2000
Copy link
Member

without numpy, pillow: /nix/store/awj8fyja99dpbc32hhifcxx7n9xqqpps-python3.8-wxPython-4.1.0 646.1M
with: /nix/store/2bbhhl3jzqj30wmiyx001lk9j9hh9wwj-python3.8-wxPython-4.1.0 834.9M

@deliciouslytyped
Copy link
Contributor Author

deliciouslytyped commented Jun 22, 2021

It's also a bit of an open question

  • how often anyone actually needs those for wxPython
  • why this doesn't appear to have been an issue for anyone else

@ofborg ofborg bot requested a review from tfmoraes June 22, 2021 08:06
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Jun 22, 2021
@deliciouslytyped
Copy link
Contributor Author

Searching nixpkgs for mentions of wxPython_4 yields:

$ grep -ir wxpython_4
development/python-modules/humblewx/default.nix:, wxPython_4_0
development/python-modules/humblewx/default.nix:  # timeline is not compatible with wxPython_4_1. reported upstream
development/python-modules/humblewx/default.nix:  propagatedBuildInputs = [ wxPython_4_0 ];
top-level/python-packages.nix:  wxPython_4_0 = callPackage ../development/python-modules/wxPython/4.0.nix {
top-level/python-packages.nix:  wxPython_4_1 = callPackage ../development/python-modules/wxPython/4.1.nix {
applications/office/timeline/default.nix:    wxPython_4_0 # not compatible with wxPython_4_1. reported upstream https://github.com/wxWidgets/Phoenix/issues/1956
applications/science/electronics/kicad/default.nix:    then python.pkgs.wxPython_4_0
applications/science/electronics/kicad/default.nix:    else python.pkgs.wxPython_4_1;
applications/science/electronics/kicad/default.nix:    # and wxPython_4_0 does not account for this
applications/science/electronics/kicad/default.nix:    # seems like more trouble than fixing wxPython_4_0 would be
applications/science/robotics/mavproxy/default.nix:, setuptools, wxPython_4_0, billiard, gnureadline }:
applications/science/robotics/mavproxy/default.nix:    wxPython_4_0
applications/radio/quisk/default.nix:, fftw, alsa-lib, pulseaudio, wxPython_4_0 }:
applications/radio/quisk/default.nix:  propagatedBuildInputs = [ wxPython_4_0 ];
applications/misc/playonlinux/default.nix:    wxPython_4_1
applications/misc/printrun/default.nix:    appdirs cython dbus-python numpy six wxPython_4_0 psutil pyglet pyopengl pyserial

Most of these don't have both numpy and pillow, what gives? (perhaps pulled in already by other deps?)

@SuperSandro2000

This comment has been minimized.

@deliciouslytyped
Copy link
Contributor Author

deliciouslytyped commented Jun 22, 2021

We can remove numpy from:

* https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/science/robotics/mavproxy/default.nix#L20

* https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/misc/printrun/default.nix#L17

Why?

Printrun for example has an explicit dependency on numpy: https://github.com/kliment/Printrun/blob/master/requirements.txt

@r-rmcgibbo
Copy link

r-rmcgibbo commented Jun 22, 2021

Result of nixpkgs-review pr 127768 at 642fadd run on aarch64-linux 1

4 packages skipped due to time constraints:
  • kicad-small
  • kicad-unstable-small
  • python38Packages.kicad
  • python39Packages.kicad
10 packages built successfully:
  • mavproxy
  • printrun
  • python38Packages.humblewx
  • python38Packages.wxPython_4_0
  • python38Packages.wxPython_4_1
  • python39Packages.humblewx
  • python39Packages.wxPython_4_0
  • python39Packages.wxPython_4_1
  • quisk
  • timeline
3 suggestions:
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/wxPython/4.1.nix:73:0:

       |
    73 |     description = "Cross platform GUI toolkit for Python, Phoenix version";
       | ^
    
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/wxPython/4.0.nix:74:0:

       |
    74 |     description = "Cross platform GUI toolkit for Python, Phoenix version";
       | ^
    
  • warning: maintainers-missing

    Package does not have a maintainer. Consider adding yourself?

    Near pkgs/development/python-modules/wxPython/4.0.nix:73:3:

       |
    73 |   meta = {
       |   ^
    

Result of nixpkgs-review pr 127768 at 642fadd run on x86_64-linux 1

4 packages skipped due to time constraints:
  • kicad-small
  • kicad-unstable-small
  • python38Packages.kicad
  • python39Packages.kicad
11 packages built successfully:
  • mavproxy
  • playonlinux
  • printrun
  • python38Packages.humblewx
  • python38Packages.wxPython_4_0
  • python38Packages.wxPython_4_1
  • python39Packages.humblewx
  • python39Packages.wxPython_4_0
  • python39Packages.wxPython_4_1
  • quisk
  • timeline
3 suggestions:
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/wxPython/4.0.nix:74:0:

       |
    74 |     description = "Cross platform GUI toolkit for Python, Phoenix version";
       | ^
    
  • warning: maintainers-missing

    Package does not have a maintainer. Consider adding yourself?

    Near pkgs/development/python-modules/wxPython/4.0.nix:73:3:

       |
    73 |   meta = {
       |   ^
    
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/wxPython/4.1.nix:73:0:

       |
    73 |     description = "Cross platform GUI toolkit for Python, Phoenix version";
       | ^
    

@SuperSandro2000
Copy link
Member

Why?

Oh, I think you are right. They are not propagating like this.

@SuperSandro2000
Copy link
Member

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).

Result of nixpkgs-review pr 127768 run on x86_64-linux 1

15 packages built:
  • kicad-small
  • kicad-unstable-small
  • mavproxy
  • playonlinux
  • printrun
  • python38Packages.humblewx
  • python38Packages.kicad
  • python38Packages.wxPython_4_0
  • python38Packages.wxPython_4_1
  • python39Packages.humblewx
  • python39Packages.kicad
  • python39Packages.wxPython_4_0
  • python39Packages.wxPython_4_1
  • quisk
  • timeline

@SuperSandro2000 SuperSandro2000 merged commit 52a64db into NixOS:master Jun 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants