Skip to content

pythonPackages.matplotlib: fix headless detection - #124439

Merged
veprbl merged 3 commits into
NixOS:masterfrom
rnhmjoj:mpl
May 26, 2021
Merged

veprbl merged 3 commits into
NixOS:masterfrom
rnhmjoj:mpl

Conversation

@rnhmjoj

@rnhmjoj rnhmjoj commented May 25, 2021

Copy link
Copy Markdown
Contributor
Motivation for this change

The default backend is chosen based on the content of the $DISPLAY
variable and a successfull call to libX11, loaded via dlopen().
The test fails because dlopen looks in /usr/lib and /lib, so matplotlib
falls back to a headless backend.

To reproduce try running:

$ nix-shell -I nixpkgs=$PWD -p \
  'python3.withPackages (p: [ p.matplotlib ])' --run python
>>> import matplotlib.pyplot as plt
>>> assert plt.get_backend() == "TkAgg"
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via the above method
  • Tested compilation of all pkgs that depend on this change
  • Tested execution of all binary files (none)
  • Added a release notes entry if the change is major or breaking (it's not)
  • Fits CONTRIBUTING.md.

@rnhmjoj
rnhmjoj requested review from FRidh and jonringer as code owners May 25, 2021 23:38
@github-actions github-actions Bot added the 6.topic: python Python is a high-level, general-purpose programming language. label May 25, 2021
@ofborg
ofborg Bot requested review from lovek323 and veprbl May 25, 2021 23:48
@ofborg ofborg Bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. labels May 25, 2021
@r-rmcgibbo

r-rmcgibbo commented May 26, 2021 •

Copy link
Copy Markdown

Result of nixpkgs-review pr 124439 at fa2fa9db run on x86_64-linux 1

57 packages marked as broken and skipped:
  • cassandra_2_1
  • cassandra_2_2
  • hadoop
  • hadoop_2_7
  • hadoop_2_8
  • hadoop_2_9
  • hydrus
  • impressive
  • linuxPackages-libre.virtualbox
  • linuxPackages_hardkernel_4_14.virtualbox
  • ...
3 packages failed to build:
363 packages skipped due to time constraints:
  • DisnixWebService
  • R
  • abcl
  • adapta-gtk-theme
  • adoptopenjdk-icedtea-web
  • alloy (alloy5)
  • alloy4
  • antlr
  • antlr3 (antlr3_5)
  • antlr3_4
  • ...
164 packages built successfully:
  • anki
  • ankisyncd
  • asciidoc-full-with-plugins
  • corrscope
  • datasette (python38Packages.datasette)
  • deeptools
  • evillimiter
  • fdroidserver
  • flatcam
  • fop
  • graph-cli
  • igraph
  • image-analyzer
  • jre8
  • liberation-sans-narrow
  • liberation_ttf (liberation_ttf_v2)
  • mavproxy
  • mnemosyne
  • pitivi
  • psrecord
  • pyspread
  • python38Packages.Pweave
  • python38Packages.androguard
  • python38Packages.ase
  • python38Packages.basemap
  • python38Packages.bayespy
  • python38Packages.booleanoperations
  • python38Packages.castepxbin
  • python38Packages.colorcet
  • python38Packages.dask-image
  • python38Packages.deap
  • python38Packages.defcon
  • python38Packages.dufte
  • python38Packages.fipy
  • python38Packages.fontmath
  • python38Packages.fontparts
  • python38Packages.fontpens
  • python38Packages.fonttools
  • python38Packages.glymur
  • python38Packages.holoviews
  • python38Packages.hvplot
  • python38Packages.image-match
  • python38Packages.imagecorruptions
  • python38Packages.ipydatawidgets
  • python38Packages.ipympl
  • python38Packages.librosa
  • python38Packages.lightning
  • python38Packages.markdown-it-py
  • python38Packages.matplotlib
  • python38Packages.mdit-py-plugins
  • python38Packages.memory_profiler
  • python38Packages.mplfinance
  • python38Packages.mutatormath
  • python38Packages.nbsmoke
  • python38Packages.nbval
  • python38Packages.osmpythontools
  • python38Packages.persim
  • python38Packages.phik
  • python38Packages.pims
  • python38Packages.pint
  • python38Packages.psautohint
  • python38Packages.pyalgotrade
  • python38Packages.pycm
  • python38Packages.pycocotools
  • python38Packages.pymatgen
  • python38Packages.pyregion
  • python38Packages.pyspice
  • python38Packages.pytest-astropy
  • python38Packages.pytest-astropy-header
  • python38Packages.pytest-mpl
  • python38Packages.pytest-regressions
  • python38Packages.python-csxcad
  • python38Packages.python-openems
  • python38Packages.pytomlpp
  • python38Packages.qcelemental
  • python38Packages.regional
  • python38Packages.ripser
  • python38Packages.scapy
  • python38Packages.scikit-bio
  • python38Packages.scikitimage
  • python38Packages.seaborn
  • python38Packages.showit
  • python38Packages.skidl
  • python38Packages.slicedimage
  • python38Packages.spyder-kernels_0_5
  • python38Packages.spyder_3
  • python38Packages.stytra
  • python38Packages.tadasets
  • python38Packages.ufoprocessor
  • python38Packages.vidstab
  • python38Packages.wordcloud
  • python39Packages.Pweave
  • python39Packages.androguard
  • python39Packages.ase
  • python39Packages.bayespy
  • python39Packages.booleanoperations
  • python39Packages.castepxbin
  • python39Packages.colorcet
  • python39Packages.dask-image
  • python39Packages.datasette
  • python39Packages.deap
  • python39Packages.defcon
  • python39Packages.dufte
  • python39Packages.fontmath
  • python39Packages.fontparts
  • python39Packages.fontpens
  • python39Packages.fonttools
  • python39Packages.glymur
  • python39Packages.holoviews
  • python39Packages.hvplot
  • python39Packages.image-match
  • python39Packages.imagecorruptions
  • python39Packages.ipydatawidgets
  • python39Packages.ipympl
  • python39Packages.kmapper
  • python39Packages.lightning
  • python39Packages.markdown-it-py
  • python39Packages.matplotlib
  • python39Packages.mdit-py-plugins
  • python39Packages.memory_profiler
  • python39Packages.mplfinance
  • python39Packages.mutatormath
  • python39Packages.nbsmoke
  • python39Packages.nbval
  • python39Packages.osmpythontools
  • python39Packages.perfplot
  • python39Packages.persim
  • python39Packages.pims
  • python39Packages.pint
  • python39Packages.psautohint
  • python39Packages.pyalgotrade
  • python39Packages.pycm
  • python39Packages.pycocotools
  • python39Packages.pymatgen
  • python39Packages.pyregion
  • python39Packages.pyspice
  • python39Packages.pytest-astropy
  • python39Packages.pytest-astropy-header
  • python39Packages.pytest-mpl
  • python39Packages.pytest-regressions
  • python39Packages.python-igraph
  • python39Packages.pytomlpp
  • python39Packages.qcelemental
  • python39Packages.regional
  • python39Packages.ripser
  • python39Packages.scapy
  • python39Packages.scikit-bio
  • python39Packages.scikit-fuzzy
  • python39Packages.scikitimage
  • python39Packages.seaborn
  • python39Packages.showit
  • python39Packages.skidl
  • python39Packages.slicedimage
  • python39Packages.spyder-kernels_0_5
  • python39Packages.spyder_3
  • python39Packages.tadasets
  • python39Packages.ufoprocessor
  • python39Packages.vidstab
  • python39Packages.wordcloud
  • python39Packages.xgboost
  • scfbuild
  • tartube
  • visidata
  • xits-math
6 suggestions:
  • warning: unused-argument

    Unused argument: nose.
    Near pkgs/development/python-modules/matplotlib/default.nix:2:28:

      |
    2 | , which, cycler, dateutil, nose, numpy, pyparsing, sphinx, tornado, kiwisolver
      |                            ^
    
  • warning: license-missing

    Package is missing a license.

    Near pkgs/development/python-modules/matplotlib/default.nix:81:3:

       |
    81 |   meta = with lib; {
       |   ^
    
  • warning: unused-argument

    Unused argument: pythonOlder.
    Near pkgs/development/python-modules/matplotlib/default.nix:13:3:

       |
    13 | , pythonOlder
       |   ^
    
  • warning: unused-argument

    Unused argument: python.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:38:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                      ^
    
  • warning: unused-argument

    Unused argument: backports_functools_lru_cache.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:83:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                                                                   ^
    
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/matplotlib/default.nix:82:0:

       |
    82 |     description = "Python plotting library, making publication quality plots";
       | ^
    

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.


Result of nixpkgs-review pr 124439 at fa2fa9db run on aarch64-linux 1

83 packages marked as broken and skipped:
  • belle-sip
  • cassandra_2_1
  • cassandra_2_2
  • cvc4
  • deeptools
  • herwig
  • hydrus
  • impressive
  • liblinphone
  • lime
  • ...
4 packages failed to build:
278 packages skipped due to time constraints:
  • DisnixWebService
  • R
  • abcl
  • adapta-gtk-theme
  • adoptopenjdk-icedtea-web
  • alloy (alloy5)
  • alloy4
  • antlr
  • antlr3 (antlr3_5)
  • antlr3_4
  • ...
157 packages built successfully:
  • anki
  • ankisyncd
  • asciidoc-full-with-plugins
  • corrscope
  • datasette (python38Packages.datasette)
  • evillimiter
  • fdroidserver
  • flatcam
  • fop
  • graph-cli
  • igraph
  • image-analyzer
  • jre8 (jre8_headless)
  • mavproxy
  • mnemosyne
  • pitivi
  • psrecord
  • pyspread
  • python38Packages.Pweave
  • python38Packages.androguard
  • python38Packages.ase
  • python38Packages.basemap
  • python38Packages.bayespy
  • python38Packages.booleanoperations
  • python38Packages.castepxbin
  • python38Packages.colorcet
  • python38Packages.deap
  • python38Packages.defcon
  • python38Packages.dufte
  • python38Packages.fontmath
  • python38Packages.fontparts
  • python38Packages.fontpens
  • python38Packages.fonttools
  • python38Packages.glymur
  • python38Packages.gpy
  • python38Packages.gpyopt
  • python38Packages.holoviews
  • python38Packages.hvplot
  • python38Packages.image-match
  • python38Packages.imagecorruptions
  • python38Packages.ipydatawidgets
  • python38Packages.ipympl
  • python38Packages.kmapper
  • python38Packages.librosa
  • python38Packages.lightning
  • python38Packages.markdown-it-py
  • python38Packages.matplotlib
  • python38Packages.mdit-py-plugins
  • python38Packages.memory_profiler
  • python38Packages.mplfinance
  • python38Packages.mutatormath
  • python38Packages.nbsmoke
  • python38Packages.nbval
  • python38Packages.osmpythontools
  • python38Packages.persim
  • python38Packages.phik
  • python38Packages.pims
  • python38Packages.pint
  • python38Packages.psautohint
  • python38Packages.pyalgotrade
  • python38Packages.pycm
  • python38Packages.pycocotools
  • python38Packages.pymatgen
  • python38Packages.pyregion
  • python38Packages.pyspice
  • python38Packages.pytest-astropy
  • python38Packages.pytest-astropy-header
  • python38Packages.pytest-mpl
  • python38Packages.pytest-regressions
  • python38Packages.python-igraph
  • python38Packages.pytomlpp
  • python38Packages.qcelemental
  • python38Packages.regional
  • python38Packages.ripser
  • python38Packages.scapy
  • python38Packages.scikit-tda
  • python38Packages.scikitimage
  • python38Packages.seaborn
  • python38Packages.showit
  • python38Packages.skidl
  • python38Packages.slicedimage
  • spyder (python38Packages.spyder)
  • python38Packages.spyder-kernels_0_5
  • python38Packages.spyder_3
  • python38Packages.stytra
  • python38Packages.tadasets
  • python38Packages.ufoprocessor
  • python38Packages.vidstab
  • python38Packages.wordcloud
  • python39Packages.Pweave
  • python39Packages.androguard
  • python39Packages.ase
  • python39Packages.bayespy
  • python39Packages.booleanoperations
  • python39Packages.castepxbin
  • python39Packages.colorcet
  • python39Packages.deap
  • python39Packages.defcon
  • python39Packages.dufte
  • python39Packages.fontmath
  • python39Packages.fontparts
  • python39Packages.fontpens
  • python39Packages.fonttools
  • python39Packages.glymur
  • python39Packages.holoviews
  • python39Packages.hvplot
  • python39Packages.image-match
  • python39Packages.imagecorruptions
  • python39Packages.ipydatawidgets
  • python39Packages.ipympl
  • python39Packages.kmapper
  • python39Packages.lightning
  • python39Packages.livelossplot
  • python39Packages.markdown-it-py
  • python39Packages.matplotlib
  • python39Packages.mdit-py-plugins
  • python39Packages.memory_profiler
  • python39Packages.mplfinance
  • python39Packages.mutatormath
  • python39Packages.nbsmoke
  • python39Packages.nbval
  • python39Packages.osmpythontools
  • python39Packages.perfplot
  • python39Packages.persim
  • python39Packages.pims
  • python39Packages.pint
  • python39Packages.psautohint
  • python39Packages.pycm
  • python39Packages.pycocotools
  • python39Packages.pymatgen
  • python39Packages.pyregion
  • python39Packages.pyspice
  • python39Packages.pytest-astropy
  • python39Packages.pytest-astropy-header
  • python39Packages.pytest-mpl
  • python39Packages.pytest-regressions
  • python39Packages.python-igraph
  • python39Packages.pytomlpp
  • python39Packages.qcelemental
  • python39Packages.regional
  • python39Packages.ripser
  • python39Packages.scapy
  • python39Packages.scikit-fuzzy
  • python39Packages.scikitimage
  • python39Packages.seaborn
  • python39Packages.showit
  • python39Packages.skidl
  • python39Packages.slicedimage
  • python39Packages.spyder-kernels_0_5
  • python39Packages.tadasets
  • python39Packages.ufoprocessor
  • python39Packages.vidstab
  • python39Packages.wordcloud
  • scfbuild
  • tartube
  • visidata
  • xits-math
6 suggestions:
  • warning: unused-argument

    Unused argument: nose.
    Near pkgs/development/python-modules/matplotlib/default.nix:2:28:

      |
    2 | , which, cycler, dateutil, nose, numpy, pyparsing, sphinx, tornado, kiwisolver
      |                            ^
    
  • warning: unused-argument

    Unused argument: pythonOlder.
    Near pkgs/development/python-modules/matplotlib/default.nix:13:3:

       |
    13 | , pythonOlder
       |   ^
    
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/matplotlib/default.nix:82:0:

       |
    82 |     description = "Python plotting library, making publication quality plots";
       | ^
    
  • warning: unused-argument

    Unused argument: backports_functools_lru_cache.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:83:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                                                                   ^
    
  • warning: license-missing

    Package is missing a license.

    Near pkgs/development/python-modules/matplotlib/default.nix:81:3:

       |
    81 |   meta = with lib; {
       |   ^
    
  • warning: unused-argument

    Unused argument: python.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:38:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                      ^
    

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.

@SuperSandro2000

Copy link
Copy Markdown
Member

We probably should rebase this against staging.

@CMCDragonkai

Copy link
Copy Markdown
Member

This just bit me. The release 21.05 has this problem. It should be backported to release 21.05.

@veprbl veprbl added the 9.needs: port to stable A PR needs a backport to the stable release. label May 26, 2021
The default backend is chosen based on the content of the $DISPLAY
variable *and* a successfull call to libX11, loaded via dlopen().
The test fails because dlopen looks in /usr/lib and /lib, so matplotlib
falls back to a headless backend.

To reproduce try running:

    $ nix-shell -I nixpkgs=$PWD -p \
      'python3.withPackages (p: [ p.matplotlib ])' --run python
    >>> import matplotlib.pyplot as plt
    >>> assert plt.get_backend() == "TkAgg"
@ofborg
ofborg Bot requested a review from veprbl May 26, 2021 06:05

@veprbl veprbl left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ofborg
ofborg Bot requested a review from veprbl May 26, 2021 06:41
@veprbl

veprbl commented May 26, 2021

Copy link
Copy Markdown
Member

@GrahamcOfBorg build python3Packages.matplotlib

@veprbl
veprbl merged commit 66e7fcf into NixOS:master May 26, 2021
@rnhmjoj rnhmjoj added the 8.has: port to stable This PR already has a backport to the stable release. label May 26, 2021
@CMCDragonkai

CMCDragonkai commented May 27, 2021 •

Copy link
Copy Markdown
Member

How to port this to 21.05? #122042

@veprbl veprbl removed the 9.needs: port to stable A PR needs a backport to the stable release. label May 27, 2021
@veprbl

veprbl commented May 27, 2021

Copy link
Copy Markdown
Member

How to port this to 21.05? #122042

Already done in #124488

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: port to stable This PR already has a backport to the stable release. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants