Skip to content

Commit e0d3e4c

Browse files
authored
Move Matplotlib backend mapping to Matplotlib (#14371)
This is WIP to move IPython's backend mapping into Matplotlib and extends it to allow backends to register themselves via [entry points](https://setuptools.pypa.io/en/latest/userguide/entry_point.html#entry-points-for-plugins). It is a coordinated effort across Matplotlib, IPython, matplotlib-inline and ipympl. Closes #14311. Most of the work is in Matplotlib (matplotlib/matplotlib#27948) but I will repeat the relevant points here. When using a Matplotlib magic command of the form `%matplotlib something` the identification of the Matplotlib backend and GUI loop are now performed in Matplotlib not IPython. This supports: 1. Matplotlib backends that IPython already supports such as `qtagg` and `tkagg`. 2. Other built-in Matplotlib backends such as `qtcairo`. 3. Backends that use `module://something` syntax such as `module://mplcairo.qt`. 4. Backends that self-register using entry points, currently `inline` and `widget`/`ipympl`. Implementation details: 1. The magic command is now explicitly `%matplotlib gui_or_backend` rather than `%matplotlib gui`. This is already effectively the case as e.g. `%matplotlib ipympl` is really a backend not GUI name. Within Matplotlib the `gui_or_backend` is checked first to see if it is a GUI name and then falls back to checking if it is a backend name. 2. If you select the `inline` backend the corresponding GUI is now `None` not `inline`. All backends which do not have a GUI loop return `None`, otherwise we have to keep explicit checks within IPython for particular backends. 3. `backends` and `backend2gui` are now deprecated but are still exposed, with a deprecation warning, if required. If using Matplotlib, ipympl, etc releases that include the corresponding changes to this PR then they are not needed as Matplotlib deals with it all. But for backward compatibility they must still be available for a while along with the remainder of the existing backend-handling code. 4. I haven't yet updated the documentation but we need to keep information about valid GUI frameworks and I propose that we should remove all lists of valid Matplotlib backends, replacing them with instructions on how to obtain the current list (pointing to the Matplotlib docs and using `%matplotlib --list`). If we keep any lists then they will inevitably become out of date. This extends to the `backend_keys` in IPython/core/shellapp.py. Because the four related projects are loosely coupled without direct dependencies on each other (except for `ipython` and `matplotlib-inline`), backward compatibility requires all possible combinations of projects before and after the new functionality (I will call these "old" and "new" from now on) to continue to work. I have tested these all locally, and the CI of this PR will test new IPython against old Matplotlib for example, but I need to add one or more new temporary CI runs to test new IPython against new Matplotlib etc. The identification of new versus old depends on version checks on the other libraries, so here is a table that I will update showing the current status of progress in the 4 projects: | Project | Relevant PRs | Possible release version | | --- | --- | --- | | matplotlib-inline | ipython/matplotlib-inline#34, ipython/matplotlib-inline#35 | 0.1.7 | | ipympl | matplotlib/ipympl#549 | 0.9.4 | | Matplotlib | matplotlib/matplotlib#27948 | 3.9.1 | | IPython | #14371 (this) | 8.24.0 | The two widget projects can be released soon, once we are happy with the entry point approach. The other two projects' PRs will have to be synchronised as each includes version checks on each other. To do - [ ] Add CI runs against the new PR branches of the other projects. - [ ] Add comments for conditions required for backward-compatibility code blocks to be removed. - [ ] Update documentation, including removal of lists of valid backends. - [ ] Update version checks before merging.
2 parents fe52b20 + b1d9d89 commit e0d3e4c

9 files changed

Lines changed: 291 additions & 63 deletions

File tree

‎.github/workflows/test.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,11 @@ jobs:
4949
- os: macos-latest
5050
python-version: "pypy-3.10"
5151
deps: test
52+
# Temporary CI run to use entry point compatible code in matplotlib-inline.
53+
- os: ubuntu-latest
54+
python-version: "3.12"
55+
deps: test_extra
56+
want-latest-entry-point-code: true
5257

5358
steps:
5459
- uses: actions/checkout@v3
@@ -82,6 +87,16 @@ jobs:
8287
- name: Check manifest
8388
if: runner.os != 'Windows' # setup.py does not support sdist on Windows
8489
run: check-manifest
90+
91+
- name: Install entry point compatible code (TEMPORARY)
92+
if: matrix.want-latest-entry-point-code
93+
run: |
94+
python -m pip list
95+
# Not installing matplotlib's entry point code as building matplotlib from source is complex.
96+
# Rely upon matplotlib to test all the latest entry point branches together.
97+
python -m pip install --upgrade git+https://github.com/ipython/matplotlib-inline.git@main
98+
python -m pip list
99+
85100
- name: pytest
86101
env:
87102
COLUMNS: 120

‎IPython/core/display_functions.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ def display(
111111
display_id=None,
112112
raw=False,
113113
clear=False,
114-
**kwargs
114+
**kwargs,
115115
):
116116
"""Display a Python object in all frontends.
117117

‎IPython/core/interactiveshell.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3657,7 +3657,7 @@ def enable_matplotlib(self, gui=None):
36573657
from IPython.core import pylabtools as pt
36583658
gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
36593659

3660-
if gui != 'inline':
3660+
if gui != None:
36613661
# If we have our first gui selection, store it
36623662
if self.pylab_gui_select is None:
36633663
self.pylab_gui_select = gui

‎IPython/core/magics/pylab.py‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,19 +18,19 @@
1818
from IPython.core.magic import Magics, magics_class, line_magic
1919
from IPython.testing.skipdoctest import skip_doctest
2020
from warnings import warn
21-
from IPython.core.pylabtools import backends
2221

2322
#-----------------------------------------------------------------------------
2423
# Magic implementation classes
2524
#-----------------------------------------------------------------------------
2625

2726
magic_gui_arg = magic_arguments.argument(
28-
'gui', nargs='?',
29-
help="""Name of the matplotlib backend to use %s.
27+
"gui",
28+
nargs="?",
29+
help="""Name of the matplotlib backend to use such as 'qt' or 'widget'.
3030
If given, the corresponding matplotlib backend is used,
3131
otherwise it will be matplotlib's default
3232
(which you can set in your matplotlib config file).
33-
""" % str(tuple(sorted(backends.keys())))
33+
""",
3434
)
3535

3636

@@ -93,8 +93,12 @@ def matplotlib(self, line=''):
9393
"""
9494
args = magic_arguments.parse_argstring(self.matplotlib, line)
9595
if args.list:
96-
backends_list = list(backends.keys())
97-
print("Available matplotlib backends: %s" % backends_list)
96+
from IPython.core.pylabtools import _list_matplotlib_backends_and_gui_loops
97+
98+
print(
99+
"Available matplotlib backends: %s"
100+
% _list_matplotlib_backends_and_gui_loops()
101+
)
98102
else:
99103
gui, backend = self.shell.enable_matplotlib(args.gui.lower() if isinstance(args.gui, str) else args.gui)
100104
self._show_matplotlib_backend(args.gui, backend)

‎IPython/core/pylabtools.py‎

Lines changed: 106 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,12 @@
1212
from IPython.core.display import _pngxy
1313
from IPython.utils.decorators import flag_calls
1414

15-
# If user specifies a GUI, that dictates the backend, otherwise we read the
16-
# user's mpl default from the mpl rc structure
17-
backends = {
15+
16+
# Matplotlib backend resolution functionality moved from IPython to Matplotlib
17+
# in IPython 8.24 and Matplotlib 3.9.1. Need to keep `backends` and `backend2gui`
18+
# here for earlier Matplotlib and for external backend libraries such as
19+
# mplcairo that might rely upon it.
20+
_deprecated_backends = {
1821
"tk": "TkAgg",
1922
"gtk": "GTKAgg",
2023
"gtk3": "GTK3Agg",
@@ -41,29 +44,44 @@
4144
# GUI support to activate based on the desired matplotlib backend. For the
4245
# most part it's just a reverse of the above dict, but we also need to add a
4346
# few others that map to the same GUI manually:
44-
backend2gui = dict(zip(backends.values(), backends.keys()))
47+
_deprecated_backend2gui = dict(
48+
zip(_deprecated_backends.values(), _deprecated_backends.keys())
49+
)
4550
# In the reverse mapping, there are a few extra valid matplotlib backends that
4651
# map to the same GUI support
47-
backend2gui["GTK"] = backend2gui["GTKCairo"] = "gtk"
48-
backend2gui["GTK3Cairo"] = "gtk3"
49-
backend2gui["GTK4Cairo"] = "gtk4"
50-
backend2gui["WX"] = "wx"
51-
backend2gui["CocoaAgg"] = "osx"
52+
_deprecated_backend2gui["GTK"] = _deprecated_backend2gui["GTKCairo"] = "gtk"
53+
_deprecated_backend2gui["GTK3Cairo"] = "gtk3"
54+
_deprecated_backend2gui["GTK4Cairo"] = "gtk4"
55+
_deprecated_backend2gui["WX"] = "wx"
56+
_deprecated_backend2gui["CocoaAgg"] = "osx"
5257
# There needs to be a hysteresis here as the new QtAgg Matplotlib backend
5358
# supports either Qt5 or Qt6 and the IPython qt event loop support Qt4, Qt5,
5459
# and Qt6.
55-
backend2gui["QtAgg"] = "qt"
56-
backend2gui["Qt4Agg"] = "qt4"
57-
backend2gui["Qt5Agg"] = "qt5"
60+
_deprecated_backend2gui["QtAgg"] = "qt"
61+
_deprecated_backend2gui["Qt4Agg"] = "qt4"
62+
_deprecated_backend2gui["Qt5Agg"] = "qt5"
5863

5964
# And some backends that don't need GUI integration
60-
del backend2gui["nbAgg"]
61-
del backend2gui["agg"]
62-
del backend2gui["svg"]
63-
del backend2gui["pdf"]
64-
del backend2gui["ps"]
65-
del backend2gui["module://matplotlib_inline.backend_inline"]
66-
del backend2gui["module://ipympl.backend_nbagg"]
65+
del _deprecated_backend2gui["nbAgg"]
66+
del _deprecated_backend2gui["agg"]
67+
del _deprecated_backend2gui["svg"]
68+
del _deprecated_backend2gui["pdf"]
69+
del _deprecated_backend2gui["ps"]
70+
del _deprecated_backend2gui["module://matplotlib_inline.backend_inline"]
71+
del _deprecated_backend2gui["module://ipympl.backend_nbagg"]
72+
73+
74+
# Deprecated attributes backends and backend2gui mostly following PEP 562.
75+
def __getattr__(name):
76+
if name in ("backends", "backend2gui"):
77+
warnings.warn(
78+
f"{name} is deprecated since IPython 8.24, backends are managed "
79+
"in matplotlib and can be externally registered.",
80+
DeprecationWarning,
81+
)
82+
return globals()[f"_deprecated_{name}"]
83+
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
84+
6785

6886
#-----------------------------------------------------------------------------
6987
# Matplotlib utilities
@@ -267,7 +285,7 @@ def select_figure_formats(shell, formats, **kwargs):
267285

268286
[ f.pop(Figure, None) for f in shell.display_formatter.formatters.values() ]
269287
mplbackend = matplotlib.get_backend().lower()
270-
if mplbackend == 'nbagg' or mplbackend == 'module://ipympl.backend_nbagg':
288+
if mplbackend in ("nbagg", "ipympl", "widget", "module://ipympl.backend_nbagg"):
271289
formatter = shell.display_formatter.ipython_display_formatter
272290
formatter.for_type(Figure, _reshow_nbagg_figure)
273291

@@ -319,7 +337,23 @@ def find_gui_and_backend(gui=None, gui_select=None):
319337

320338
import matplotlib
321339

322-
has_unified_qt_backend = getattr(matplotlib, "__version_info__", (0, 0)) >= (3, 5)
340+
if _matplotlib_manages_backends():
341+
backend_registry = matplotlib.backends.registry.backend_registry
342+
343+
# gui argument may be a gui event loop or may be a backend name.
344+
if gui in ("auto", None):
345+
backend = matplotlib.rcParamsOrig["backend"]
346+
backend, gui = backend_registry.resolve_backend(backend)
347+
else:
348+
backend, gui = backend_registry.resolve_gui_or_backend(gui)
349+
350+
return gui, backend
351+
352+
# Fallback to previous behaviour (Matplotlib < 3.9)
353+
mpl_version_info = getattr(matplotlib, "__version_info__", (0, 0))
354+
has_unified_qt_backend = mpl_version_info >= (3, 5)
355+
356+
from IPython.core.pylabtools import backends
323357

324358
backends_ = dict(backends)
325359
if not has_unified_qt_backend:
@@ -338,6 +372,7 @@ def find_gui_and_backend(gui=None, gui_select=None):
338372
backend = matplotlib.rcParamsOrig['backend']
339373
# In this case, we need to find what the appropriate gui selection call
340374
# should be for IPython, so we can activate inputhook accordingly
375+
from IPython.core.pylabtools import backend2gui
341376
gui = backend2gui.get(backend, None)
342377

343378
# If we have already had a gui active, we need it and inline are the
@@ -346,6 +381,11 @@ def find_gui_and_backend(gui=None, gui_select=None):
346381
gui = gui_select
347382
backend = backends_[gui]
348383

384+
# Matplotlib before _matplotlib_manages_backends() can return "inline" for
385+
# no gui event loop rather than the None that IPython >= 8.24.0 expects.
386+
if gui == "inline":
387+
gui = None
388+
349389
return gui, backend
350390

351391

@@ -431,3 +471,48 @@ def configure_inline_support(shell, backend):
431471
)
432472

433473
configure_inline_support_orig(shell, backend)
474+
475+
476+
# Determine if Matplotlib manages backends only if needed, and cache result.
477+
# Do not read this directly, instead use _matplotlib_manages_backends().
478+
_matplotlib_manages_backends_value: bool | None = None
479+
480+
481+
def _matplotlib_manages_backends() -> bool:
482+
"""Return True if Matplotlib manages backends, False otherwise.
483+
484+
If it returns True, the caller can be sure that
485+
matplotlib.backends.registry.backend_registry is available along with
486+
member functions resolve_gui_or_backend, resolve_backend, list_all, and
487+
list_gui_frameworks.
488+
"""
489+
global _matplotlib_manages_backends_value
490+
if _matplotlib_manages_backends_value is None:
491+
try:
492+
from matplotlib.backends.registry import backend_registry
493+
494+
_matplotlib_manages_backends_value = hasattr(
495+
backend_registry, "resolve_gui_or_backend"
496+
)
497+
except ImportError:
498+
_matplotlib_manages_backends_value = False
499+
500+
return _matplotlib_manages_backends_value
501+
502+
503+
def _list_matplotlib_backends_and_gui_loops() -> list[str]:
504+
"""Return list of all Matplotlib backends and GUI event loops.
505+
506+
This is the list returned by
507+
%matplotlib --list
508+
"""
509+
if _matplotlib_manages_backends():
510+
from matplotlib.backends.registry import backend_registry
511+
512+
ret = backend_registry.list_all() + backend_registry.list_gui_frameworks()
513+
else:
514+
from IPython.core import pylabtools
515+
516+
ret = list(pylabtools.backends.keys())
517+
518+
return sorted(["auto"] + ret)

0 commit comments

Comments
 (0)