[Bug]: VSCode matplotlib interactive mode cannot import Qt bindings #25673

Activity

1. tacaswell commented on Apr 13, 2023

   

   tacaswell

   on Apr 13, 2023

   Member

   More actions

   Are you sure that pyside6 is installed in the environment that is being used for the terminal.
2. kwsp commented on Apr 13, 2023

   

   kwsp

   on Apr 13, 2023

   Last edited by kwsp

   ContributorAuthor

   More actions

   Yes. In the same interactive shell, I can run the following cell

   # %%
   import PySide6
   print(PySide6.__version__)  # Prints 6.5.0
3. kwsp commented on Apr 13, 2023

   

   kwsp

   on Apr 13, 2023

   ContributorAuthor

   More actions

   When I start python from the command line and try to plot things, interactive plots work. It's only in VS Code interactive mode, when I try to enable interactive plots with %matplotlib qt does this happen.
4. tacaswell commented on Apr 13, 2023

   

   tacaswell

   on Apr 13, 2023

   Member

   More actions

   If you import pyside6 before you do %matplotlib qt does it work?
5. kwsp commented on Apr 13, 2023

   

   kwsp

   on Apr 13, 2023

   Last edited by kwsp

   ContributorAuthor

   More actions

   It does not unfortunately. Just tested in a fresh shell:

   # %%
   import PySide6
   import matplotlib.pyplot as plt
   
   %matplotlib qt

   ImportError: Failed to import any of the following Qt binding modules: PyQt6, PySide6, PyQt5, PySide2
   

   I also want to add that this has been happening to me for many months, on two separate machines (Windows 10 and Windows 11).
6. tacaswell commented on Apr 13, 2023

   

   tacaswell

   on Apr 13, 2023

   Member

   More actions

   Can you try the code in

   matplotlib/lib/matplotlib/backends/qt_compat.py

   Lines 82 to 89 in bff4681

   	elif QT_API == QT_API_PYSIDE6:
   	from PySide6 import QtCore, QtGui, QtWidgets, __version__
   	import shiboken6
   	def _isdeleted(obj): return not shiboken6.isValid(obj)
   	if parse_version(__version__) >= parse_version('6.4'):
   	_to_int = operator.attrgetter('value')
   	else:
   	_to_int = int

   to verify that works as expected?

   does %gui qt work?

   Does

   import PySide6
   import matplotlib.pyplot as plt
   plt.ion()
   fig, ax = plt.subplots()
   

   work?

   What version of pyside6?

   In the shells where it works can you check

   fig = plt.gcf() # or get a Figure object however you want
   print(type(fig.canvas).mro()
   

   to make sure it really is using pyside6 in those cases.

   Sorry for asking many questions, I do not have a window system set up to reproduce this.
7. kwsp commented on Apr 13, 2023

   

   kwsp

   on Apr 13, 2023

   Last edited by kwsp

   ContributorAuthor

   More actions

   Can you try the code in

   matplotlib/lib/matplotlib/backends/qt_compat.py

   Lines 82 to 89 in bff4681

   	elif QT_API == QT_API_PYSIDE6:
   	from PySide6 import QtCore, QtGui, QtWidgets, __version__
   	import shiboken6
   	def _isdeleted(obj): return not shiboken6.isValid(obj)
   	if parse_version(__version__) >= parse_version('6.4'):
   	_to_int = operator.attrgetter('value')
   	else:
   	_to_int = int

   to verify that works as expected?

   This works if I run directly in the shell.

   does %gui qt work?

   This line runs without exception, but matplotlib would still use the Agg backend.

   Does

   import PySide6
   import matplotlib.pyplot as plt
   plt.ion()
   fig, ax = plt.subplots()
   

   work?

   This executes without exception, but the result plot is still inline.

   What version of pyside6?

   6.5.0

   In the shells where it works can you check

   fig = plt.gcf() # or get a Figure object however you want
   print(type(fig.canvas).mro()
   

   to make sure it really is using pyside6 in those cases.

   It's not using the PySide6 backend: [<class 'matplotlib.backends.backend_agg.FigureCanvasAgg'>, <class 'matplotlib.backend_bases.FigureCanvasBase'>, <class 'object'>]

   Sorry for asking many questions, I do not have a window system set up to reproduce this.

   No I understand. Thanks for your help. I'm an absolute noob with Windows but I need to develop native Windows apps for work :(
8. tacaswell commented on Apr 14, 2023

   

   tacaswell

   on Apr 14, 2023

   Member

   More actions

   Can you get a Pyside6 "hello world" app to work in the vscode terminal?

   Another thing I just noticed is that there is "conda" in your paths, but you said you installed via pip.

   Try making a fresh environment and installing everything from conda. Mixing conda and wheels can go bad in odd ways (see https://pypackaging-native.github.io).

   Can you try older versions of pyside?

   If you use

   plt.switch('qtagg')
   

   early does that make any difference?
9. kwsp commented on Apr 24, 2023

   

   kwsp

   on Apr 24, 2023

   ContributorAuthor

   More actions

   plt.switch_backend('qtagg') gave the same error: ImportError: Failed to import any of the following Qt binding modules: PyQt6, PySide6, PyQt5, PySide2

   In the same VS Code interactive shell, I can run the following hello world program with PySide6 and a Qt window pops up.

   from PySide6 import QtWidgets
   
   app = QtWidgets.QApplication()
   win = QtWidgets.QWidget()
   win.show()
   app.exec()

   This actually isn't specific to Windows and the above results are from an Ubuntu 22.04 machine.
10. kwsp commented on Apr 24, 2023

    

    kwsp

    on Apr 24, 2023

    Last edited by kwsp

    ContributorAuthor

    More actions

    The reason I'm using conda + pip:

    1. Conda is great at managing virtual environments. pip doesn't do that.
    2. My packages are all declared with a pyproject.toml file. I can then easily install my packages with pip install . and have pip manage the dependencies for me. I have yet to find a painless way to manage my own packages with conda.

    That said, in this case, I'm exclusively using pip to manage packages in this environment and only using conda as the python version/virtual environment manager, which I heard is innocuous.

    Re: mixing conda env and pip - I just took a look, even matplotlib (when using conda virtual environments) needs to use pip install -e . to install matplotlib...: https://matplotlib.org/devdocs/devel/development_setup.html#create-a-dedicated-environment
11. tacaswell commented on Apr 24, 2023

    

    tacaswell

    on Apr 24, 2023

    Member

    More actions

    I can not reproduce this (I got the interactive window by right click -> "run in interactive window" on an empty file and selected the system Python (which on my system has enough of the stack installed) for the Python that vscode is using).

    Looking at the code in qt_compat.py can you sort out exactly what is failing when we try each of the bindings?
12. kwsp commented on Apr 25, 2023

    

    kwsp

    on Apr 25, 2023

    ContributorAuthor

    More actions

    I looked into qt_compat.py, and when I execute

    import matplotlib.backends.qt_compat
    matplotlib.backends.qt_compat._setup_pyqt5plus()

    I get the same exception.

    However, I just found a fix! Since I know I'm using PySide6, I can run this line

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Line 83 in bff4681

    from PySide6 import QtCore, QtGui, QtWidgets, __version__

    no problem, since PySide6 has always been installed. Once I do that, the above imports of qt_compat._setup_pyqt5plus() works and %matplotlib qt works.

    from PySide6 import QtCore, QtGui, QtWidgets, __version__
    import shiboken6
13. kwsp commented on Apr 25, 2023

    

    kwsp

    on Apr 25, 2023

    Last edited by kwsp

    ContributorAuthor

    More actions

    Ok I stepped through qt_compat.py with a debugger and found the issue.

    When I run %matplotlib qt, mpl loads qt_compat.py which in turn executes the file.

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Lines 39 to 46 in bff4681

    	if sys.modules.get("PyQt6.QtCore"):
    	QT_API = QT_API_PYQT6
    	elif sys.modules.get("PySide6.QtCore"):
    	QT_API = QT_API_PYSIDE6
    	elif sys.modules.get("PyQt5.QtCore"):
    	QT_API = QT_API_PYQT5
    	elif sys.modules.get("PySide2.QtCore"):
    	QT_API = QT_API_PYSIDE2

    After this, QT_API remains undefined because no Qt bindings have been imported yet. So, we fall through to this case:

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Lines 47 to 59 in bff4681

    	# Otherwise, check the QT_API environment variable (from Enthought). This can
    	# only override the binding, not the backend (in other words, we check that the
    	# requested backend actually matches). Use _get_backend_or_none to avoid
    	# triggering backend resolution (which can result in a partially but
    	# incompletely imported backend_qt5).
    	elif (mpl.rcParams._get_backend_or_none() or "").lower().startswith("qt5"):
    	if QT_API_ENV in ["pyqt5", "pyside2"]:
    	QT_API = _ETS[QT_API_ENV]
    	else:
    	_QT_FORCE_QT5_BINDING = True # noqa
    	QT_API = None
    	# A non-Qt backend was selected but we still got there (possible, e.g., when
    	# fully manually embedding Matplotlib in a Qt app without using pyplot).

    For my environment, mpl.rcParams._get_backend_or_none() returns Qt5Agg, even though I have no Qt5 bindings installed. Still, we run into this case

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Line 56 in bff4681

    _QT_FORCE_QT5_BINDING = True # noqa

    So now, QT_API = None and _QT_FORCE_QT5_BINDING = True, and we fall though to this case while _setup_qt5plus() is never executed.

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Line 115 in bff4681

    if _QT_FORCE_QT5_BINDING:

    Obviously, we now get to this line since no Qt5 bindings are installed.

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Line 134 in bff4681

    raise ImportError(

    My work around of importing PySide6.QtCore manually above basically negates this issue because once Qt6 bindings are in sys.modules, qt_compat.py would set QT_API = QT_API_PYSIDE6.
14. kwsp commented on Apr 25, 2023

    

    kwsp

    on Apr 25, 2023

    Last edited by kwsp

    ContributorAuthor

    More actions

    On a first pass look, it seems for me this is caused by

    matplotlib/lib/matplotlib/backends/qt_compat.py

    Line 56 in bff4681

    _QT_FORCE_QT5_BINDING = True # noqa

    which if I check git blame, you committed 2bc0c1c

    So the question is: is this line of forcing Qt5 causing this bug, or is it the fact that MPL is using Qt5Agg in my environment when I only have Qt6 bindings installed the problem? Am not familiar with Agg backends and what they mean so would love your input
15. tacaswell commented on Apr 25, 2023

    

    tacaswell

    on Apr 25, 2023

    Member

    More actions

    For my environment, mpl.rcParams._get_backend_or_none() returns Qt5Agg, even though I have no Qt5 bindings installed.

    This says something in your system is setting the backend to 'qt5agg' which we take to mean "I want to us Qt5" (see #22005 and the linked issuse). Is there something in your environment forcing the backend to 'qt5agg'? Could be an matplotlibrc or a mpl.use in a start up script.

    The bug to fix here is that if we are restricting to Qt5 then the error message should not list the Qt6 bindings!
16. kwsp commented on Apr 25, 2023

    

    kwsp

    on Apr 25, 2023

    Last edited by kwsp

    ContributorAuthor

    More actions

    I doubt its an environment issue, since I can reproduce this problem on 3 separate machines (Windows 10, Windows 11, Ubuntu 22.04) with a clean conda environment. Could you point me to how mpl selects the default backend? I'm curious to see where in code MPL decides to use the Qt5Agg backend by default. I just tested again in a clean conda environment on Windows 10, with just matplotlib and PySide6 installed, and trying %matplotlib qt still gives me that exception. Here's my pip list

    (test) PS C:\Users\tnie\code\tmp> pip list
    Package                       Version
    ----------------------------- -------
    asttokens                     2.2.1
    backcall                      0.2.0
    backports.functools-lru-cache 1.6.4
    colorama                      0.4.6
    contourpy                     1.0.7
    cycler                        0.11.0
    debugpy                       1.5.1
    decorator                     5.1.1
    executing                     1.2.0
    fonttools                     4.39.3
    importlib-metadata            6.6.0
    ipykernel                     6.15.0
    ipython                       8.12.0
    jedi                          0.18.2
    jupyter_client                8.2.0
    jupyter_core                  5.3.0
    kiwisolver                    1.4.4
    matplotlib                    3.7.1
    matplotlib-inline             0.1.6
    nest-asyncio                  1.5.6
    numpy                         1.24.3
    packaging                     23.1
    parso                         0.8.3
    pickleshare                   0.7.5
    Pillow                        9.5.0
    pip                           23.0.1
    platformdirs                  3.3.0
    prompt-toolkit                3.0.38
    psutil                        5.9.0
    pure-eval                     0.2.2
    Pygments                      2.15.1
    pyparsing                     3.0.9
    PySide6                       6.5.0
    PySide6-Addons                6.5.0
    PySide6-Essentials            6.5.0
    python-dateutil               2.8.2
    pywin32                       305.1
    pyzmq                         23.2.0
    setuptools                    66.0.0
    shiboken6                     6.5.0
    six                           1.16.0
    stack-data                    0.6.2
    tornado                       6.2
    traitlets                     5.9.0
    typing_extensions             4.5.0
    wcwidth                       0.2.6
    wheel                         0.38.4
    zipp                          3.15.0
    
17. tacaswell commented on Apr 25, 2023

    

    tacaswell

    on Apr 25, 2023

    Member

    More actions

    I'm curious to see where in code MPL decides to use the Qt5Agg backend by default.

    It should not,

    matplotlib/lib/matplotlib/pyplot.py

    Lines 233 to 267 in b3bd929

    	if newbackend is rcsetup._auto_backend_sentinel:
    	current_framework = cbook._get_running_interactive_framework()
    	mapping = {'qt': 'qtagg',
    	'gtk3': 'gtk3agg',
    	'gtk4': 'gtk4agg',
    	'wx': 'wxagg',
    	'tk': 'tkagg',
    	'macosx': 'macosx',
    	'headless': 'agg'}
    	best_guess = mapping.get(current_framework, None)
    	if best_guess is not None:
    	candidates = [best_guess]
    	else:
    	candidates = []
    	candidates += [
    	"macosx", "qtagg", "gtk4agg", "gtk3agg", "tkagg", "wxagg"]
    	# Don't try to fallback on the cairo-based backends as they each have
    	# an additional dependency (pycairo) over the agg-based backend, and
    	# are of worse quality.
    	for candidate in candidates:
    	try:
    	switch_backend(candidate)
    	except ImportError:
    	continue
    	else:
    	rcParamsOrig['backend'] = candidate
    	return
    	else:
    	# Switching to Agg should always succeed; if it doesn't, let the
    	# exception propagate out.
    	switch_backend("agg")
    	rcParamsOrig["backend"] = "agg"
    	return

    is our fallback logic which is why I think it is something else setting the backend to 'qt5agg'.
18. kwsp commented on Apr 26, 2023

    

    kwsp

    on Apr 26, 2023

    Last edited by kwsp

    ContributorAuthor

    More actions

    Here's are the exact steps to reproduce on Windows 10 and Windows 11.

    1. Create a clean Conda environment and install deps

    conda create -y -n test python=3.10
    conda activate test
    # make sure env is actually active and make sure pip comes from this env.
    pip install matplotlib ipykernel PySide6  # all packages are pip installed.
    

    2. In VS Code interactive, run the following cells step by step

    # %%
    import matplotlib.pyplot as plt
    import matplotlib as mpl
    
    # %%
    mpl.get_backend()  # returns 'module://matplotlib_inline.backend_inline'
    
    # %%
    plt.plot(range(10))  # Plots inline OK
    
    # %%
    mpl.get_backend()  # Still returns 'module://matplotlib_inline.backend_inline'
    
    # %%
    %matplotlib qt
    plt.plot(range(10))  # ImportError: Failed to import any of the following Qt binding modules...
    
    # %%
    mpl.get_backend()  # returns 'Qt5Agg'
19. kwsp commented on Apr 26, 2023

    

    kwsp

    on Apr 26, 2023

    ContributorAuthor

    More actions

    I'm curious to see where in code MPL decides to use the Qt5Agg backend by default.

    It should not,

    matplotlib/lib/matplotlib/pyplot.py

    Lines 233 to 267 in b3bd929

    	if newbackend is rcsetup._auto_backend_sentinel:
    	current_framework = cbook._get_running_interactive_framework()
    	mapping = {'qt': 'qtagg',
    	'gtk3': 'gtk3agg',
    	'gtk4': 'gtk4agg',
    	'wx': 'wxagg',
    	'tk': 'tkagg',
    	'macosx': 'macosx',
    	'headless': 'agg'}
    	best_guess = mapping.get(current_framework, None)
    	if best_guess is not None:
    	candidates = [best_guess]
    	else:
    	candidates = []
    	candidates += [
    	"macosx", "qtagg", "gtk4agg", "gtk3agg", "tkagg", "wxagg"]
    	# Don't try to fallback on the cairo-based backends as they each have
    	# an additional dependency (pycairo) over the agg-based backend, and
    	# are of worse quality.
    	for candidate in candidates:
    	try:
    	switch_backend(candidate)
    	except ImportError:
    	continue
    	else:
    	rcParamsOrig['backend'] = candidate
    	return
    	else:
    	# Switching to Agg should always succeed; if it doesn't, let the
    	# exception propagate out.
    	switch_backend("agg")
    	rcParamsOrig["backend"] = "agg"
    	return

    is our fallback logic which is why I think it is something else setting the backend to 'qt5agg'.

    I set a breakpoint on line 234 here, stepped through the code that raised the ImportError, and confirmed this branch was never hit.
20. kwsp commented on Apr 26, 2023

    

    kwsp

    on Apr 26, 2023

    Last edited by kwsp

    ContributorAuthor

    More actions

    Ok what the hell. The problem isn't with matplotlib, but IPython. I hardcore stepped through the code this time, and it turns out if you do %matplotlib qt, qt always maps to Qt5Agg. So if I only have Qt6 bindings, this would always break because IPython tells matplotlibto use Qt5Agg first, before matplotlib tries to actually import bindings.

    https://github.com/ipython/ipython/blob/main/IPython/core/pylabtools.py#L26

    https://github.com/ipython/ipython/blob/main/IPython/core/pylabtools.py#L301-L322

    👍React with 👍2Reacted by Axel and Tao Yang
21. ksunden commented on Apr 26, 2023

    

    ksunden

    on Apr 26, 2023

    Member

    More actions

    (I was about to post the same link, in fact just for posterity, I'll post the permalink rather than the main branch which can change: https://github.com/ipython/ipython/blob/396593e7ad8cab3a9c36fb0f3e26cbf79cff069c/IPython/core/pylabtools.py#L26)

    Short term, you should be resolved by doing %matplotlib qt6 instead... longer term, perhaps IPython should update that mapping.

    Regardless, going to close as this is not a change we can do, as far as I can tell
22. ksunden

    closed this as completedon Apr 26, 2023
23. ksunden commented on Apr 26, 2023

    

    ksunden

    on Apr 26, 2023

    Member

    More actions

    Thought a little bit more and opened #25772, which would at least have helped narrow in on this, by not purporting to have tried qt6 when it does not actually do so.

    👍React with 👍1Reacted by Taylor
24. QuLogic

    added

    status: downstream fix required

    on Apr 26, 2023