[Bug]: Calling matplotlib.pyplot.show() outside of matplotlib.pyplot.rc_context no longer works

Bug summary

Creating a plot inside of matplotlib.pyplot.rc_context and then calling matplotlib.pyplot.show() outside said context causes the program to freeze.

Code for reproduction

import matplotlib.pyplot as plt

with plt.rc_context():
    plt.plot([1, 3, 2])

plt.show()

Actual outcome

The program freezes.

Expected outcome

The plot is shown.

Additional information

matplotlib.pyplot.savefig() works as expected.
The bug did not occur in earlier versions.
The bug does not occur when using Jupyter, only when using the regular Python interpreter.
The bug does not occur if plt.show() is called inside of plt.rc_context.

Operating system

Linux, 5.17.9-1-MANJARO, all packages up-to-date

Matplotlib Version

3.5.2

Matplotlib Backend

QtAgg

Python version

3.10.4

Jupyter version

3.4.2

Installation

Linux package manager

I cannot reproduce on master, MacOS, Qt5Agg, from the interpreter. Do you have something interesting in your matplotlibrc?

I have also tried on main with CentOS, both using python and ipython interpreters, and it does not happen here either.

Do you have something interesting in your matplotlibrc?

I have made no changes to my global matplotlibrc file.

I also get the bug when using ipython.

Also perhaps I should attach the stack trace I get when interrupting the process:

KeyboardInterrupt                         Traceback (most recent call last)
File ~/Projects/snek/plot_no_show.py:6, in <module>
      3 with plt.rc_context():
      4     plt.plot([1, 3, 2])
----> 6 plt.show()

File /usr/lib/python3.10/site-packages/matplotlib/pyplot.py:389, in show(*args, **kwargs)
    345 """
    346 Display all open figures.
    347 
   (...)
    386 explicitly there.
    387 """
    388 _warn_if_gui_out_of_main_thread()
--> 389 return _get_backend_mod().show(*args, **kwargs)

File /usr/lib/python3.10/site-packages/matplotlib/backend_bases.py:3544, in _Backend.show(cls, block)
   3542         block = True
   3543 if block:
-> 3544     cls.mainloop()

File /usr/lib/python3.10/site-packages/matplotlib/backends/backend_qt.py:1054, in _BackendQT.mainloop()
   1052 @staticmethod
   1053 def mainloop():
-> 1054     with _maybe_allow_interrupt(qApp):
   1055         qt_compat._exec(qApp)

File /usr/lib/python3.10/contextlib.py:142, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    140 if typ is None:
    141     try:
--> 142         next(self.gen)
    143     except StopIteration:
    144         return False

File /usr/lib/python3.10/site-packages/matplotlib/backends/qt_compat.py:276, in _maybe_allow_interrupt(qapp)
    274 signal.signal(signal.SIGINT, old_sigint_handler)
    275 if handler_args is not None:
--> 276     old_sigint_handler(*handler_args)

KeyboardInterrupt:

Switching the backend seems to fix the issue.
With the backend "GTK3Agg" the bug does not occur.

Are you sure the program is freezing or is the QT window just not raising to the front?

I think what actually happens is that the plot window is not shown.

I think your install is just bad - perhaps crossed Qt libraries. I don't think we have an action here, but if you have further debugging details we can chase down, let us know!

Something strange is definitely going on here.
I was trying to figure out which of the backends specifically causes problems.
However, when manually setting a backend with the MPLBACKEND environmental variable, I was not able to reproduce the behavior I described in the OP with any of the available backends.
The backends I tested: ['GTK3Agg', 'GTK3Cairo', 'GTK4Agg', 'GTK4Cairo', 'MacOSX', 'nbAgg', 'QtAgg', 'QtCairo', 'Qt5Agg', 'Qt5Cairo', 'TkAgg', 'TkCairo', 'WebAgg', 'WX', 'WXAgg', 'WXCairo', 'agg', 'cairo', 'pdf', 'pgf', 'ps', 'svg', 'template']

What's more, when adding the following print statement to my code the plot is shown correctly:

import matplotlib as mpl
import matplotlib.pyplot as plt

print(mpl.get_backend())

with plt.rc_context():
    plt.plot([1, 3, 2])

plt.show()

Console output: QtAgg

This suggests to me that the issue may be caused by matplotlib after all.

So I'm deeply confused. What does cause the error?

Perhaps it's an issue with garbage collection?
I don't know how matplotlib works internally but if the backend is only initialized when needed and if essential plot components do not have references from outside the rc_context then maybe they are lost when the context closes.
So when mpl.get_backend() is called beforehand the backend is already initialized and the components survive when the context closes.

Some of the package managers do things with backends as well. It would be good to know how we are to repo this.

I did some more testing:
I installed matplotlib in a virtual environment (using virtualenv).
The default backend in this virtual environment was TkAgg.
With matplotlib v3.5.2 the plot is again not correctly shown.
However, what happens in this case is that the plot is shown for a brief moment before the window closes automatically and the program terminates without any console output.
With matplotlib v3.5.1 the plot is correctly shown.
When adding a print statement as above the plot is also shown correctly when using matplotlib v3.5.2 .

I think I'll try to identify the specific commit that causes the problem.

I used git bisect to try and identify the commit that introduced the bug.
Unfortunately I was not able to identify the exact commit due to another bug that caused matplotlib import to fail:

Traceback (most recent call last):
  File "/home/johannesg/Projects/test/plot_no_show.py", line 1, in <module>
    import matplotlib.pyplot as plt
  File "/home/johannesg/Projects/test/lib/python3.10/site-packages/matplotlib/__init__.py", line 896, in <module>
    defaultParams = rcsetup.defaultParams = {
  File "/home/johannesg/Projects/test/lib/python3.10/site-packages/matplotlib/__init__.py", line 899, in <dictcomp>
    rcParamsDefault[key]),
  File "/home/johannesg/Projects/test/lib/python3.10/site-packages/matplotlib/__init__.py", line 672, in __getitem__
    return dict.__getitem__(self, key)
KeyError: 'webagg.port'

The result of git bisect:

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
2bc0c1c4b97d00c0f6e849f799cc9a670a736238
89159614b349ff5f45bc4d49202362ddcd28345f
5336d0e45382878fa185634f8816a113726ebe02
fdfbf7ce9dfe457a3e6d7f0198e51386397f08da
48db078218fcc08fd2202a4409726a18d610b9cc
efc7f81cf0ee0f9f2875bd1dc5eabf48b06ae14e
c68f9d887934209d0fdf45fc8635a1b7d387620a
We cannot bisect more!

You likely need to remove your matplotlibrc. Also how are you doing your install at each step? We usually recommend pip install -e from the source directory. That will cut down on the pip installs you need to do.

Thanks for putting so much effort into this.

I don't understand why but when adding the --editable flag I no longer get an error on import.
(Thanks for the pointer, this is a useful flag.)

The output of git bisect:

89159614b349ff5f45bc4d49202362ddcd28345f is the first bad commit
commit 89159614b349ff5f45bc4d49202362ddcd28345f
Author: Thomas A Caswell <[email protected]>
Date:   Sat Dec 18 20:24:15 2021 -0500

    FIX: delay resolving automatic backend until actually needed
    
    This is to prevent the early importing of GUI bindings in the case where the
    user wants to use something later in our search list, but also has a toolkit
    higher in our list installed which we select.
    
    Thanks to @anntzer for the implementation suggestion.

 lib/matplotlib/pyplot.py                          | 40 +++++++++++++++--------
 lib/matplotlib/tests/test_backend_tk.py           |  1 +
 lib/matplotlib/tests/test_backends_interactive.py | 24 ++++++++++++++
 lib/matplotlib/tests/test_rcparams.py             |  3 +-
 4 files changed, 53 insertions(+), 15 deletions(-)

I strongly suspect that this will be fixed by #23299

This is my story of what is going on:

the default backend is a "find a working GUI". When you access the value we resolve it and then write that value back (remember this point)
we used to resolve this at pyplot import time, but not actually start the UI application etc (which caused the bug 8915961 fixed when we picked pyqt6 and the user tried to use pyqt5)
to solve that we stopped resolving the backend at pyplot import time (it seemed innocent enough!)
rcParams is a dict subsclass with some validation on __getitem__ / __setitem__
with rc_context we stash everything in the rcparams via the dict methods which go around our logic (because we know they are valid, no reason to re-do the work)
in the context manager you create a figure which finally triggers the backend resolution (that as as side effect updates the backend in rcParams)
when exiting the context we use dict methods to reset the rcparam values including going back to our internal sentinel value. Which half resets the backend.
This go wrong (I would have to trace the code to sort out exactly what is going wrong)

The reason adding the print fixes it is you resolve the backend before the context manager so it does not get changed. Similarly if you explicitly set the backedn via MPLBACKEND the fallback logic is never involved so it all works.

#23298 is a different description of the same underlying problem.

@JohannesGaessler

@JohannesGaessler Thank you for putting so much effort into chasing this down and I apologize for sending you on a wild goose chase.

I can confirm that when using the example code in the OP the plot is shown correctly on my machine when installing from the matplotlib master branch.
The more complex application that I actually care about also works correctly with the matplotlib master branch.
I would consider this issue to be resolved from my end.

OK, that makes sense as to why it worked for us and not you! Sorry for not checking out 3.5.2 on my end!

Navigation Menu

Repository navigation

[Bug]: Calling matplotlib.pyplot.show() outside of matplotlib.pyplot.rc_context no longer works #23172

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug]: Calling matplotlib.pyplot.show() outside of matplotlib.pyplot.rc_context no longer works #23172

Description

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

Activity

jklymak commented on Jun 1, 2022

oscargus commented on Jun 1, 2022

JohannesGaessler commented on Jun 1, 2022

JohannesGaessler commented on Jun 1, 2022

jklymak commented on Jun 1, 2022

JohannesGaessler commented on Jun 1, 2022

jklymak commented on Jun 30, 2022

JohannesGaessler commented on Jun 30, 2022

jklymak commented on Jun 30, 2022

JohannesGaessler commented on Jun 30, 2022

jklymak commented on Jun 30, 2022

JohannesGaessler commented on Jun 30, 2022

JohannesGaessler commented on Jun 30, 2022

jklymak commented on Jun 30, 2022

JohannesGaessler commented on Jun 30, 2022

tacaswell commented on Jun 30, 2022

tacaswell commented on Jun 30, 2022

tacaswell commented on Jun 30, 2022

JohannesGaessler commented on Jun 30, 2022

jklymak commented on Jun 30, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions