[Bug]: Python interpreter becomes slow at reading inputs after plotting with matplotlib #27515

Activity

1. tacaswell commented on Dec 14, 2023

   

   tacaswell

   on Dec 14, 2023

   Member

   More actions

   How exactly are you running this code? Are you sending the whole script at once or copy-pasting one line at a time into a shell (I'm on poor internet and can not watch the video)?

   One theory is that this sounds like the "app nap" issue again.

   My memory of the problem is that osX will "nap" applications that you are not using. When you show the plot Python becomes a GUI application and is now eligible for napping. When you close the window it looks to the OS like you are no longer using that application (because how can you use a program without a GUI?) and so Python only wakes up every so often making it slow.

   An alternate theory is that this is the event hook for osx we addded behaving badly.

   Things to test:

   • if you downgrade Matplotlib does it start working better?
   • if you use a different backend does it start working better?
   • if you use IPython (or any non-readline based prompt) does it work better?
   • Does this depend on putting a small number of long strings into numpy arrays (they are dtype <U124 which while valid is an odd thing to put in an array)?
   • if you add plt.ion() to the top does it get better?

   Contrary to me expectations running this as a script on linux show a small effect (6-8x slow down but like 1e-6 vs 8e-6). Adding plt.ion() removes the slow down, but using plt.ion() with plt.show(block=True) returns the slow down.

   import numpy as np
   import matplotlib.pyplot as plt
   import time
   import gc
   
   # plt.switch_backend("agg")
   plt.ion()
   # Before plotting, reading and creating the below array is very fast.
   pre = []
   for j in range(50):
       start_1 = time.monotonic()
       for j in range(100):
           Test_Array_1 = np.array(
               [
                   "An incredibly simple array that has lots of characters in it to show how slow VSCode starts to run after plotting something.",
                   "Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah",
                   "Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah",
                   "Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah",
               ]
           )
   
       end_1 = time.monotonic()
       print(end_1 - start_1)
       pre.append(end_1 - start_1)
   # Create and show a simple plot
   plt.plot(np.arange(10))
   plt.show(block=True)
   # gc.collect()
   post = []
   for j in range(50):
       # Now if I create the same array as before, the python interpreter takes much longer to read the input.
       start_2 = time.monotonic()
       for j in range(100):
           Test_Array_2 = np.array(
               [
                   "An incredibly simple array that has lots of characters in it to show how slow VSCode starts to run after plotting something.",
                   "Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah",
                   "Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah",
                   "Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah",
               ]
           )
   
       end_2 = time.monotonic()
       print(end_2 - start_2)
       post.append(end_2 - start_2)
   # After plotting, the interpreter is magnitudes slower (ratio given by below calculation)
   print(f"{np.mean(pre)=}, {np.std(pre)}")
   print(f"{np.mean(post)=}, {np.std(post)}")
   
   print(np.mean(post) / np.min(pre))
   
   print(Test_Array_1.dtype)

   is the slightly modified script I've been testing with via python test.py
2. aradley commented on Dec 14, 2023

   

   aradley

   on Dec 14, 2023

   Author

   More actions

   I just downgraded to matplotlib 3.6 (random choice for downgrade) and it removed the problem, and this problem only started a month of so ago, so it appears to be an issue with the new version?
3. aradley commented on Dec 14, 2023

   

   aradley

   on Dec 14, 2023

   Author

   More actions

   Adding "plt.ion()" did not improve the issue for matplotlib==3.8.2
4. oscargus commented on Dec 14, 2023

   

   oscargus

   on Dec 14, 2023

   Member

   More actions

   Thanks for checking!

   It turns out that there are modification to both 3.8.1 (#27221 and #26970) and 3.8.2 (#27290) for the mac backend, that, at least theoretically without knowing the details, possibly may cause something like this. Would it be possible for you to downgrade to 3.8.1 (and if the problem is still there, to 3.8.0)? (Note that 3.7.4 has the same fix as 3.8.2, so if the problem is still on 3.8.0, 3.7.3 is the next to try).
5. aradley commented on Dec 14, 2023

   

   aradley

   on Dec 14, 2023

   Author

   More actions

   The problem is not fixed in 3.8.1, appears improved but not gone in 3.8.0 (seems weird but that's what I found), and appears entirely gone in 3.7.3.
6. oscargus commented on Dec 14, 2023

   

   oscargus

   on Dec 14, 2023

   Member

   More actions

   Thanks for checking again! Then there a few more PRs that can have caused it. I think @greglucas is the one with the most knowledge in the area (and he has authored some of those PRs as well).

   👍React with 👍1Reacted by Greg Lucas
7. greglucas commented on Dec 16, 2023

   

   greglucas

   on Dec 16, 2023

   Contributor

   More actions

   The reason for this is that we are running the main runloop for 0.01 seconds each new character we get, which adds up in a hurry in this situation. I just put up a PR to remove that pause in #27528, which sped things up for me locally.
8. jklymak

   closed this as completedin #27528on Dec 16, 2023
9. QuLogic

   added this to the v3.8.3 milestone on Dec 16, 2023
10. greglucas

    added a commit that references this issue on Oct 23, 2024

    Rework stdin handling on macos objective C

    18909c7