bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread#13630
Conversation
1st1
left a comment
There was a problem hiding this comment.
Is it possible to make MultiChildWatcher to work like FastChildWatcher? Can we just fix them without introducing a new class? I think we discussed these options but I don't remember the resolution.
Sorry, no.
Currently we have O(N)
That's how asyncio works now and from the very beginning. I'm not very comfortable with changing policy on-demand initialization too much now. Calling |
|
|
I've quickly prototyped this: #13656 |
How about we don't allow users to use subprocess APIs from non-main threads if we don't have a working child watcher ready? |
|
I suspect that
I can implement
Also, I don't like to create a waiting thread per subprocess. I suspect that this approach doesn't scale well for 10,000 subprocesses. I don't want to apply this change in the last day before the feature freeze. We can return to a discussion about subprocess support redesign in Python 3.9 |
|
How about we rename ThreadedChildWatcher to DefaultChildWatcher? My reasoning: if it's called the latter, it could implement different watching strategy based on OS or Python version. On macOS we could use kwueue, on linux threads etc. Alternatively, we can keep this as is, implement KQueue watcher (in 3.9) and let asyncio decide which one should be the default at import time. Maybe this option is better and less confusing. Thoughts? |
|
Let's keep it named ThreadedChildWatcher :) |
Codecov Report
@@ Coverage Diff @@
## master #13630 +/- ##
==========================================
+ Coverage 82.76% 82.77% +<.01%
==========================================
Files 1843 1843
Lines 559912 559893 -19
Branches 41403 41431 +28
==========================================
+ Hits 463417 463457 +40
+ Misses 87421 87350 -71
- Partials 9074 9086 +12
Continue to review full report at Codecov.
|
|
@asvetlov: Please replace |
|
…is executed in non-main thread (python#13630)"
…en loop is executed in non-main thread (python#13630)" (pythonGH-13793)" This reverts commit 9535aff.
…ted in non-main thread (python#13630)
Implementation adds
MultiLoopChildWatcherand uses it by default.https://bugs.python.org/issue35621