refactoring : removed explicits os.fork(), exceptions are propagated from child process to parent#1228
Conversation
alalazo
changed the title
os.fork(), exceptions are propagated from child process to parentos.fork(), exceptions are propagated from child process to parent
alalazo
changed the title
os.fork(), exceptions are propagated from child process to parentos.fork(), exceptions are propagated from child process to parent
|
Ready to be reviewed and tested |
|
@tgamblin |
|
@tgamblin I got the point failing spuriously here : I'll write a fix a ping you when done. |
|
@tgamblin Done : now it should be 99,9% safe 😄 |
|
@tgamblin I pushed some cosmetic changes to this in #1186 , let me know if you want them ported here |
|
@tgamblin Will check with my co-worker who uses Windows (he was the original reason for the request) and get back to you |
|
@goxberry: I looked into this. There are a couple more things that probably need to happen:
|
…ing/removed_explicit_fork Conflicts: lib/spack/spack/package.py
|
Closing as it will be difficult to maintain this alongside #1186 |
|
If it were merged would it be easy or is it a fundamental incompatibility? |
|
@tgamblin No fundamental incompatibility : all the features here are also in #1186 except that maintaining both will create conflicts (due to synchronization with In the end I think it will be easier to re-create something like #1228 if you need just that rather than continue with this PR. |
|
@alalazo: ok -- sounds good. Sorry for being slow! |
|
@tgamblin No worries. I closed it as leaving as it was was just adding noise to the list of PRs |
3 participants
Modifications
os.fork()in favor ofmultiprocessing.Processlog_outputlogic has been modified and does not require calls tosys.settraceanymorePackage.do_install.build_processand the exceptions are propagated to parent processChanges in
log_ouputIn
develop:__enter__the parent process forks , set his frame explicitly to avoid executing twice the same code (quite fragile to re-positioning of thewithstatement) , reads from a pipe and writes to file__enter__the child does the necessary to write to a pipe and then returns for executionIn the PR:
acquiremethod and is set to read from the pipereleasemethodNotes
In #1186 I wanted to be able to forward exceptions to parent to have a neater workflow in
do_install. As I saw a couple ofTODOsreferring to the same issue, I extracted this PR. This has also the great advantage of not tweaking withsys.settraceand frames (which is a major pain for the debugger sometimes).