Improve async workers to handle forking for async compile #142072
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/142072
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit a8b24b7 with merge base a8cc19b ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorch-bot
bot
added
ciflow/inductor
module: inductor
release notes: fx
masnesral
left a comment
masnesral
left a comment
There was a problem hiding this comment.
LG; I'm not sure about the FakeTensor part. If that's intentional, perhaps a comment?
| log.warning("SubprocPool unclean exit") | ||
| self.read_pipe.close() | ||
| while True: | ||
| job_id, data = _recv_msg(self.read_pipe) |
There was a problem hiding this comment.
Are you sure we no longer need a try around the recv? I'd assume that could raise if the pipe on the other end closes, for example
There was a problem hiding this comment.
I looked at it closer. I think we're both right - with the except in the place it was before then there was a problem with exceptions being caught and ignored. I think this latest version has better handling for more variety of problems.
cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv voznesenskym penguinwu Guobing-Chen XiaobingSuper zhuhaozhe blzheng jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: Check Labels / Check labels Details for Dev Infra teamRaised by workflow job |
cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv voznesenskym penguinwu Guobing-Chen XiaobingSuper zhuhaozhe blzheng jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Stack from ghstack (oldest at bottom):
cc @ezyang @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv @voznesenskym @penguinwu @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov