Ensure capture_output does not output to captured file descriptors#3537
Merged
jsiirola merged 30 commits intoPyomo:mainfrom Mar 27, 2025
Merged
Ensure capture_output does not output to captured file descriptors#3537jsiirola merged 30 commits intoPyomo:mainfrom
capture_output does not output to captured file descriptors#3537jsiirola merged 30 commits intoPyomo:mainfrom
Conversation
mrmundt
requested changes
Mar 27, 2025
mrmundt
approved these changes
Mar 27, 2025
blnicho
approved these changes
Mar 27, 2025
pyomo/common/tee.py
Outdated
Comment on lines
+334
to
+335
| # if not hasattr(stream, 'write'): | ||
| # raise |
Member
There was a problem hiding this comment.
Is this debugging code that should be removed or history that could use context? If it can be removed then you could combine this except with the one below for OSError.
Member
Author
There was a problem hiding this comment.
I think it can be removed. At one point I was using that to better ensure that the stream was an actual stream (I think I was accidentally getting stream to be a tuple of streams, but that is handled elsewhere now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3520
Summary/Motivation:
This further improves
capture_outputwhencapture_fd=True. The core issue was that we were creating output loops when capturing the file descriptor ifcapture_outputwas capturing a file descriptor shared (directly or indirectly through a logger) with the stream that it was sending output to. While this solution is not bulletproof (I am not sure that we can ever be bullet proof against things like user-defined classes), this is significantly more robust than the previous implementations.This primarily resolves issues with solvers where
teeis implemented withcapture_fd=True(cyipopt, highs).Thanks to @jlinderoth for finding the issue and providing the MRE.
Changes proposed in this PR:
LoggingInterceptcapture_outputto better guarantee all contexts are unwound correctlyLegal Acknowledgement
By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution: