Allow tests to pass even when stderr got populated #10241
Conversation
|
I thought about this a bit: apparently some people use the stderr output to determine whether a test emitted e.g. sanitizer warnings. |
…sed with warnings"
jonasschnelli
force-pushed
the
2017/04/test_stderr
branch
from
7e0f99f to
41f392c
Compare
|
Added a new status "Passed with warnings" which gets set when the test passed but stderr was not empty. The test will be flagged as "successful" for that new status. |
| color = BLUE | ||
| glyph = TICK | ||
| if self.status == "Passed with warnings": | ||
| color = ORANGE |
There was a problem hiding this comment.
NameError: name 'ORANGE' is not defined
jnewbery
left a comment
jnewbery
left a comment
There was a problem hiding this comment.
Couple of comments inline.
If you add a new status string, you'll need to update print justification (otherwise I think your new category will just be truncated). Justification is on L337 in print_results() and L422 in TestResult.__repr__().
Now that we have 4 categories, it may make sense to define all the categories as constants at the top of the file and have a new MAX_STATUS_WIDTH constant as well.
One you've done that, can you add a screenshot of what this looks like?
| color = BLUE | ||
| glyph = TICK | ||
| if self.status == "Passed with warnings": | ||
| color = ORANGE |
There was a problem hiding this comment.
ORANGE would need to be a constant defined at the top of this file (see around L47)
| if self.status == "Passed": | ||
| color = BLUE | ||
| glyph = TICK | ||
| if self.status == "Passed with warnings": |
|
Concept ACK, but I will close this pull in case no one picks it up in the next couple of days. |
|
Closing for now. Feel free to pick this up again. |
d64ac3f [tests] Allow tests to pass when stderr is non-empty (Jonas Schnelli) Pull request description: Resurrect #10241 with nits addressed Not sure how much people want this. Would be useful for functional tests which cause bitcoind to print to stderr. Tree-SHA512: 28caccf7818fb3ed5a38caef7f77161b1678aa9b8fd12c2d1e76032f409f0d33c40f7ac91e0c8d908df4a44fd01cf97d657a08bae50c6ff17d07f5b2e20c3a6e
4 participants
No description provided.