The following issue describes the current behavior and outlines a few questions that need to be figured out to write a proposal.

Current Behavior (pre-rewrite)

Currently, there are three places where users can view different output types from their test run. First is the "python test log" under the output panel, second is the "Test Result" panel, and finally the in-line pop-up for failed tests. The "Test Result" panel and the in-line pop-up are identical in output (when the in-line is displayed since it isn't displayed on pass). Therefore for this discussion, we will just consider the "python test log" and the "Test Result" panel but all things printed to the "Test Result" panel will show up on the in-line pop-up if changes are made.

Due to prior constraints, the extension used to parse output to determine the result of test run and update the UI. This is the primary reason for the two different output views but due to the testing rewrite this is no longer a constraint. This means a decision needs to be made moving forward regarding these two channels and their information.

The current difference between the two are noted below:

comparison on the differences just as text for a test that fails and has a print statement. (right: python test output, left: Test Result)

comparison on the difference just as text for a test that fails with logging enabled. (right: python test output, left: Test Result)

Moving Forward

The following questions arise after the above analysis. Please add your feedback in comments below.

1. Would be better if we could instead switch all the necessary information to the Test Result panel? If we do this then do we even need the "python test output"?
2. If we want to keep the two then what is the difference between them?
3. What of the given information is most useful? Should we have the Test Results panel look as similar to the pytest output as possible (like what pytest returns to the command line)? If we do this do we still want custom additions to the Test Result panel such as "workspace directory", "args", and "test summary (section that goes "Total number of tests expected to run: 1.....")"?
4. When we do these types of changes what should the "show output button on the testing explorer go to? What should tests open on default when test run finish? Are there any more commands / ways to pull up these different outputs that should be added?

Pros and Cons

• If we switch to using the Test Result panel then we can support color in the output since it's all parsed by xterm.js

The following issues are related to this spike and those involved in these issues are encouraged to give their opinion. Thanks!

• Add "Python Test Log" to the command palette #21694
• pytest results in VSCode should show logging. #21781
• pytest assertions should generate rich diffs #17112
• Syntax highlighting in the output of tests explorer #17411
• Test runner monopolizes stdin, stdout, stderr #18346
• Open correct position in Pytest Test Log with a single click on any test #17371
• Add "Python Test Log" to the command palette #21694
• weird behavior of "Debug Test" using pytest #21833
• Python testing using new EOT token PYTEST - test output #22083
• if we go with the test panel then color will be supported, likely addressing these issues:
  • pytest colored output (with --color=yes) makes the test output log unreadable (by humans) #18119
  • No output panel color support #12615