Summary of discussion with T-libs-api about providing a richer API in test:

The biggest concern was with the compatibility surface that would have to be maintained and instead we went the opposite direction. Our aim is to make custom test harnesses first-class and shift the focus towards them rather than extending libtest. Hopefully, we can consolidate down on just a couple of frameworks with libtest providing us with the baseline API, reducing the chance that test writing between projects is too disjoint. I also hope we can share code between these projects to improve consistency and make it easier to conform to expectations.

See https://epage.github.io/blog/2023/06/iterating-on-test/

Tasks

• Vet all of the below with a reference implementation of a custom test harness: https://github.com/epage/pytest-rs/
• Opt-in to certain levels of cargo test / libtest interactions (--quiet propagation is unavailable to custom test harnesses cargo#15465)
• Allow specifying defaults for build target types cargo#6945
• #[distributed_slice] aka a method to enumerate tests #3
• Custom preludes to pull in custom test harness API #6
• main from a dependency? Might get this mostly for free by putting it in a prelude thanks for 1.79's imported_main
• Capturing stdout/stderr in custom test harnesses #8