Skip to content

Comments

[ty] Filter out revealed-type and undefined-reveal diagnostics from mdtest snapshots#20820

Merged
AlexWaygood merged 2 commits intomainfrom
alex/suppress-reveal-type-snapshots
Oct 12, 2025
Merged

[ty] Filter out revealed-type and undefined-reveal diagnostics from mdtest snapshots#20820
AlexWaygood merged 2 commits intomainfrom
alex/suppress-reveal-type-snapshots

Conversation

@AlexWaygood
Copy link
Member

Summary

As we discussed in ty planning on Friday, revealed-type and undefined-reveal diagnostics often make mdtest snapshots very noisy. They're almost never relevant to the reason why you've enabled snapshotting for a specific mdtest. The undefined-reveal diagnostics can be avoided by explicitly importing reveal_type (which is not something you'd usually do in an mdtest), but the revealed-type diagnostics can only be avoided by having a separate mdtest entirely that enables snapshotting and avoids having any reveal_type calls in it.

This PR suppresses revealed-type and undefined-reveal diagnostics from the diagnostics snapshotted during an mdtest run. If we do want to capture a snapshot that shows how these diagnostics are displayed, we can still do that as an end-to-end CLI test in the ty crate.

Test Plan

Snapshots!

@AlexWaygood AlexWaygood requested a review from carljm as a code owner October 12, 2025 15:31
@AlexWaygood AlexWaygood added the testing Related to testing Ruff itself label Oct 12, 2025
@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Oct 12, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 12, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@github-actions
Copy link
Contributor

github-actions bot commented Oct 12, 2025

mypy_primer results

No ecosystem changes detected ✅
No memory usage changes detected ✅

@MichaReiser
Copy link
Member

I think I'd prefer if we disable the rule instead of filtering out the diagnostics here:

let rule_selection = RuleSelection::all(default_lint_registry(), Severity::Info);

We could then (in the future, not in this PR), add a rule_selection option to mdtests, to re-enable the rule when necessary

@AlexWaygood
Copy link
Member Author

I think I'd prefer if we disable the rule instead of filtering out the diagnostics here:

That would disable these rules for mdtest as a whole, rather than just the snapshots, though, right? There's a fair amount of logic in mdtest that's specifically there so that we don't have to do that. I suggested just suppressing these rules instead in mdtest just over a year ago, but @carljm was against it, which is why the approach I'm taking here attempts to specifically filter the diagnostics out of mdtest snapshots without disabling these diagnostics for regular mdtest.

@AlexWaygood AlexWaygood force-pushed the alex/suppress-reveal-type-snapshots branch from c9ede83 to 07b5c29 Compare October 12, 2025 18:29
@AlexWaygood AlexWaygood enabled auto-merge (squash) October 12, 2025 18:31
@AlexWaygood AlexWaygood merged commit 7064c38 into main Oct 12, 2025
40 checks passed
@AlexWaygood AlexWaygood deleted the alex/suppress-reveal-type-snapshots branch October 12, 2025 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Related to testing Ruff itself ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants