Skip to content

Comments

[ty] Fix benchmark assertion#22003

Merged
MichaReiser merged 1 commit intomainfrom
micha/benchmark-assertion
Dec 16, 2025
Merged

[ty] Fix benchmark assertion#22003
MichaReiser merged 1 commit intomainfrom
micha/benchmark-assertion

Conversation

@MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Dec 16, 2025

The intention of the assertion is to catch cases where an edit didn't introduce any new diagnostics (or we failed to fetch them). However, simply asserting on the counts isn't sufficient because the assertion then also fails if the number of new and fixed diagnostics by an edit are equal.

This PR changes the assertion to diff the diagnostics first and assert if there are any new diagnostics.

@MichaReiser MichaReiser added testing Related to testing Ruff itself ty Multi-file analysis & type inference labels Dec 16, 2025
@MichaReiser MichaReiser marked this pull request as ready for review December 16, 2025 11:09
@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 16, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@MichaReiser MichaReiser merged commit 01c0a3e into main Dec 16, 2025
39 checks passed
@MichaReiser MichaReiser deleted the micha/benchmark-assertion branch December 16, 2025 11:24
dcreager added a commit that referenced this pull request Dec 16, 2025
* main:
  [ty] Document `TY_CONFIG_FILE` (#22001)
  [ty] Cache `KnownClass::to_class_literal` (#22000)
  [ty] Fix benchmark assertion (#22003)
  Add uv and ty to the Ruff README (#21996)
  [ty] Infer precise types for `isinstance(…)` calls involving typevars (#21999)
  [ty] Use `FxHashMap` in `Signature::has_relation_to` (#21997)
  [ty] Avoid enforcing standalone expression for tests in f-strings (#21967)
  [ty] Use `title` for configuration code fences in ty reference documentation (#21992)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Related to testing Ruff itself ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant