py-fuzzer: fix minimization logic when --only-new-bugs is passed#17739
Merged
AlexWaygood merged 2 commits intomainfrom Apr 30, 2025
Merged
py-fuzzer: fix minimization logic when --only-new-bugs is passed#17739AlexWaygood merged 2 commits intomainfrom
--only-new-bugs is passed#17739AlexWaygood merged 2 commits intomainfrom
Conversation
Contributor
|
dcreager
added a commit
that referenced
this pull request
May 1, 2025
* main: [red-knot] Preliminary `NamedTuple` support (#17738) [red-knot] Add tests for classes that have incompatible `__new__` and `__init__` methods (#17747) Update dependency vite to v6.2.7 (#17746) [red-knot] Update call binding to return all matching overloads (#17618) [`airflow`] apply Replacement::AutoImport to `AIR312` (#17570) [`ruff`] Add fix safety section (`RUF028`) (#17722) [syntax-errors] Detect single starred expression assignment `x = *y` (#17624) py-fuzzer: fix minimization logic when `--only-new-bugs` is passed (#17739) Fix example syntax for pydocstyle ignore_var_parameters option (#17740) [red-knot] Update salsa to prevent panic in custom panic-handler (#17742) [red-knot] Ban direct instantiation of generic protocols as well as non-generic ones (#17741) [red-knot] Lookup of `__new__` (#17733) [red-knot] Check decorator consistency on overloads (#17684) [`flake8-use-pathlib`] Avoid suggesting `Path.iterdir()` for `os.listdir` with file descriptor (`PTH208`) (#17715) [red-knot] Check overloads without an implementation (#17681) Expand Semantic Syntax Coverage (#17725) [red-knot] Check for invalid overload usages (#17609)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes an issue in the py-fuzzer script described in #17702 (comment). Namely, if run with
--only-new-bugs, it's not sufficient for the script to just check that the bug exists in the new version of Ruff/red-knot when the script is minimizing the code that triggers the bug. The script must also check that the minimized version of the code also continues to not cause a panic on the baseline executable if--only-new-bugsis passed. Otherwise, the minimization has gone "too far". It's no longer a new bug if it's been minimized to a snippet that causes a panic on both the test executable and the baseline executable.The first commit here just simplifies the way we run red-knot on the randomly generated code: the
--projectCLI argument is no longer necessary, and it's no longer necessary to write apyproject.tomlfile to the temporary directory. The second commit here is the one that meaningfully fixes the bug.Test Plan
I checked that the script continues to pass with mypy using
uv run --directory ./python/py-fuzzer --dev mypyfrom the root of my local Ruff clone.I also did the following to test this change:
I checked out 4a621c2 (the commit prior to 8a6787b)
I ran
cargo build -p red_knot --target-dir ./target/baselineto create a baseline executable.I checked out this branch again.
I ran the following command:
which produced the following output:
I then confirmed that the script had correctly minimized the snippet to something that caused a panic on this branch but had not caused a panic prior to 8a6787b, unlike the buggy behaviour that the script displays on
main.