Remove and deprecate coverage-guided testing#1564
Conversation
|
😭
Deleting code is easy. 😁
I did say. 😉 I wasn't just suggesting this out of petty deletionism. |
|
Right. All tests passing, should be ready for review. |
Zac-HD
left a comment
There was a problem hiding this comment.
- See comment below about (of all ironies) the deprecation pathway.
- Do you anticipate reusing TargetSelector? If not, might it be worth removing it or opening an issue to do so later?
Otherwise looks good to me 😄
| settings._define_setting( | ||
| 'use_coverage', | ||
| default=True, | ||
| default=False, |
There was a problem hiding this comment.
We should default this to not_set or None, so that we can warn users with an explicit use_coverage=False, which will also break when we fully remove the setting.
Maybe. I think there's scope for making its logic smarter in useful ways (and have some ideas that I may or may not pursue), and it's a reasonable thing to have encapsulated either way, so I thought I'd leave it as is. |
|
I'm a little surprised to see the structural-tag machinery disappear entirely. Did you decide that the diversity it adds to mutation isn't worth the complexity of retaining the full target selector, now that coverage no longer exists? (I'm not familiar with how useful that diversity was in practice, so I don't have strong opinions about this. Just curious to see it go.) |
Decided is maybe a bit of a strong word. Truth be told I started deleting code and just got a bit overenthusiastic. I think this is the right thing to do though. The More generally, I don't think we have a good sense of how much of the current generation code is actually useful, or a solid way of acquiring that sense, so I think we probably need a stronger design pressure towards "simple and fast" for the main loop than we currently have, especially if the goal is to have an entirely separate fuzzer. |
As per #1551, this deprecates the
use_coveragesetting and removes the coverage guided testing feature.Fixes #1551, fixes #1518, fixes #1493, fixes #1392, fixes #879,