[web] allow small golden deltas in HTML renderer #102791
Conversation
flutter-dashboard
bot
added
a: tests
|
It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!). If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix? Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. |
|
What's out testing strategy for the skia gold client? We don't have a |
Found it! |
| } | ||
| } | ||
|
|
||
| // Constructs arguments for `goltctl` for controlling how pixels are compared. |
| // because Chromium cannot exactly reproduce the same golden on all computers. | ||
| // It seems to depend on the hardware/OS/driver combination. However, those | ||
| // differences are very small (typically not noticeable to human eye). | ||
| List<String> _getPixelMatchingArguments(String testName) { |
There was a problem hiding this comment.
testName isn't used
4 participants
Tests running in HTML mode on LUCI sometimes produce tiny pixel deltas. They don't originate from the framework rendering something different, or any differences in the resulting HTML. Instead, in some environments Chromium renders a few pixels a tiny bit differently (worst case seen so far is 1/256 per component per pixel for a total of 8 pixels, example). This is not noticeable to a human eye, but Skia Gold catches it and fails the test, leading to flakiness.
This PR instructs Skia Gold to ignore differences of no more than 4 (component-wise manhattan distance) in no more than 20 pixels.
This is heavily inspired by the pixel matching strategy used in the web engine.
Fixes #102665