Made wide gamut tests give more information in their failure #179415
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request significantly improves the diagnostic capabilities of wide gamut tests by providing more detailed failure information. Previously, tests would simply indicate whether a target color was found. Now, the modified _findRGBAF16Color, _findBGRA10Color, and _findBGR10Color functions return not only a boolean indicating color presence but also the closest color encountered in the image data. This is facilitated by the new _distanceSquared helper function and encapsulated within a custom _HasColor Matcher. The _HasColor Matcher's describeMismatch method now leverages this additional information to report the closest color, making test failures much easier to debug. This change directly aligns with the Flutter Style Guide's philosophy that error messages should be useful [^1].
| expect(bytes.lengthInBytes, byteData.lengthInBytes); | ||
| var foundDeepRed = false; | ||
| double minDistance = double.infinity; | ||
| List<double> closestColor = <double>[0, 0, 0]; |
There was a problem hiding this comment.
The closestColor is initialized to [0, 0, 0]. If the input bytes array were empty or if no pixels were processed for some reason (though expect statements mitigate this in the current test setup), closestColor would remain [0, 0, 0]. This could be misleading in the describeMismatch output of _HasColor, as [0, 0, 0] would be reported as the "closest color" even if no actual colors were sampled. Consider initializing closestColor to a value that clearly indicates no color has been processed yet, or ensure it's only assigned if at least one pixel is processed.
| expect(bytes.lengthInBytes, byteData.lengthInBytes); | ||
| var foundDeepRed = false; | ||
| double minDistance = double.infinity; | ||
| List<double> closestColor = <double>[0, 0, 0]; |
There was a problem hiding this comment.
Similar to the _findRGBAF16Color function, the closestColor is initialized to [0, 0, 0]. This could be misleading in the describeMismatch output if no pixels are processed. Consider a more explicit initialization or conditional assignment.
| expect(bytes.lengthInBytes, byteData.lengthInBytes); | ||
| var foundDeepRed = false; | ||
| double minDistance = double.infinity; | ||
| List<double> closestColor = <double>[0, 0, 0]; |
There was a problem hiding this comment.
Similar to the other _findColor variants, the closestColor is initialized to [0, 0, 0]. This could be misleading in the describeMismatch output if no pixels are processed. Consider a more explicit initialization or conditional assignment.
dev/integration_tests/wide_gamut_test/integration_test/app_test.dart
Outdated
Show resolved
Hide resolved
| if (_isAlmost(red, color[0], epsilon) && | ||
| _isAlmost(green, color[1], epsilon) && | ||
| _isAlmost(blue, color[2], epsilon)) { | ||
| foundDeepRed = true; |
There was a problem hiding this comment.
The "found" check is based on an epsilon for each rgb component, but the "closest" check is based on vector distance (squared) of the colors. I think it's weird that we're using a different color comparison check for finding the direct match vs finding the closest color.
To me, the vector distance seems like a more appropriate method of comparing colors than than the individual rgb component checks. Should we consider changing the "found" check to be distance < epsilon instead of comparing each individual component against epsilon? This way both the "found" and "closest" checks are based on the same method of color comparison. A side benefit is that the code would be simplified considerably too.
Making this change would be changing the behavior of the current "found" check. But I wouldn't expect it to affect any results assuming the epsilon is reasonable.
There was a problem hiding this comment.
Yea, that's a fair observation. The reason the epsilon is applied per component is that it's just easier to reason about. Whereas what does it mean for one color to be 0.01 euclidean distance from another?
I'd rather keep it this way for now.
There was a problem hiding this comment.
A color being 0.01 distance from another makes an equal amount of sense to me as a color's r/g/b value being 0.01 distance from another color's.
But I don't feel super strongly about this, so if you prefer keeping it as-is I'm fine with it.
There was a problem hiding this comment.
Thinking more about what it actually means, a 0.01 euclidean distance means that in a 3D color space, we're seeing if the color we're checking is contained within a sphere of diameter 0.02 that is centered on our target color.
And with the epsilon check for each individual rgb component, we're seeing if the color we're checking is contained within a cube with side length=0.02 that is centered on our target color.
There was a problem hiding this comment.
Color comparison is actually more complicated than that and it's a deep rabbit hole to get into. Check this out: https://en.wikipedia.org/wiki/CIE_1931_color_space
The Euclidean distance falls apart because that's not how human perception works. The deltas in the 3d space aren't uniformly perceived, that's why it's hard to chose an epsilon for distances.
dev/integration_tests/wide_gamut_test/integration_test/app_test.dart
Outdated
Show resolved
Hide resolved
| if (_isAlmost(red, color[0], epsilon) && | ||
| _isAlmost(green, color[1], epsilon) && | ||
| _isAlmost(blue, color[2], epsilon)) { | ||
| foundDeepRed = true; |
There was a problem hiding this comment.
A color being 0.01 distance from another makes an equal amount of sense to me as a color's r/g/b value being 0.01 distance from another color's.
But I don't feel super strongly about this, so if you prefer keeping it as-is I'm fine with it.
gaaclarke
added
the
autosubmit
flutter-dashboard
bot
removed
the
autosubmit
Roll Flutter from 69d8710fadcd to 5b8720312a94 (42 revisions) flutter/flutter@69d8710...5b87203 2025-12-05 [email protected] Roll Packages from d39e481 to b17d3ff (3 revisions) (flutter/flutter#179505) 2025-12-05 [email protected] Move target_cpu into use_rbe block in Fuchsia build (flutter/flutter#179458) 2025-12-05 [email protected] Roll Skia from fe6bf18a3f6b to a31411879251 (3 revisions) (flutter/flutter#179499) 2025-12-05 [email protected] Force WASM single threading in Chrome extensions. (flutter/flutter#179400) 2025-12-05 [email protected] Roll Skia from aea282ea0bcd to fe6bf18a3f6b (7 revisions) (flutter/flutter#179489) 2025-12-05 [email protected] fix: check both pointer count and action before reusing MotionEvent (flutter/flutter#178528) 2025-12-05 [email protected] Roll Fuchsia Linux SDK from VtDPX2k1kosHxjKUE... to vDeTl_KBeLJY8nCAx... (flutter/flutter#179478) 2025-12-05 [email protected] [ Widget Preview ] Add embedded Widget Inspector support (flutter/flutter#178116) 2025-12-05 [email protected] bubble up fragment shader errors (flutter/flutter#179363) 2025-12-05 [email protected] Roll Dart SDK from eda82318e193 to 42fe3327abca (1 revision) (flutter/flutter#179472) 2025-12-05 [email protected] [Android 16] Use AVD With New Android Renderer (flutter/flutter#179306) 2025-12-05 [email protected] Unpin `package:dwds` dependency (flutter/flutter#179462) 2025-12-05 [email protected] [Impeller] include uniform info in impellerc json reflections (flutter/flutter#179317) 2025-12-05 [email protected] Roll Skia from 1591b066f49b to aea282ea0bcd (1 revision) (flutter/flutter#179468) 2025-12-04 [email protected] Roll Skia from e1923478562b to 1591b066f49b (3 revisions) (flutter/flutter#179460) 2025-12-04 [email protected] Cherry-pick flutter 3.38.4 changelog back to master. (flutter/flutter#179463) 2025-12-04 [email protected] [web] Add clone method to LayerPicture and dispose pictures in PictureLayer (flutter/flutter#179162) 2025-12-04 [email protected] Fixes merge conflict from high bitrate texture tests (flutter/flutter#179416) 2025-12-04 [email protected] Fix typos in `VirtualDisplayController.java` (flutter/flutter#179411) 2025-12-04 [email protected] Roll Dart SDK from 2de44cc08970 to eda82318e193 (1 revision) (flutter/flutter#179453) 2025-12-04 [email protected] Roll Skia from 55d94a54f453 to e1923478562b (1 revision) (flutter/flutter#179449) 2025-12-04 [email protected] Roll Packages from 8cb4903 to d39e481 (8 revisions) (flutter/flutter#179451) 2025-12-04 [email protected] Add DropdownMenu.selectOnly (flutter/flutter#179189) 2025-12-04 [email protected] Roll Dart SDK from 7e6bfc6af55c to 2de44cc08970 (6 revisions) (flutter/flutter#179443) 2025-12-04 [email protected] Roll Skia from b8f79d7316c0 to 55d94a54f453 (1 revision) (flutter/flutter#179439) 2025-12-04 [email protected] Revise README for link updates and terminology changes (flutter/flutter#179357) 2025-12-04 [email protected] Roll Skia from ce19122e3982 to b8f79d7316c0 (3 revisions) (flutter/flutter#179436) 2025-12-04 [email protected] Update some BottomNavigationBar comments to reflect theme normalization (flutter/flutter#179404) 2025-12-04 [email protected] Replace use of eglCreateImage with eglCreateImageKHR to reduce EGL requirement (flutter/flutter#179310) 2025-12-04 [email protected] Roll Skia from 81a9a0751f00 to ce19122e3982 (5 revisions) (flutter/flutter#179430) 2025-12-04 [email protected] [impellerc] adds entry prefix flag to avoid shader collisions (flutter/flutter#179160) 2025-12-04 [email protected] [Windows] Allow apps to prefer high power GPUs (flutter/flutter#177653) 2025-12-04 [email protected] [ Infra ] Shard `Windows tool_tests_commands` (flutter/flutter#179409) 2025-12-04 [email protected] [flutter_tools] Fix filename typo (flutter/flutter#179427) 2025-12-04 [email protected] Marks Linux_pixel_7pro draw_arcs_all_stroke_styles_perf__timeline_summary to be unflaky (flutter/flutter#179392) 2025-12-03 [email protected] Marks Mac_ios draw_arcs_all_fill_styles_perf_ios__timeline_summary to be unflaky (flutter/flutter#179391) 2025-12-03 [email protected] Marks Linux_pixel_7pro draw_arcs_all_fill_styles_perf__timeline_summary to be unflaky (flutter/flutter#179390) 2025-12-03 [email protected] Roll Abseil to 564023aa5376 (flutter/flutter#179421) 2025-12-03 [email protected] Made wide gamut tests give more information in their failure (flutter/flutter#179415) 2025-12-03 [email protected] Roll Skia from c20f797ab6f9 to 81a9a0751f00 (5 revisions) (flutter/flutter#179417) 2025-12-03 [email protected] Migrate samples and docs to RadioGroup (flutter/flutter#179158) 2025-12-03 [email protected] Roll Dart SDK to 3.11.0-200.1.beta (flutter/flutter#179399) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages ...
…#179415) fixes flutter#179396 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
2 participants
fixes #179396
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.