Repository navigation
Revert: Allow color delta tolerance in in-memory fuzzy image comparator - #196067
Merged
auto-submit[bot] merged 1 commit intoNov 6, 2026
Merged
Conversation
…or (flutter#193937)" This reverts commit 426be96.
11 tasks done
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request simplifies the image comparison logic in goldens.dart by removing tolerance thresholds and updates canvas_test.dart to use the simplified exact comparison. Feedback suggests refactoring the updated fuzzyCompareImages method to avoid unsafe null assertions on toByteData() and to improve performance by replacing the nested loops with a single-pass Uint32List comparison.
Comment on lines
+70
to
79
| int getPixel(ByteData data, int x, int y) => data.getUint32((x + y * golden.width) * 4); | ||
| final ByteData goldenData = (await golden.toByteData())!; | ||
| final ByteData testImageData = (await testImage.toByteData())!; | ||
| for (var y = 0; y < golden.height; y++) { | ||
| for (var x = 0; x < golden.width; x++) { | ||
| if (getPixel(goldenData, x, y) != getPixel(testImageData, x, y)) { | ||
| return false; | ||
| } | ||
| } | ||
| return true; | ||
| } | ||
|
|
||
| // When tolerance thresholds are configured, inspect each pixel's color | ||
| // channels and count those exceeding [maxColorDelta] to determine whether | ||
| // the proportion of different pixels is within [maxDifferentPixelsRate]. | ||
| var differentPixels = 0; | ||
| var maxObservedDelta = 0.0; | ||
|
|
||
| final Uint8List goldenBytes = goldenData.buffer.asUint8List( | ||
| goldenData.offsetInBytes, | ||
| totalPixels * 4, | ||
| ); | ||
| final Uint8List testBytes = testImageData.buffer.asUint8List( | ||
| testImageData.offsetInBytes, | ||
| totalPixels * 4, | ||
| ); | ||
|
|
||
| for (var i = 0; i < totalPixels; i++) { | ||
| final double pixelDelta = _colorDistance(goldenBytes, testBytes, i * 4); | ||
| if (pixelDelta > maxObservedDelta) { | ||
| maxObservedDelta = pixelDelta; | ||
| } | ||
| if (pixelDelta > maxColorDelta) { | ||
| differentPixels++; | ||
| } | ||
| } | ||
|
|
||
| final double diffRate = differentPixels / totalPixels; | ||
| if (diffRate > maxDifferentPixelsRate) { | ||
| print( | ||
| 'fuzzyCompareImages failed: ' | ||
| 'maxObservedDelta=${maxObservedDelta.toStringAsFixed(3)} (threshold=$maxColorDelta), ' | ||
| 'differentPixels=$differentPixels/$totalPixels ' | ||
| '(${(diffRate * 100).toStringAsFixed(3)}%, maxAllowed=${(maxDifferentPixelsRate * 100).toStringAsFixed(3)}%)', | ||
| ); | ||
| return false; | ||
| } |
Contributor
There was a problem hiding this comment.
The reverted implementation uses the unsafe null assertion operator ! on toByteData(), which can throw a runtime exception if the image is disposed or if the data retrieval fails. Additionally, performing nested loops with pixel-by-pixel getUint32 calls is highly inefficient.
We can improve both safety and performance by safely handling null values and using a single-pass Uint32List comparison, which is significantly faster.
final ByteData? goldenData = await golden.toByteData();
final ByteData? testImageData = await testImage.toByteData();
if (goldenData == null || testImageData == null) {
return false;
}
final int totalPixels = golden.width * golden.height;
final Uint32List goldenUint32 = goldenData.buffer.asUint32List(goldenData.offsetInBytes, totalPixels);
final Uint32List testUint32 = testImageData.buffer.asUint32List(testImageData.offsetInBytes, totalPixels);
for (int i = 0; i < totalPixels; i++) {
if (goldenUint32[i] != testUint32[i]) {
return false;
}
}References
- Suggest simplification and refactoring: Assess whether the code can be made simpler or refactored to enhance readability and maintainability. (link)
andywolff
approved these changes
Nov 5, 2026
andywolff
left a comment
andywolff
left a comment
Contributor
There was a problem hiding this comment.
I don't see any new callsites which use the added arg, so this should be safe
andywolff
added
the
autosubmit
flutter-dashboard
Bot
removed
the
autosubmit
auto-submit Bot
pushed a commit
to flutter/packages
that referenced
this pull request
Nov 6, 2026
Roll Flutter from 3623d2d34543 to 09175cdeac89 (61 revisions) flutter/flutter@3623d2d...09175cd 2026-11-06 [email protected] Roll Skia from ae35b7b2221f to 793b8ac73d4b (3 revisions) (flutter/flutter#196091) 2026-11-06 [email protected] [web] Rasterize scenes at the size passed to `FlutterView.render` (flutter/flutter#196052) 2026-11-06 [email protected] Roll Skia from 4aa043a81e0c to ae35b7b2221f (1 revision) (flutter/flutter#196086) 2026-11-06 [email protected] Roll Fuchsia Linux SDK from Fqe3UlLwAohsPbsCS... to KIc0XZbhLmvDpTluZ... (flutter/flutter#196085) 2026-11-06 [email protected] Allow a zero scale change in InteractiveViewer (flutter/flutter#194949) 2026-11-06 [email protected] Roll Skia from dd5c2553a856 to 4aa043a81e0c (2 revisions) (flutter/flutter#196079) 2026-11-06 [email protected] [web] Keep ImageFilter.blur bounds in equality and description (flutter/flutter#196013) 2026-11-06 [email protected] iOS: Remove unused PlatformViewIOS setup from FlutterPlatformViewsTest (flutter/flutter#196033) 2026-11-06 [email protected] Serialize LoggerTests that mutate the shared Logger (flutter/flutter#196028) 2026-11-06 [email protected] Use reverseDuration for the AnimatedCrossFade size animation (flutter/flutter#194860) 2026-11-05 [email protected] Roll Skia from a5aacd103e68 to dd5c2553a856 (8 revisions) (flutter/flutter#196073) 2026-11-05 [email protected] Fix ListWheelScrollView null check error when it reads itemIndex before content dimensions are set (flutter/flutter#194859) 2026-11-05 [email protected] Allow zero cross axis extent in SliverGridDelegateWithMaxCrossAxisExtent (flutter/flutter#194858) 2026-11-05 [email protected] Revert: Allow color delta tolerance in in-memory fuzzy image comparator (flutter/flutter#196067) 2026-11-05 [email protected] Revert: Mitigate snapshot race in 3D transform test by reducing loop density (flutter/flutter#196066) 2026-11-05 [email protected] Delegate createStream in ResizeImage and ScrollAwareImageProvider (flutter/flutter#194756) 2026-11-05 [email protected] [iOS] Introduce FlutterShell and move thread host and profiler (flutter/flutter#195388) 2026-11-05 [email protected] [web] Support FragmentShader.getImageSampler on CanvasKit and Skwasm (flutter/flutter#195854) 2026-11-05 [email protected] Deprecates setFloat API (flutter/flutter#195505) 2026-11-05 [email protected] [flutter_tools] Remove public GlobalToolContext and globals from runner (flutter/flutter#195561) 2026-11-05 [email protected] Defer TLHC PlatformViewWrapper frame post and avoid initial 1x1 buffer resize (flutter/flutter#195861) 2026-11-05 [email protected] Roll Skia from b5abed66da86 to a5aacd103e68 (17 revisions) (flutter/flutter#196054) 2026-11-05 [email protected] Made `FlutterWindowControllerSizeTest.SizedToContentNotResizable` more robust (flutter/flutter#195845) 2026-11-05 [email protected] Update docs for contributor ladder (flutter/flutter#196011) 2026-11-05 [email protected] [Impeller] Use the transfer stage for the Vulkan readback barrier (flutter/flutter#194991) 2026-11-05 [email protected] Remove Mac mac_unopt builder (flutter/flutter#196027) 2026-11-05 [email protected] Updated Instructions for Determining Gradle Version for `gradle_dists` in CIPD (flutter/flutter#193310) 2026-11-05 [email protected] Move ios_app_with_extensions_test into the tool integration shard (flutter/flutter#195609) 2026-11-05 [email protected] [web] Respect Uint8List view bounds when detecting animated WebP/GIF (flutter/flutter#196010) 2026-11-05 [email protected] Add platform_views_layout to build_tests so iOS/Android build breaks are caught in presubmit (flutter/flutter#192985) 2026-11-05 [email protected] [flutter_tools] Detect macOS TCC permission crashes and provide diagnostic guidance (flutter/flutter#192432) 2026-11-05 [email protected] Roll Packages from 670b30e to 0478162 (15 revisions) (flutter/flutter#196047) 2026-11-05 [email protected] [web] Support Skwasm rendering without bitmap capture (flutter/flutter#195103) 2026-11-05 [email protected] [flutter_tools] Reland fix for local CanvasKit asset paths on Windows (flutter/flutter#195770) 2026-11-05 [email protected] Roll Skia from f96be51af95d to b5abed66da86 (14 revisions) (flutter/flutter#196017) 2026-11-05 [email protected] Split chrome web engine test drone (flutter/flutter#195873) 2026-11-05 [email protected] Roll Dart SDK from c9153304136a to 6623b84628f3 (1 revision) (flutter/flutter#196026) 2026-11-05 [email protected] Roll ICU from 4fe960221710 to a0ea93436c52 (6 revisions) (flutter/flutter#196025) 2026-11-05 [email protected] Add more guidance in README and AGENTS for contributing to flutter_tools (flutter/flutter#195998) 2026-11-05 [email protected] Roll Fuchsia Linux SDK from p0HDXLISPIElJfSz7... to Fqe3UlLwAohsPbsCS... (flutter/flutter#196019) 2026-11-05 [email protected] Roll Dart SDK from e362a23e9738 to c9153304136a (18 revisions) (flutter/flutter#196014) 2026-11-05 [email protected] [Impeller] Fix dilation input bounds when clipping saveLayers (flutter/flutter#195652) 2026-11-05 [email protected] Add `variantOf` to the M3E migration checklist (flutter/flutter#195099) 2026-11-05 [email protected] [flutter_tools] Eliminate globals from remaining subsystem files (flutter/flutter#195555) 2026-11-05 [email protected] Fix gap flipping when dragging variable height reorderable items (flutter/flutter#194478) 2026-11-05 [email protected] [A11y] Send un-merged SemanticsNodes in the same update that first references them (flutter/flutter#195829) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts: Allow color delta tolerance in in-memory fuzzy image comparator
Initiated by: @andywolff
Reason for reverting: not needed because #196063 is the true root cause of the flakiness
Original PR Author: @andywolff
Reviewed By: @gaaclarke
The original PR description is provided below:
Part of #193029
Pre-requisite for #192937
This PR adds color delta and pixel rate tolerance parameters to the engine's in-memory
ImageComparer.fuzzyCompareImageshelper, and updatesCanvas.drawParagraph renders tab as space instead of tofuto tolerate minor subpixel anti-aliasing variations across consecutive passes.While verifying #192937,
Linux linux_host_engine_testfailed repeatedly incanvas_test.dartunder Vulkan with SwiftShader.The test
Canvas.drawParagraph renders tab as space instead of tofurenders>\t<and> <into separate in-memory images viatoImageand compares them usingcomparer.fuzzyCompareImages(tabImage, spaceImage). Despite the namefuzzyCompareImages, the previous implementation was pixel exact with no fuzziness at all. How un-comfy!Under CPU-based software Vulkan drivers (such as SwiftShader on Linux CI runners), anti-aliasing edge rounding differences between separate snapshot passes produce 1–2 channel delta variations on glyph boundaries, causing the comparison to fail.
Previously, consecutive snapshots left un-cleared host buffer data behind that masked subpixel variations. With my leak fix PR, resetting the buffer on each snapshot forces every text pass to allocate from scratch, exposing small driver-level raster differences on software Vulkan.
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.