Skip to content

Commit 426be96

Browse files
andywolffandywolff
authored
Allow color delta tolerance in in-memory fuzzy image comparator (#193937)
Part of #193029 Pre-requisite for #192937 This PR adds color delta and pixel rate tolerance parameters to the engine's in-memory `ImageComparer.fuzzyCompareImages` helper, and updates `Canvas.drawParagraph renders tab as space instead of tofu` to tolerate minor subpixel anti-aliasing variations across consecutive passes. While verifying #192937, `Linux linux_host_engine_test` failed repeatedly in `canvas_test.dart` under Vulkan with SwiftShader. The test `Canvas.drawParagraph renders tab as space instead of tofu` renders `>\t<` and `> <` into separate in-memory images via `toImage` and compares them using `comparer.fuzzyCompareImages(tabImage, spaceImage)`. Despite the name `fuzzyCompareImages`, the previous implementation was pixel exact with no fuzziness at all. How un-comfy! Under CPU-based software Vulkan drivers (such as SwiftShader on Linux CI runners), anti-aliasing edge rounding differences between separate snapshot passes produce 1–2 channel delta variations on glyph boundaries, causing the comparison to fail. Previously, consecutive snapshots left un-cleared host buffer data behind that masked subpixel variations. With my leak fix PR, resetting the buffer on each snapshot forces every text pass to allocate from scratch, exposing small driver-level raster differences on software Vulkan. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent a4896ba commit 426be96

2 files changed

Lines changed: 103 additions & 9 deletions

File tree

‎engine/src/flutter/testing/dart/canvas_test.dart‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -761,8 +761,17 @@ void main() async {
761761
final Image tofuImage = await drawText('>\b<');
762762

763763
// The tab's image should be identical to the space's image but not the tofu's image.
764-
final bool tabToSpaceComparison = await comparer.fuzzyCompareImages(tabImage, spaceImage);
765-
final bool tabToTofuComparison = await comparer.fuzzyCompareImages(tabImage, tofuImage);
764+
// Allow a small color delta for anti-aliasing variations across consecutive passes.
765+
final bool tabToSpaceComparison = await comparer.fuzzyCompareImages(
766+
tabImage,
767+
spaceImage,
768+
maxColorDelta: 2.0,
769+
);
770+
final bool tabToTofuComparison = await comparer.fuzzyCompareImages(
771+
tabImage,
772+
tofuImage,
773+
maxColorDelta: 2.0,
774+
);
766775

767776
expect(tabToSpaceComparison, isTrue);
768777
expect(tabToTofuComparison, isFalse);

‎engine/src/flutter/testing/dart/goldens.dart‎

Lines changed: 92 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
// found in the LICENSE file.
44

55
import 'dart:io';
6+
import 'dart:math' as math;
67
import 'dart:typed_data';
78
import 'dart:ui';
89

@@ -63,22 +64,106 @@ class ImageComparer {
6364
});
6465
}
6566

66-
Future<bool> fuzzyCompareImages(Image golden, Image testImage) async {
67+
/// Compares two images pixel by pixel.
68+
///
69+
/// Currently only [ColorSpace.sRGB] images are supported.
70+
///
71+
/// If [maxColorDelta] is greater than 0.0, pixel differences whose vec3 RGB
72+
/// Euclidean distance and alpha delta are less than or equal to [maxColorDelta]
73+
/// are considered identical.
74+
///
75+
/// If [maxDifferentPixelsRate] is greater than 0.0, the comparison passes if
76+
/// the ratio of different pixels to total pixels does not exceed this value.
77+
Future<bool> fuzzyCompareImages(
78+
Image golden,
79+
Image testImage, {
80+
double maxColorDelta = 0.0,
81+
double maxDifferentPixelsRate = 0.0,
82+
}) async {
83+
assert(
84+
golden.colorSpace == ColorSpace.sRGB && testImage.colorSpace == ColorSpace.sRGB,
85+
'fuzzyCompareImages currently only supports sRGB images.',
86+
);
6787
if (golden.width != testImage.width || golden.height != testImage.height) {
6888
return false;
6989
}
70-
int getPixel(ByteData data, int x, int y) => data.getUint32((x + y * golden.width) * 4);
71-
final ByteData goldenData = (await golden.toByteData())!;
72-
final ByteData testImageData = (await testImage.toByteData())!;
73-
for (var y = 0; y < golden.height; y++) {
74-
for (var x = 0; x < golden.width; x++) {
75-
if (getPixel(goldenData, x, y) != getPixel(testImageData, x, y)) {
90+
final ByteData? goldenData = await golden.toByteData();
91+
final ByteData? testImageData = await testImage.toByteData();
92+
if (goldenData == null || testImageData == null) {
93+
return false;
94+
}
95+
96+
final int totalPixels = golden.width * golden.height;
97+
98+
// When no tolerance thresholds are specified, compare full 32-bit RGBA pixel
99+
// values directly in a single pass with early exit on first mismatch.
100+
if (maxColorDelta == 0.0 && maxDifferentPixelsRate == 0.0) {
101+
final Uint32List goldenUint32 = goldenData.buffer.asUint32List(
102+
goldenData.offsetInBytes,
103+
totalPixels,
104+
);
105+
final Uint32List testUint32 = testImageData.buffer.asUint32List(
106+
testImageData.offsetInBytes,
107+
totalPixels,
108+
);
109+
for (var i = 0; i < totalPixels; i++) {
110+
if (goldenUint32[i] != testUint32[i]) {
76111
return false;
77112
}
78113
}
114+
return true;
115+
}
116+
117+
// When tolerance thresholds are configured, inspect each pixel's color
118+
// channels and count those exceeding [maxColorDelta] to determine whether
119+
// the proportion of different pixels is within [maxDifferentPixelsRate].
120+
var differentPixels = 0;
121+
var maxObservedDelta = 0.0;
122+
123+
final Uint8List goldenBytes = goldenData.buffer.asUint8List(
124+
goldenData.offsetInBytes,
125+
totalPixels * 4,
126+
);
127+
final Uint8List testBytes = testImageData.buffer.asUint8List(
128+
testImageData.offsetInBytes,
129+
totalPixels * 4,
130+
);
131+
132+
for (var i = 0; i < totalPixels; i++) {
133+
final double pixelDelta = _colorDistance(goldenBytes, testBytes, i * 4);
134+
if (pixelDelta > maxObservedDelta) {
135+
maxObservedDelta = pixelDelta;
136+
}
137+
if (pixelDelta > maxColorDelta) {
138+
differentPixels++;
139+
}
140+
}
141+
142+
final double diffRate = differentPixels / totalPixels;
143+
if (diffRate > maxDifferentPixelsRate) {
144+
print(
145+
'fuzzyCompareImages failed: '
146+
'maxObservedDelta=${maxObservedDelta.toStringAsFixed(3)} (threshold=$maxColorDelta), '
147+
'differentPixels=$differentPixels/$totalPixels '
148+
'(${(diffRate * 100).toStringAsFixed(3)}%, maxAllowed=${(maxDifferentPixelsRate * 100).toStringAsFixed(3)}%)',
149+
);
150+
return false;
79151
}
80152
return true;
81153
}
154+
155+
static double _colorDistance(Uint8List a, Uint8List b, int offset) {
156+
final int dr = a[offset] - b[offset];
157+
final int dg = a[offset + 1] - b[offset + 1];
158+
final int db = a[offset + 2] - b[offset + 2];
159+
final int da = (a[offset + 3] - b[offset + 3]).abs();
160+
final int rgbDistSquared = dr * dr + dg * dg + db * db;
161+
if (rgbDistSquared == 0 && da == 0) {
162+
return 0.0;
163+
}
164+
final double rgbDistance = math.sqrt(rgbDistSquared);
165+
return math.max(rgbDistance, da.toDouble());
166+
}
82167
}
83168

84169
// TODO(dnfield): add local comparison against baseline,

0 commit comments

Comments
 (0)