Inconsistent PNG encoding by Image.toByteData() causes false positives in golden file tests #30036
Description
Problem
I have a situation where matchesGoldenFile tests fail when golden files are created on one machine and the tests are run on another. Both are Linux machines with the same Flutter and Dart versions.
Cause
I tracked it down to this line:
This seems to produce a PNG presentation that differs bytewise on different machines even though the images are identical pixelwise.
The matchesGoldenFile tests compare the PNG images bytewise and this causes the test to fail even with pixelwise identical files.
flutter/packages/flutter_test/lib/src/goldens.dart
Lines 178 to 185 in bafe7cb
To reproduce
I created this short code that creates an image and prints its raw hash and png hash.
import 'dart:typed_data';
import 'dart:ui';
import 'package:collection/collection.dart';
void main() async {
final pictureRecorder = PictureRecorder();
Canvas(pictureRecorder).drawCircle(
const Offset(100, 100),
100,
Paint()..color = const Color.fromARGB(255, 255, 0, 0),
);
final picture = pictureRecorder.endRecording();
final image = await picture.toImage(200, 200);
final hashFunc = const ListEquality().hash;
final pngByteData = await image.toByteData(format: ImageByteFormat.png);
final pngHash = hashFunc(Uint8List.view(pngByteData.buffer));
final rawByteData = await image.toByteData(format: ImageByteFormat.rawRgba);
final rawHash = hashFunc(Uint8List.view(rawByteData.buffer));
print('png hash: $pngHash');
print('raw hash: $rawHash');
}Test runs
Machine 1:
❯ flutter test image_test.dart
00:01 +0: loading /home/antti/projects/correct_pitch/image_test.dart
png hash: 1951256462
raw hash: 1641621570
No tests ran.
Machine 2:
❯ flutter test image_test.dart
00:01 +0: loading /home/antti/projects/correct_pitch/image_test.dart
png hash: 1038304024
raw hash: 1641621570
No tests ran.
As you can see, the raw hashes are identical, i.e. the images are identical pixelwise. But the PNG hashes differ, i.e. the bytes in the PNG presentation are not consistent.
flutter doctor -v
Machine 1:
[✓] Flutter (Channel beta, v1.3.8, on Linux, locale en_US.UTF-8)
• Flutter version 1.3.8 at /home/antti/tools/flutter
• Framework revision e5b1ed7a7f (3 weeks ago), 2019-03-06 14:23:37 -0800
• Engine revision f4951df193
• Dart version 2.2.1 (build 2.2.1-dev.0.0 571ea80e11)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
• Android SDK at /opt/android-sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-28, build-tools 28.0.3
• ANDROID_HOME = /opt/android-sdk
• Java binary at: /opt/android-studio/jre/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
• All Android licenses accepted.
[✓] Android Studio (version 3.3)
• Android Studio at /opt/android-studio
• Flutter plugin version 33.3.1
• Dart plugin version 182.5215
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
[!] Connected device
! No devices available
! Doctor found issues in 1 category.
Machine 2:
[✓] Flutter (Channel beta, v1.3.8, on Linux, locale en_US.UTF-8)
• Flutter version 1.3.8 at /home/antti/tools/flutter
• Framework revision e5b1ed7a7f (3 weeks ago), 2019-03-06 14:23:37 -0800
• Engine revision f4951df193
• Dart version 2.2.1 (build 2.2.1-dev.0.0 571ea80e11)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
• Android SDK at /home/antti/Android/Sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-28, build-tools 28.0.3
• ANDROID_HOME = /home/antti/Android/Sdk
• Java binary at: /usr/bin/java
• Java version OpenJDK Runtime Environment (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4)
• All Android licenses accepted.
[!] Android Studio (not installed)
• Android Studio not found; download from https://developer.android.com/studio/index.html
(or visit https://flutter.io/setup/#android-setup for detailed instructions).
[✓] VS Code (version 1.32.3)
• VS Code at /usr/share/code
• Flutter extension version 2.24.0
[!] Connected device
! No devices available
! Doctor found issues in 2 categories.