toImageSync throws away *part of* the oldest content, both violating its API semantics and seems to violate rasterization #122566
Description
Hi thanks for the great Flutter framework! I am having this bug in production environment, and it makes the app very buggy.
Explain the minimal reproducible sample a little bit: In each frame, the code is going to:
- simply repaint a ui.Image (the cache)
- and then paint a colorful dot
- and then use toImageSync to make the whole thing into the cache (to be used later)
Therefore, surely we expect there are more and more dots in the screen as frame goes by, and no dots every disappear. However, the bug shows that, some oldest dots do disappear!
Motivation (why it is in my production environment): This is like the generalization of SnapshotWidget, because in my app, I have some quite complex but incremental painting logic, so I use this way to avoid repaint the whole thing each time.
What's even more weird is that, we know toImageSync rasterizes everything into bitmap. And here, we are seeings dots painted in (say) 20 frames ago gets disappeared. This means toImageSync's bitmap has magic to understand something is painted 20 frames ago...
Given that this is related to toImageSync, maybe @dnfield @jonahwilliams for faster triage (not sure whether this is allowed, if not just tell me!)
EDIT: For future readers, some discussions are also in Flutter's official discord https://discord.com/channels/608014603317936148/608021010377080866/1084987180822302740
Steps to Reproduce
- run it, I use an Android real device (so not sure whether other device reproduces)
- tap the "refresh xN", so that the screen is populated with many colorful dots
- turn off your android screen
- turn on your android screen again (and unlock your phone if needed)
Expected results:
The UI should be exactly the same as the UI before turning off screen.
Actual results:
Screenshot 1: before screen turn off
Screenshot 2: after screen turn on. Note that below screenshot is just a sample. If you try it many times, you will see the number of disappeared dots vary a lot.
As you can see, some dots disappear!
Code sample
import 'dart:math';
import 'dart:ui' as ui;
import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';
import 'package:flutter/scheduler.dart';
ui.Image? _cache;
var hackIndex = 0;
Future<void> main() async => runApp(MyApp());
class MyApp extends StatefulWidget {
State<MyApp> createState() => _MyAppState();
}
class _MyAppState extends State<MyApp> {
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
appBar: AppBar(),
body: Column(
children: [
GestureDetector(
onTap: () async {
for (var i = 0; i < 100; ++i) {
setState(() {});
await SchedulerBinding.instance.endOfFrame;
}
},
child: const Text('refresh xN'),
),
Container(
width: double.infinity,
height: 400,
decoration: BoxDecoration(border: Border.all()),
child: CustomPaint(painter: MyPainter()),
),
],
),
),
);
}
}
class MyPainter extends CustomPainter {
@override
void paint(Canvas canvas, Size size) {
final effectivePixelRatio = SchedulerBinding.instance.window.devicePixelRatio;
final newCache = paintToImageSync(
size,
pixelRatio: effectivePixelRatio,
(context, size) {
if (_cache != null) _drawCachedImage(canvas: context.canvas, pixelRatio: effectivePixelRatio, im: _cache!);
hackIndex++;
print('hi paintToImageSync hackIndex=$hackIndex');
const K = 20.0;
context.canvas.drawCircle(
const Offset(20, 20) + Offset(hackIndex % K, (hackIndex ~/ K).toDouble()) * 9,
4,
Paint()
..color = Colors.primaries[Random().nextInt(100000) % Colors.primaries.length]
..style = PaintingStyle.fill,
);
},
);
_cache?.dispose();
_cache = newCache;
_drawCachedImage(canvas: canvas, pixelRatio: effectivePixelRatio, im: _cache!);
}
@override
bool shouldRepaint(covariant CustomPainter oldDelegate) => true;
// ref [SnapshotWidget]
static void _drawCachedImage({
required Canvas canvas,
required double pixelRatio,
required ui.Image im,
}) {
const src = Rect.fromLTWH(0, 0, 600, 600);
final dst = Rect.fromLTWH(0, 0, 600 / pixelRatio, 600 / pixelRatio);
canvas.drawImageRect(im, src, dst, Paint());
}
// ref [SnapshotWidget]
ui.Image paintToImageSync(
Size size,
void Function(PaintingContext context, Size size) painter, {
required double pixelRatio,
}) {
final offsetLayer = OffsetLayer();
final context = PaintingContext(offsetLayer, Offset.zero & size);
painter(context, size);
context.stopRecordingIfNeeded(); // ignore: invalid_use_of_protected_member
final image = offsetLayer.toImageSync(Offset.zero & size, pixelRatio: pixelRatio);
offsetLayer.dispose();
return image;
}
}Logs
N/A
noproblem
flutter doctor -v
[✓] Flutter (Channel stable, 3.7.7, on macOS 12.6 21G115 darwin-x64, locale zh-Hans-CN)
• Flutter version 3.7.7 on channel stable at /Users/tom/fvm/versions/3.7.7
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 2ad6cd72c0 (5 days ago), 2023-03-08 09:41:59 -0800
• Engine revision 1837b5be5f
• Dart version 2.19.4
• DevTools version 2.20.1
• Pub download mirror https://pub.flutter-io.cn
• Flutter download mirror https://storage.flutter-io.cn
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
• Android SDK at /Users/tom/Library/Android/sdk
• Platform android-33, build-tools 33.0.0
• Java binary at: /Users/tom/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/221.6008.13.2211.9514443/Android
Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 13F100
• CocoaPods version 1.11.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2022.1)
• Android Studio at /Users/tom/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/221.6008.13.2211.9514443/Android
Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
[✓] Android Studio (version 2021.2)
• Android Studio at /Users/tom/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/212.5712.43.2112.8609683/Android
Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
[✓] IntelliJ IDEA Ultimate Edition (version 2022.3.2)
• IntelliJ at /Users/tom/Applications/JetBrains Toolbox/IntelliJ IDEA Ultimate.app
• Flutter plugin version 72.0.4
• Dart plugin version 223.8617.8
[✓] IntelliJ IDEA Ultimate Edition (version 2022.3.2)
• IntelliJ at /Users/tom/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/223.8617.56/IntelliJ IDEA.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] IntelliJ IDEA Ultimate Edition (version 2022.2)
• IntelliJ at /Users/tom/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/222.3345.118/IntelliJ IDEA.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] VS Code (version 1.76.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.60.0
[!] Proxy Configuration
• HTTP_PROXY is set
! NO_PROXY is not set
[✓] Connected device (4 available)
• M2104K10AC (mobile) • zdfueipf99t4hmpn • android-arm64 • Android 11 (API 30)
• iPhone 12 Pro Max (mobile) • A7259FE9-9C45-4BD0-8F60-ECA346EC38B5 • ios • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• macOS (desktop) • macos • darwin-x64 • macOS 12.6 21G115 darwin-x64
• Chrome (web) • chrome • web-javascript • Google Chrome 111.0.5563.64
[✓] HTTP Host Availability
• All required HTTP hosts are available
! Doctor found issues in 1 category.