Skip to content

Commit cbd8912

Browse files
Revert "Reverts "[reland] Unify canvas creation and Surface code in S…kwasm and CanvasKit (#179473)" (#180152)" (#180610)
This PR introduces a significant refactoring of the web engine's rendering layer by unifying the Surface and Rasterizer implementations. These components have been moved from being renderer-specific to a generic compositing directory, making the architecture more modular and easier to maintain. The rasterizers are now renderer-agnostic and are provided with renderer-specific surface factories via dependency injection. A new CanvasProvider abstraction has also been introduced to manage the lifecycle of the underlying canvas elements. A key outcome of this work is that the Skwasm backend now correctly handles WebGL context loss events. This was achieved by refactoring SkwasmSurface to allow the Dart side to manage the OffscreenCanvas lifecycle. A communication channel between the main thread and the web worker is now used to gracefully handle context loss and recovery. This effort also included fixing several related bugs around surface sizing, resource cleanup, and callback handling in multi-surface scenarios. To validate these changes, new testing APIs have been added to allow for the creation of renderer-agnostic surface tests. A new test file, surface_context_lost_test.dart, has been added to verify the context loss and recovery behavior across all supported renderers, ensuring the new architecture is robust and reliable. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent 4064451 commit cbd8912

38 files changed

Lines changed: 1704 additions & 1363 deletions

‎engine/src/flutter/lib/web_ui/lib/src/engine.dart‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,7 @@ export 'engine/canvaskit/image_filter.dart';
2828
export 'engine/canvaskit/image_wasm_codecs.dart';
2929
export 'engine/canvaskit/image_web_codecs.dart';
3030
export 'engine/canvaskit/mask_filter.dart';
31-
export 'engine/canvaskit/multi_surface_rasterizer.dart';
3231
export 'engine/canvaskit/native_memory.dart';
33-
export 'engine/canvaskit/offscreen_canvas_rasterizer.dart';
3432
export 'engine/canvaskit/painting.dart';
3533
export 'engine/canvaskit/path.dart';
3634
export 'engine/canvaskit/path_metrics.dart';
@@ -44,10 +42,14 @@ export 'engine/canvaskit/util.dart';
4442
export 'engine/canvaskit/vertices.dart';
4543
export 'engine/clipboard.dart';
4644
export 'engine/color_filter.dart';
45+
export 'engine/compositing/canvas_provider.dart';
4746
export 'engine/compositing/composition.dart';
4847
export 'engine/compositing/display_canvas_factory.dart';
48+
export 'engine/compositing/multi_surface_rasterizer.dart';
49+
export 'engine/compositing/offscreen_canvas_rasterizer.dart';
4950
export 'engine/compositing/rasterizer.dart';
5051
export 'engine/compositing/render_canvas.dart';
52+
export 'engine/compositing/surface.dart';
5153
export 'engine/configuration.dart';
5254
export 'engine/display.dart';
5355
export 'engine/dom.dart';

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/display_canvas_factory.dart‎

Lines changed: 0 additions & 131 deletions
This file was deleted.

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/image.dart‎

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -500,7 +500,7 @@ class CkImage implements ui.Image, StackTraceDebugger {
500500
}
501501

502502
@override
503-
Future<ByteData> toByteData({ui.ImageByteFormat format = ui.ImageByteFormat.rawRgba}) {
503+
Future<ByteData> toByteData({ui.ImageByteFormat format = ui.ImageByteFormat.rawRgba}) async {
504504
assert(_debugCheckIsNotDisposed());
505505
switch (imageSource) {
506506
case ImageElementImageSource():
@@ -531,11 +531,7 @@ class CkImage implements ui.Image, StackTraceDebugger {
531531
}
532532
ByteData? data = _readPixelsFromSkImage(format);
533533
data ??= _readPixelsFromImageViaSurface(format);
534-
if (data == null) {
535-
return Future<ByteData>.error('Failed to encode the image into bytes.');
536-
} else {
537-
return Future<ByteData>.value(data);
538-
}
534+
return data;
539535
}
540536

541537
@override
@@ -555,13 +551,16 @@ class CkImage implements ui.Image, StackTraceDebugger {
555551
return data;
556552
}
557553

558-
ByteData? _readPixelsFromImageViaSurface(ui.ImageByteFormat format) {
559-
final Surface surface = CanvasKitRenderer.instance.pictureToImageSurface;
560-
final CkSurface ckSurface = surface.createOrUpdateSurface(BitmapSize(width, height));
561-
final CkCanvas ckCanvas = ckSurface.getCanvas();
554+
ByteData _readPixelsFromImageViaSurface(ui.ImageByteFormat format) {
555+
final CkSurface surface = CanvasKitRenderer.instance.pictureToImageSurface;
556+
surface.setSize(BitmapSize(width, height));
557+
final SkSurface skiaSurface = surface.skSurface!;
558+
559+
final ckCanvas = CkCanvas.fromSkCanvas(skiaSurface.getCanvas());
562560
ckCanvas.clear(const ui.Color(0x00000000));
563561
ckCanvas.drawImage(this, ui.Offset.zero, CkPaint());
564-
final SkImage skImage = ckSurface.surface.makeImageSnapshot();
562+
final SkImage skImage = skiaSurface.makeImageSnapshot();
563+
565564
final imageInfo = SkImageInfo(
566565
alphaType: canvasKit.AlphaType.Premul,
567566
colorType: canvasKit.ColorType.RGBA_8888,
@@ -570,6 +569,8 @@ class CkImage implements ui.Image, StackTraceDebugger {
570569
height: height.toDouble(),
571570
);
572571
final Uint8List? pixels = skImage.readPixels(0, 0, imageInfo);
572+
skImage.delete();
573+
573574
if (pixels == null) {
574575
throw StateError('Unable to convert read pixels from SkImage.');
575576
}

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/multi_surface_rasterizer.dart‎

Lines changed: 0 additions & 84 deletions
This file was deleted.

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/picture.dart‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -113,12 +113,18 @@ class CkPicture implements LayerPicture, StackTraceDebugger {
113113
}) {
114114
assert(debugCheckNotDisposed('Cannot convert picture to image.'));
115115

116-
final Surface surface = CanvasKitRenderer.instance.pictureToImageSurface;
117-
final CkSurface ckSurface = surface.createOrUpdateSurface(BitmapSize(width, height));
118-
final CkCanvas ckCanvas = ckSurface.getCanvas();
116+
final CkSurface surface = CanvasKitRenderer.instance.pictureToImageSurface;
117+
surface.setSize(BitmapSize(width, height));
118+
final SkSurface skiaSurface = surface.skSurface!;
119+
120+
final ckCanvas = CkCanvas.fromSkCanvas(skiaSurface.getCanvas());
119121
ckCanvas.clear(const ui.Color(0x00000000));
120122
ckCanvas.drawPicture(this);
121-
final SkImage skImage = ckSurface.surface.makeImageSnapshot();
123+
final SkImage skImage = skiaSurface.makeImageSnapshot();
124+
125+
// TODO(hterkelsen): This is a hack to get the pixels from the SkImage.
126+
// We should be able to do this without creating a new image. This is
127+
// a workaround for a bug in CanvasKit.
122128
final imageInfo = SkImageInfo(
123129
alphaType: canvasKit.AlphaType.Premul,
124130
colorType: canvasKit.ColorType.RGBA_8888,
@@ -127,8 +133,9 @@ class CkPicture implements LayerPicture, StackTraceDebugger {
127133
height: height.toDouble(),
128134
);
129135
final Uint8List? pixels = skImage.readPixels(0, 0, imageInfo);
136+
skImage.delete();
130137
if (pixels == null) {
131-
throw StateError('Unable to read pixels from SkImage.');
138+
throw StateError('Unable to convert read pixels from SkImage.');
132139
}
133140
final SkImage? rasterImage = canvasKit.MakeImage(imageInfo, pixels, (4 * width).toDouble());
134141
if (rasterImage == null) {
@@ -153,4 +160,7 @@ class CkPicture implements LayerPicture, StackTraceDebugger {
153160

154161
@override
155162
StackTrace get debugStackTrace => _debugStackTrace;
163+
164+
@override
165+
bool get isDisposed => _isDisposed;
156166
}

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/renderer.dart‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,20 +32,21 @@ class CanvasKitRenderer extends Renderer {
3232

3333
static Rasterizer _createRasterizer() {
3434
if (configuration.canvasKitForceMultiSurfaceRasterizer || isSafari || isFirefox) {
35-
return MultiSurfaceRasterizer();
35+
return MultiSurfaceRasterizer(
36+
(OnscreenCanvasProvider canvasProvider) => CkOnscreenSurface(canvasProvider),
37+
);
3638
}
37-
return OffscreenCanvasRasterizer();
39+
return OffscreenCanvasRasterizer(
40+
(OffscreenCanvasProvider canvasProvider) => CkOffscreenSurface(canvasProvider),
41+
);
3842
}
3943

4044
@override
4145
void debugResetRasterizer() {
4246
rasterizer = _createRasterizer();
47+
_pictureToImageSurface = rasterizer.createPictureToImageSurface() as CkSurface;
4348
}
4449

45-
/// A surface used specifically for `Picture.toImage` when software rendering
46-
/// is supported.
47-
final Surface pictureToImageSurface = Surface();
48-
4950
@override
5051
Future<void> initialize() async {
5152
_initialized ??= () async {
@@ -59,6 +60,7 @@ class CanvasKitRenderer extends Renderer {
5960
windowFlutterCanvasKit = canvasKit;
6061
}
6162
rasterizer = _createRasterizer();
63+
_pictureToImageSurface = rasterizer.createPictureToImageSurface() as CkSurface;
6264
_instance = this;
6365
await super.initialize();
6466
}();
@@ -499,4 +501,9 @@ class CanvasKitRenderer extends Renderer {
499501
}
500502
}
501503
}
504+
505+
late CkSurface _pictureToImageSurface;
506+
507+
@override
508+
CkSurface get pictureToImageSurface => _pictureToImageSurface;
502509
}

0 commit comments

Comments
 (0)