Skip to content

Commit 8c1cad1

Browse files
[web] Support FragmentShader.getImageSampler on CanvasKit and Skwasm (flutter#195854)
<!-- Thanks for filing a pull request! Reviewers are typically assigned within a week of filing a request. To learn more about code review, see our documentation on Tree Hygiene: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md --> Implements `FragmentShader.getImageSampler(name)` on the CanvasKit and Skwasm web renderers. Before this change, both backends threw `UnsupportedError`, so web apps could bind float and matrix uniforms by name but not image samplers. The sampler index lookup now lives in `ShaderData.getImageSamplerIndex`, which both backends share. It is a port of the native `_getImageSamplerIndex`: the index counts only sampler uniforms, and the error messages are the same as native. `BackendFragmentShader` now exposes a plain `int getImageSamplerIndex(String name)` query instead of returning a slot. `EngineFragmentShader` builds an `EngineImageSamplerSlot` whose `set` calls `EngineFragmentShader.setImageSampler`, so binding through a slot gets the same sampler disposal, finalizer handling, and picture image retention as calling `setImageSampler` directly. Fixes flutter#194290 ## Behavioral Changes - `FragmentShader.getImageSampler(name)` on CanvasKit and Skwasm: - Before: always threw `UnsupportedError('getImageSampler is not supported on the web.')`. - After: returns an `ImageSamplerSlot`. Its `name` is the queried name, and its `shaderIndex` is the 0-based position of that sampler among the shader's sampler uniforms (float uniforms are not counted). `slot.set(image)` does the same thing as `shader.setImageSampler(slot.shaderIndex, image)`. - Invalid names now throw `ArgumentError` with the native messages: - a non-sampler uniform: `Uniform "<name>" is not an image sampler.` - an unknown name: `No uniform named "<name>".` - Calling `slot.set` after the shader is disposed fails the same debug assertion as `setImageSampler`. - Web Impeller (wimp) does not implement runtime effects yet (flutter#175431), so nothing changes there. ## Where to focus - **Where the slot is built:** the backend only answers a name-to-index query, and the frontend `EngineFragmentShader` builds the slot. That keeps image lifetime bookkeeping (`samplerImages`, which the picture image tracker reads, disposal of the replaced sampler, and finalizers) in one place. A slot built by the backend that called the backend's `setImageSampler` directly would skip that bookkeeping. This is also why the sampler slot is built in a different layer from the float uniform slots. - **Index semantics:** `ShaderData.getImageSamplerIndex` counts only `SampledImage` uniforms that come before the match. That matches the index space that `CkFragmentShader.setImageSampler` (`samplers[index]`) and `SkwasmFragmentShader.setImageSampler` (child shader `index`) already use. - **`CkFragmentProgram`** now keeps its `ShaderData`, as `SkwasmFragmentProgram` already did, and exposes `uniforms`, `floatCount` and `textureCount` as getters. - **Tests:** a new `two_samplers` SkSL fixture puts samplers between float uniforms. It covers index ordering and binding in reverse declaration order through real pixels. - **Files to review first:** 1. `lib/src/engine/primitives/shader.dart` 2. `lib/src/engine/shader_data.dart` 3. `test/ui/fragment_shader_test.dart` (the `getImageSampler` group) ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent 451069b commit 8c1cad1

7 files changed

Lines changed: 470 additions & 27 deletions

File tree

‎engine/src/flutter/lib/web_ui/lib/src/engine/backend/shader.dart‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,9 @@ abstract class BackendGradient extends BackendShader {
1919
}
2020

2121
/// The native backend representation of a [ui.ImageShader].
22-
abstract class BackendImageShader extends BackendShader {}
22+
abstract class BackendImageShader extends BackendShader {
23+
bool get debugDisposed;
24+
}
2325

2426
/// The native backend representation of a [ui.FragmentShader].
2527
abstract class BackendFragmentShader extends BackendShader {
@@ -34,7 +36,6 @@ abstract class BackendFragmentShader extends BackendShader {
3436
ui.UniformMat2Slot getUniformMat2(String name);
3537
ui.UniformMat3Slot getUniformMat3(String name);
3638
ui.UniformMat4Slot getUniformMat4(String name);
37-
ui.ImageSamplerSlot getImageSampler(String name);
3839
ui.UniformArray<ui.UniformFloatSlot> getUniformFloatArray(String name);
3940
ui.UniformArray<ui.UniformVec2Slot> getUniformVec2Array(String name);
4041
ui.UniformArray<ui.UniformVec3Slot> getUniformVec3Array(String name);

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/fragment_shader.dart‎

Lines changed: 7 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import 'package:ui/src/engine.dart';
1010
import 'package:ui/ui.dart' as ui;
1111

1212
class CkFragmentProgram implements ui.FragmentProgram {
13-
CkFragmentProgram(this.name, this.effect, this.uniforms, this.floatCount, this.textureCount);
13+
CkFragmentProgram(this.name, this.effect, this._shaderData);
1414

1515
factory CkFragmentProgram.fromBytes(String name, Uint8List data) {
1616
final shaderData = ShaderData.fromBytes(data);
@@ -19,24 +19,19 @@ class CkFragmentProgram implements ui.FragmentProgram {
1919
throw const FormatException('Invalid Shader Source');
2020
}
2121

22-
return CkFragmentProgram(
23-
name,
24-
effect,
25-
shaderData.uniforms,
26-
shaderData.floatCount,
27-
shaderData.textureCount,
28-
);
22+
return CkFragmentProgram(name, effect, shaderData);
2923
}
3024

3125
final String name;
3226
final SkRuntimeEffect effect;
33-
final List<UniformData> uniforms;
34-
final int floatCount;
35-
final int textureCount;
27+
final ShaderData _shaderData;
28+
List<UniformData> get uniforms => _shaderData.uniforms;
29+
int get floatCount => _shaderData.floatCount;
30+
int get textureCount => _shaderData.textureCount;
3631

3732
@override
3833
ui.FragmentShader fragmentShader() {
39-
return EngineFragmentShader(CkFragmentShader(name, effect, this));
34+
return EngineFragmentShader(CkFragmentShader(name, effect, this), _shaderData);
4035
}
4136

4237
UniformData _getUniformFloatInfo(String name) {
@@ -317,11 +312,6 @@ class CkFragmentShader extends BackendFragmentShader implements CkShader {
317312
);
318313
}
319314

320-
@override
321-
ui.ImageSamplerSlot getImageSampler(String name) {
322-
throw UnsupportedError('getImageSampler is not supported on the web.');
323-
}
324-
325315
List<CkUniformFloatSlot> _getUniformFloatSlots(String name, int size) {
326316
final UniformData info = _program._getUniformFloatInfo(name);
327317

‎engine/src/flutter/lib/web_ui/lib/src/engine/canvaskit/shader.dart‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,7 @@ class CkImageShader extends BackendImageShader implements CkShader {
174174

175175
bool _isDisposed = false;
176176

177+
@override
177178
bool get debugDisposed {
178179
bool? result;
179180
assert(() {

‎engine/src/flutter/lib/web_ui/lib/src/engine/primitives/shader.dart‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -326,11 +326,17 @@ class EngineImageShader extends EngineShader implements ui.ImageShader {
326326
}
327327

328328
class EngineFragmentShader extends EngineShader implements ui.FragmentShader {
329-
EngineFragmentShader(BackendFragmentShader delegate) {
329+
EngineFragmentShader(BackendFragmentShader delegate, this._shaderData) {
330330
_backendShader = _cacheAndAttach(delegate) as BackendFragmentShader;
331331
}
332332

333333
late final BackendFragmentShader _backendShader;
334+
335+
/// The uniform metadata of the program this shader was created from.
336+
///
337+
/// Lookups that only depend on this metadata, such as resolving an image
338+
/// sampler name to its index, live here so they are shared by all backends.
339+
final ShaderData _shaderData;
334340
final Map<int, BackendImageShader> _imageSamplers = <int, BackendImageShader>{};
335341
final Map<int, ui.Image> _samplerImages = <int, ui.Image>{};
336342

@@ -408,7 +414,8 @@ class EngineFragmentShader extends EngineShader implements ui.FragmentShader {
408414
@override
409415
ui.UniformMat4Slot getUniformMat4(String name) => _backendShader.getUniformMat4(name);
410416
@override
411-
ui.ImageSamplerSlot getImageSampler(String name) => _backendShader.getImageSampler(name);
417+
ui.ImageSamplerSlot getImageSampler(String name) =>
418+
EngineImageSamplerSlot._(this, name, _shaderData.getImageSamplerIndex(name));
412419
@override
413420
ui.UniformArray<ui.UniformFloatSlot> getUniformFloatArray(String name) =>
414421
_backendShader.getUniformFloatArray(name);
@@ -431,3 +438,29 @@ class EngineFragmentShader extends EngineShader implements ui.FragmentShader {
431438
ui.UniformArray<ui.UniformMat4Slot> getUniformMat4Array(String name) =>
432439
_backendShader.getUniformMat4Array(name);
433440
}
441+
442+
/// A binding to an image sampler of an [EngineFragmentShader].
443+
///
444+
/// Calling [set] updates the image bound to the sampler.
445+
///
446+
/// Unlike the uniform float slots, which write directly into backend uniform
447+
/// storage, this slot is created by the frontend so that [set] goes through
448+
/// [EngineFragmentShader.setImageSampler], which owns the lifetime of the
449+
/// backend image shaders and keeps the bound image alive for recorded
450+
/// pictures.
451+
class EngineImageSamplerSlot implements ui.ImageSamplerSlot {
452+
EngineImageSamplerSlot._(this._shader, this.name, this.shaderIndex);
453+
454+
final EngineFragmentShader _shader;
455+
456+
@override
457+
final String name;
458+
459+
@override
460+
final int shaderIndex;
461+
462+
@override
463+
void set(ui.Image val) {
464+
_shader.setImageSampler(shaderIndex, val);
465+
}
466+
}

‎engine/src/flutter/lib/web_ui/lib/src/engine/shader_data.dart‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,28 @@ class ShaderData {
9595
List<UniformData> uniforms;
9696
int floatCount;
9797
int textureCount;
98+
99+
/// Returns the index of the image sampler uniform named [name] among the
100+
/// shader's image sampler uniforms, as passed to
101+
/// `FragmentShader.setImageSampler`.
102+
///
103+
/// Throws an [ArgumentError] if there is no uniform named [name], or if that
104+
/// uniform is not an image sampler.
105+
int getImageSamplerIndex(String name) {
106+
var index = 0;
107+
for (final UniformData uniform in uniforms) {
108+
if (uniform.name == name) {
109+
if (uniform.type != UniformType.SampledImage) {
110+
throw ArgumentError('Uniform "$name" is not an image sampler.');
111+
}
112+
return index;
113+
}
114+
if (uniform.type == UniformType.SampledImage) {
115+
index += 1;
116+
}
117+
}
118+
throw ArgumentError('No uniform named "$name".');
119+
}
98120
}
99121

100122
class UniformData {

‎engine/src/flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/shaders.dart‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,7 @@ class SkwasmImageShader extends BackendImageShader implements SkwasmShader {
195195
}
196196
}
197197

198+
@override
198199
bool get debugDisposed => _handle == null;
199200
}
200201

@@ -226,7 +227,8 @@ class SkwasmFragmentProgram extends SkwasmObjectWrapper<RawRuntimeEffect>
226227
final ShaderData _shaderData;
227228

228229
@override
229-
ui.FragmentShader fragmentShader() => EngineFragmentShader(SkwasmFragmentShader(this));
230+
ui.FragmentShader fragmentShader() =>
231+
EngineFragmentShader(SkwasmFragmentShader(this), _shaderData);
230232

231233
int get uniformSize => runtimeEffectGetUniformSize(handle);
232234

@@ -474,11 +476,6 @@ class SkwasmFragmentShader extends BackendFragmentShader implements SkwasmShader
474476
);
475477
}
476478

477-
@override
478-
ui.ImageSamplerSlot getImageSampler(String name) {
479-
throw UnsupportedError('getImageSampler is not supported on the web.');
480-
}
481-
482479
List<SkwasmUniformFloatSlot> _getUniformFloatSlots(String name, int size) {
483480
final UniformData info = _program._getUniformFloatInfo(name);
484481

0 commit comments

Comments
 (0)