Repository navigation
Commit 4db99a7
authored
[Embedder] Support render texture for vulkan (flutter#188855)
<!--
Thanks for filing a pull request!
Reviewers are typically assigned within a week of filing a request.
To learn more about code review, see our documentation on Tree Hygiene:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
-->
This PR adds external texture support for the Vulkan embedder API.
Embedders using Flutter in Vulkan mode can now register a texture frame
callback and have the engine directly sample embedder-owned `VkImage`s
during composition.
Both the Impeller and Skia Vulkan backends are supported, including
RGBA/BGRA formats and multi-planar YUV formats (e.g. NV12) via YCbCr
conversion.
## API changes (`embedder.h`)
- Added `FlutterVulkanExternalTexture`, describing an embedder-owned
`VkImage` handed to the engine:
- `image`: handle to the `VkImage`, which must be in the
`VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL` layout when provided to the
engine.
- `format`: the `VkFormat` of the image (for example
`VK_FORMAT_R8G8B8A8_UNORM`).
- Optional `width`/`height` (non-zero specifies the texture size).
- Optional `destruction_callback` + `user_data`, invoked on an
engine-managed thread when the texture can be collected.
- Added `FlutterVulkanTextureFrameCallback`, invoked by the engine when
a texture marked with a new frame available needs to be resolved. The
embedder must perform a host sync before returning so the engine can
sample the `VkImage` without additional synchronization.
- Added `external_texture_frame_callback` to
`FlutterVulkanRendererConfig`.
## Engine changes
- Added `EmbedderExternalTextureVulkan`
(`embedder_external_texture_vulkan.{h,cc}`), a `flutter::Texture`
implementation that resolves embedder textures on both backends:
- Impeller: `EmbedderExternalTextureSourceVulkan` implements
`impeller::TextureSourceVK`, creating the image view and a
`YUVConversionVK` when the format requires YCbCr conversion.
- Skia: wraps the `VkImage` into a Skia image, dynamically selecting the
`SkColorType` based on the `VkFormat`.
- Extended `EmbedderExternalTextureResolver` to create Vulkan external
textures.
- Wired the new callback through `embedder.cc` and
`embedder_surface_vulkan.cc`.
## Tests
- Added unit tests in `embedder_vk_unittests.cc` covering RGBA, BGRA and
NV12 textures on both Impeller and Skia, plus destruction-callback
variants (`RenderTextureWith*Vulkan`, `Render*TextureWith*Vulkan`,
`RenderTextureWith*VulkanDestructCallback`).
- Added NV12 fixtures (`texture.nv12`, `external_texture_nv12.png`).
- Extended `TestVulkanContext` and `EmbedderTestContextVulkan` to
support external texture rendering and sampler YCbCr conversion.
Addresses: flutter#117937
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.
**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md1 parent b56fbef commit 4db99a7
15 files changed
Lines changed: 1507 additions & 19 deletions
File tree
- engine/src/flutter
- shell/platform/embedder
- fixtures
- tests
- testing
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
167 | 167 | | |
168 | 168 | | |
169 | 169 | | |
| 170 | + | |
| 171 | + | |
170 | 172 | | |
171 | 173 | | |
172 | 174 | | |
| |||
176 | 178 | | |
177 | 179 | | |
178 | 180 | | |
| 181 | + | |
179 | 182 | | |
180 | 183 | | |
181 | 184 | | |
| |||
257 | 260 | | |
258 | 261 | | |
259 | 262 | | |
| 263 | + | |
| 264 | + | |
260 | 265 | | |
261 | 266 | | |
262 | 267 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2385 | 2385 | | |
2386 | 2386 | | |
2387 | 2387 | | |
| 2388 | + | |
| 2389 | + | |
| 2390 | + | |
| 2391 | + | |
| 2392 | + | |
| 2393 | + | |
| 2394 | + | |
| 2395 | + | |
| 2396 | + | |
| 2397 | + | |
| 2398 | + | |
| 2399 | + | |
| 2400 | + | |
| 2401 | + | |
| 2402 | + | |
| 2403 | + | |
| 2404 | + | |
| 2405 | + | |
| 2406 | + | |
| 2407 | + | |
| 2408 | + | |
| 2409 | + | |
| 2410 | + | |
2388 | 2411 | | |
2389 | 2412 | | |
2390 | 2413 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
968 | 968 | | |
969 | 969 | | |
970 | 970 | | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
| 1002 | + | |
971 | 1003 | | |
972 | 1004 | | |
973 | 1005 | | |
| |||
1031 | 1063 | | |
1032 | 1064 | | |
1033 | 1065 | | |
1034 | | - | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
| 1070 | + | |
| 1071 | + | |
| 1072 | + | |
1035 | 1073 | | |
1036 | 1074 | | |
1037 | 1075 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
24 | 30 | | |
25 | 31 | | |
26 | 32 | | |
| |||
37 | 43 | | |
38 | 44 | | |
39 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
40 | 53 | | |
41 | 54 | | |
42 | 55 | | |
| |||
53 | 66 | | |
54 | 67 | | |
55 | 68 | | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
56 | 75 | | |
57 | 76 | | |
58 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
20 | 24 | | |
21 | 25 | | |
22 | 26 | | |
| |||
34 | 38 | | |
35 | 39 | | |
36 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
37 | 46 | | |
38 | 47 | | |
39 | 48 | | |
| |||
47 | 56 | | |
48 | 57 | | |
49 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
50 | 62 | | |
51 | 63 | | |
52 | 64 | | |
| |||
0 commit comments