Repository navigation
Description
Note
This is a quasi-placeholder issue, as I only learned what a "Surface Texture" is ~5 minutes ago, so part of the resolution will be updating this issue with actual useful information. Thanks for your patience 😄
An Android SurfaceTexture is a combination of a surface and an OpenGL ES (GLES) texture 1.
For example, this would be used to display frames from a camera plugin:
On newer devices (or is it APIs? I'm not sure), Android provides a backend-agnostic hardware buffer that can be used (directly) by Vulkan but on older devices we'll receive an OpenGL-backed surface texture, and need to adapt it for use within Vulkan - that will involve creating a new OpenGL context, and some interop.
Some possible references:
- https://stackoverflow.com/questions/56970011/how-to-pass-a-texture-between-opengl-es-and-vulkan
- https://stackoverflow.com/questions/38907764/how-to-render-to-opengl-from-vulkan?rq=3
As an initial look, I'll try package:camera in an Impeller+Vulkan demo. It might also be worth looking into Skia's codebase to see how they handle this.
/cc @jonahwilliams @johnmccutchan (feel free to edit this description if inaccurate!)
Footnotes
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fields✅ Done

Activity
jonahwilliams commentedon Oct 31, 2023
Individual APIs and packages may still provide surface textures even on devices that support hardware buffers.
The Flutter Android engine also has public APIs for registering external textures via SurfaceTexture.
johnmccutchan commentedon Oct 31, 2023
Jonah is correct, SurfaceTexture support is part of our public API and so any application is free to give us a SurfaceTexture and ask Flutter to render it as a texture layer forever regardless of Android version.
Based on research I did, a tractable path to solving this problem is to:
I've also dug into the implementation of SurfaceTextures and internally they are backed by HardwareBuffers, there is even a non-public NDK function that will retrieve the HardwareBuffer from the SurfaceTexture. I was planning on proposing that Android make the function public in a future SDK release. We would still need the above approach in the meantime.
matanlurey commentedon Oct 31, 2023
This was useful (and somewhat buried): https://docs.flutter.dev/tools/android-studio#android-ide.
matanlurey commentedon Oct 31, 2023
I have a basic app that draws a yellow texture from the Android side to render on the Dart side.
Kotlin
Dart
With the Skia backend, it works as expected:
With Impeller (Vulkan, Pixel 5):
matanlurey commentedon Nov 1, 2023
For my own self discovery, how this works from framework <-> engine is:
ui.SceneBuilderhas anvoid addTexture(int textureId, ...)method.SceneBuilderreceives it, and creates aflutter::TextureLayer.TextureLayer::Paintis what actually passes this information on to the canvas/displaylist:https://github.com/flutter/engine/blob/406f0c477afb8b57caa24bd1040af5596815c2c6/flow/layers/texture_layer.cc#L50-L69
matanlurey commentedon Nov 1, 2023
flutter-team-archive/engine#36498 added support for iOS (though I think this PR meant to say, "on iOS in Impeller"). Also a quick check, if I try on an OpenGL device (Moto G4), I get a hard crash (where on a Vulkan device, nothing renders but there is no crash):
matanlurey commentedon Nov 1, 2023
@jonahwilliams Can you clarify if the goal is to only wire up Vulkan and did you expect GLES to be working already?
jonahwilliams commentedon Nov 1, 2023
GLES is a separate issue
matanlurey commentedon Nov 1, 2023
Chatted with @jonahwilliams:
Some examples:
See
surface_texture_external_texture.handsurface_texture_external_texture_gl.h.matanlurey commentedon Nov 1, 2023
@jonahwilliams Just to clarify before I get started (I realize it will take a few iterations), my goal is to do something similar to flutter-team-archive/engine#44734 (comment), i.e. I'd end up with a
surface_texture_external_texture_vk.[cc|h], and obviously have it working within the Vulkan backend as well :)jonahwilliams commentedon Nov 1, 2023
Yeah that seems like a good place to start.
matanlurey commentedon Nov 7, 2023
I made some progress in flutter-team-archive/engine#47632 on being able to test the texture selection logic, but ran into too many hickups (#138015, for example), and found out that it doesn't appear possible to setup a (real) EGL surface in the Android emulator (or it was not trivial):
Given this isn't making any real progress towards getting surface textures working, I'm going to halt trying to have a good testing workflow (outside of manual tests) until I have something working end-to-end locally, and then explore some refactors (like #137798) to make testing the code easier.
matanlurey commentedon Dec 6, 2023
Closing this issue in favor of #139702. Thanks!
github-actions commentedon Dec 26, 2023
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.