-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Description
Blocks #156488.
In #156488, we wanted to simplify how ImageReaderSurfaceProducer works, and, for external texture plugins, stop listening to TRIM_MEMORY Android events as a determination to invalidate the surface (and rebuild). A blocker to the approach is some incompatible logic in how the C++ engine works with ImageReader.
In a bit more detail, the following stanza conditionally draws the texture image if available:
flutter/engine/src/flutter/shell/platform/android/image_external_texture.cc
Lines 39 to 50 in d6a6e95
| if (dl_image_) { | |
| context.canvas->DrawImageRect( | |
| dl_image_, // image | |
| SkRect::Make(dl_image_->bounds()), // source rect | |
| bounds, // destination rect | |
| sampling, // sampling | |
| context.paint, // paint | |
| flutter::DlSrcRectConstraint::kStrict // enforce edges | |
| ); | |
| } else { | |
| FML_LOG(INFO) << "No DlImage available for ImageExternalTexture to paint."; | |
| } |
However, in practice, we see that, if the following occurs:
- An
ImageReaderSurfaceProduceris created and drawn to (statically, i.e. not streaming an image from another source) - You background the app, triggering
OnGrContextDestroyed - You resume the app, we no longer have a
dl_image_, but nothing will trigger a new image to become available
... we (at HEAD), hit No DlImage available for ImageExternalTexture to paint.
This can be observed in android_engine_test with the following patch:
# engine/src/flutter/shell/platform/android/io/flutter/embedding/engine/renderer/FlutterRenderer.java#L447
- private static final boolean CLEANUP_ON_MEMORY_PRESSURE = true;
+ private static final boolean CLEANUP_ON_MEMORY_PRESSURE = false;... and then run an integration test:
cd dev/integration_tests/android_engine_test
et run lib/external_texture/surface_producer_smiley_face_main.dart -c android_debug_unopt_arm64... and using the home button on a device (or emulator), switching to another app, and switching back.
A "fix" (that is not proper) is the following:
diff --git a/engine/src/flutter/shell/platform/android/image_external_texture.cc b/engine/src/flutter/shell/platform/android/image_external_texture.cc
index ed9e7b3c7f9..60bc7d20c9c 100644
--- a/engine/src/flutter/shell/platform/android/image_external_texture.cc
+++ b/engine/src/flutter/shell/platform/android/image_external_texture.cc
@@ -66,7 +66,7 @@ void ImageExternalTexture::OnGrContextCreated() {
// Implementing flutter::ContextListener.
void ImageExternalTexture::OnGrContextDestroyed() {
if (state_ == AttachmentState::kAttached) {
- dl_image_.reset();
+ // dl_image_.reset();
image_lru_.Clear();
Detach();
}What we should do is instead, is if we fail to acquire the latest image, check the previous image instead:
flutter/engine/src/flutter/shell/platform/android/image_external_texture_vk_impeller.cc
Lines 40 to 43 in d6a6e95
| JavaLocalRef image = AcquireLatestImage(); | |
| if (image.is_null()) { | |
| return; | |
| } |
flutter/engine/src/flutter/shell/platform/android/image_external_texture.cc
Lines 76 to 85 in d6a6e95
| JavaLocalRef ImageExternalTexture::AcquireLatestImage() { | |
| JNIEnv* env = fml::jni::AttachCurrentThread(); | |
| FML_CHECK(env != nullptr); | |
| // ImageTextureEntry.acquireLatestImage. | |
| JavaLocalRef image_java = | |
| jni_facade_->ImageProducerTextureEntryAcquireLatestImage( | |
| JavaLocalRef(image_texture_entry_)); | |
| return image_java; | |
| } |
Ideally this is done where it impacts all subclasses of ImageExternalTexture (this is not specific to Vulkan).