Skip to content

ImageExternalTexture should fallback to previous image if next image is unavailable #163561

@matanlurey

Description

@matanlurey

Blocks #156488.


In #156488, we wanted to simplify how ImageReaderSurfaceProducer works, and, for external texture plugins, stop listening to TRIM_MEMORY Android events as a determination to invalidate the surface (and rebuild). A blocker to the approach is some incompatible logic in how the C++ engine works with ImageReader.

In a bit more detail, the following stanza conditionally draws the texture image if available:

if (dl_image_) {
context.canvas->DrawImageRect(
dl_image_, // image
SkRect::Make(dl_image_->bounds()), // source rect
bounds, // destination rect
sampling, // sampling
context.paint, // paint
flutter::DlSrcRectConstraint::kStrict // enforce edges
);
} else {
FML_LOG(INFO) << "No DlImage available for ImageExternalTexture to paint.";
}

However, in practice, we see that, if the following occurs:

  1. An ImageReaderSurfaceProducer is created and drawn to (statically, i.e. not streaming an image from another source)
  2. You background the app, triggering OnGrContextDestroyed
  3. You resume the app, we no longer have a dl_image_, but nothing will trigger a new image to become available

... we (at HEAD), hit No DlImage available for ImageExternalTexture to paint.

This can be observed in android_engine_test with the following patch:

# engine/src/flutter/shell/platform/android/io/flutter/embedding/engine/renderer/FlutterRenderer.java#L447
-  private static final boolean CLEANUP_ON_MEMORY_PRESSURE = true;
+  private static final boolean CLEANUP_ON_MEMORY_PRESSURE = false;

... and then run an integration test:

cd dev/integration_tests/android_engine_test
et run lib/external_texture/surface_producer_smiley_face_main.dart -c android_debug_unopt_arm64

... and using the home button on a device (or emulator), switching to another app, and switching back.

A "fix" (that is not proper) is the following:

diff --git a/engine/src/flutter/shell/platform/android/image_external_texture.cc b/engine/src/flutter/shell/platform/android/image_external_texture.cc
index ed9e7b3c7f9..60bc7d20c9c 100644
--- a/engine/src/flutter/shell/platform/android/image_external_texture.cc
+++ b/engine/src/flutter/shell/platform/android/image_external_texture.cc
@@ -66,7 +66,7 @@ void ImageExternalTexture::OnGrContextCreated() {
 // Implementing flutter::ContextListener.
 void ImageExternalTexture::OnGrContextDestroyed() {
   if (state_ == AttachmentState::kAttached) {
-    dl_image_.reset();
+    // dl_image_.reset();
     image_lru_.Clear();
     Detach();
   }

What we should do is instead, is if we fail to acquire the latest image, check the previous image instead:

JavaLocalRef image = AcquireLatestImage();
if (image.is_null()) {
return;
}

JavaLocalRef ImageExternalTexture::AcquireLatestImage() {
JNIEnv* env = fml::jni::AttachCurrentThread();
FML_CHECK(env != nullptr);
// ImageTextureEntry.acquireLatestImage.
JavaLocalRef image_java =
jni_facade_->ImageProducerTextureEntryAcquireLatestImage(
JavaLocalRef(image_texture_entry_));
return image_java;
}

Ideally this is done where it impacts all subclasses of ImageExternalTexture (this is not specific to Vulkan).

Metadata

Metadata

Assignees

Labels

P1High-priority issues at the top of the work listc: renderingUI glitches reported at the engine/skia or impeller rendering levelplatform-androidAndroid applications specificallyteam-androidOwned by Android platform team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions