This is the Android version of #144438.

Removing built-in image codecs (registered through Skia here) makes the engine fall back to trying to use Android's platform decoders for generating/decoding images. Our Android-platform-based image generator, AndroidImageGenerator, is registered in AndroidShellHolder here. It is registered with a priority of -1 so that it is used as a fallback option after attempting to use the other image generators in ImageGeneratorRegisty, including the built-in Skia codecs mentioned earlier.

We should consider removing the built-in codecs for image types which are supported by the fallback Android platform decoder. Removing these codecs would save on binary size.

Android supports decoding GIF, JPEG, PNG, WebP, and HEIF (source). So we can consider removing the built-in codecs for these image types.

HEIF

Flutter does not use a built-in codec for HEIF/HEIC files. These images are already using the Android platform fallback. There is no work needed for removing built-in codecs for HEIF.

PNG, WebP, and GIF

These image types may contain animation. Our existing AndroidImageGenerator implementation does not support animated images. It always reports a frame count of 1, and it ignores the input frame_index in its GetPixels method.

The built-in Skia decoders do support animation for these image formats (except for PNG - see next paragraph). So to remove these built-in codecs and preserve existing functionality, we need to update our AndroidImageGenerator to support animated images. This may be feasible, but is more complicated than simply removing the built-in codec and relying on our existing fallback behavior.

The built-in Skia PNG decoder that we use does not support animated PNGs. We implemented our own APNGImageGenerator which wraps the built-in Skia PNG decoder to support animation (#37247). If we want to make AndroidImageGenerator support animation, this may be a helpful example to reference.

JPEG

This leaves jpeg as the lowest hanging fruit for removing built-in codecs. I investigated what it would take to remove the built-in jpeg codec.

AndroidImageGenerator works by making a JNI call to FlutterJNI's decodeImage method. Currently this method only works for Android API level >= 28. It relies on Android's ImageDecoder API (used here). ImageDecoder requires API level 28+.

API levels 24-27

We need to support Android 24+. So we need a decoding method which doesn't depend on ImageDecoder. This can be implemented with a BitmapFactory.

public static Bitmap decodeImage(@NonNull ByteBuffer buffer, long imageGeneratorAddress) {
    if (Build.VERSION.SDK_INT >= API_LEVELS.API_28) {
      // ... existing code
+    } else {
+      // Decode just the bounds for nativeImageHeaderCallback.
+      BitmapFactory.Options opts = new BitmapFactory.Options();
+      opts.inJustDecodeBounds = true;
+      byte[] bytes = new byte[buffer.limit()];
+      buffer.get(bytes);
+      BitmapFactory.decodeByteArray(bytes, 0, bytes.length, opts);
+      nativeImageHeaderCallback(imageGeneratorAddress, opts.outWidth, opts.outHeight);
+
+      // Decode and return the full image.
+      opts.inJustDecodeBounds = false;
+      opts.inSampleSize = 2;
+      return BitmapFactory.decodeByteArray(bytes, 0, bytes.length, opts);
     }
-    return null;
}

OOM on API levels 24-25 for high resolution images

From testing this implementation, it works most of the time. However, for API levels 24 and 25, I sometimes get an OOM and app crash when trying to decode high resolution images. Using a 2770x4155 image resulted in the error:

E/flutter ( 4637): [ERROR:flutter/fml/platform/android/jni_util.cc(206)] java.lang.OutOfMemoryError: Failed to allocate a 46037412 byte allocation with 4194304 free bytes and 41MB until OOM

It is trying to allocate 46037412 bytes, which is 4 bytes per pixel for the 2770x4155 image. This causes an issue specifically for API levels 25 and below. These versions store bitmap data on the Dalvik heap. Starting with API level 26, bitmap data is stored on the native heap. (source) The Dalvik heap is much more limited than the native heap, so this causes an OOM when trying to allocate large bitmaps for API levels 25 and below.

The recommended approach for loading large bitmaps is to load a smaller scaled down version based on the size of the target UI component where the image will be displayed. The Flutter engine does not currently use information target sizing when decoding images. A potential solution may be to change the engine so that it takes target image size into account when decoding images. I do not have enough context to know what is required to do this. But I expect it would be complicated, if it is even feasible. Implementing this may be further complicated by needing to support interactions like zooming in on an image or displaying different sub-regions of a large image.

Crashing when decoding high resolution images on API levels 24-25 is a blocker. We cannot remove any of our built-in codecs until we solve this issue. The solution may be taking in target image size as described in the previous paragraph. Or it may be to simply wait until Flutter's minimum supported Android API level is bumped to 26+.

JPEG performance testing

Putting aside the blocker of OOMs on API levels 24-25, I did some performance testing for using the Android platform codecs compared to our current built-in codec for JPEGs.

For my performance testing, I used the following test app:

import 'package:flutter/material.dart';

const assets = [
  '',
  'assets/jpg.jpg',
  'assets/sample-red-200x200.jpg',
  'assets/anotherjpg.jpg',
  'assets/sample-clouds-400x300.jpg',
  'assets/JPEG_example_flower.jpg',
  'assets/photo-1730407263774-d879b24d393a.jpg',
  'assets/sample-birch-400x300.jpg',
];

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(home: MyHomePage());
  }
}

class MyHomePage extends StatefulWidget {
  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      _counter++;
      _counter %= assets.length;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Column(
          mainAxisAlignment: .center,
          children: _buildContents(_counter),
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        child: const Icon(Icons.add),
      ),
    );
  }
}

List<Widget> _buildContents(int i) {
  if (i == 0) return [Text('Press button for image')];

  return [
    Image.asset(assets[i]),
    Text('Image $i of ${assets.length - 1}'),
    Text(assets[i]),
  ];
}

With the following images:

The app shows each of the 7 images in sequence as the FAB is tapped.

I measured performance by logging the timing in three calls made in image_descriptor.cc:

registry->CreateCompatibleGenerator(immutable_buffer->data())

--- a/engine/src/flutter/lib/ui/painting/image_descriptor.cc
+++ b/engine/src/flutter/lib/ui/painting/image_descriptor.cc
@@ -59,8 +59,12 @@ Dart_Handle ImageDescriptor::initEncoded(Dart_Handle descriptor_handle,
         "https://github.com/flutter/flutter/issues.");
   }

+  const auto start = std::chrono::steady_clock::now();
   auto generator =
       registry->CreateCompatibleGenerator(immutable_buffer->data());
+  const auto end = std::chrono::steady_clock::now();
+  const std::chrono::duration<double, std::milli> diff = end - start;
+  FML_LOG(ERROR) << std::fixed << std::setprecision(3) << "ImageGeneratorRegistry::CreateCompatibleGenerator: " << diff;

   if (!generator) {
     // No compatible image decoder was found.
@@ -122,13 +126,23 @@ void ImageDescriptor::instantiateCodec(Dart_Handle codec_handle,
 }

 bool ImageDescriptor::get_pixels(const SkPixmap& pixmap) const {
   FML_DCHECK(generator_);
-  return generator_->GetPixels(pixmap.info(), pixmap.writable_addr(),
+  const auto start = std::chrono::steady_clock::now();
+  auto ret = generator_->GetPixels(pixmap.info(), pixmap.writable_addr(),
                                pixmap.rowBytes());
+  const auto end = std::chrono::steady_clock::now();
+  const std::chrono::duration<double, std::milli> diff = end - start;
+  FML_LOG(ERROR) << std::fixed << std::setprecision(3) << "ImageDescriptor::get_pixels: " << diff;
+  return ret;
 }

ImageGeneratorRegistry::CreateCompatibleGenerator is called when an ImageDescriptor is instantiated for an image. It reads header metadata from the image to determine a compatible codec, and reads some other image information (e.g. width, height, orientation) to store in the created ImageDescriptor object. This code runs on the main/UI thread, so it needs to be very quick to avoid UI jank.

ImageDescriptor::get_pixels is called to decode the image. This happens before the image can be rendered. This is expected to take longer, and it happens on a separate thread.

I ran the test app with 3 different configurations:

• Current behavior. This uses the built-in Skia codec for jpeg decoding, which uses libjpeg-turbo under the hood. This is what is launched to users.
• Removing the built-in jpeg codec. This will use the Android codec fallback, using the ImageDecoder API.
• Removing the built-in jpeg codec and forcing the Android codec fallback to use the BitmapFactory API.

For each of these configurations, I ran the app 3 times to get average timing numbers. I ran the app on a Pixel 8 device in release mode.

CreateCompatibleGenerator

Both versions of the platform codec are slower than the current behavior for CreateCompatibleGenerator. The current behavior is consistently sub 0.5ms. ImageDecoder is mostly around 1ms, but jpeg 1 was 2ms. (There may be some warmup time which makes the initial jpeg a bit slower). BitmapFactory is mostly around 1ms, but jpeg 6 (notably a larger jpeg than the rest) was much slower at 4.51ms.

We strongly want CreateCompatibleGenerator to be as fast as possible, since this runs on the UI thread. If we are considering removing the jpeg codec, we must weigh the benefit of reducing the binary size with the con of slowing down these calls to CreateCompatibleGenerator.

get_pixels

Both versions of the platform codec have generally comparable get_pixels timing compared to the current behavior. The exception is with jpeg 6 (notably a larger jpeg, and notably it uses a non-sRGB color space). The current behavior is much slower to decode this image compared with the platform codecs. I dug in further to this slowdown, and believe #177805 is causing the current behavior to be extra slow.

Summary

• Android's platform codec supports decoding GIF, JPEG, PNG, and WebP, so we can consider removing the built-in codecs for these image formats to save binary size.
• API levels 24-25 will crash with an OOM when using the platform codec with high resolution images. We should not remove the built-in codecs until this can be addressed. This may be worked around with code (may be complicated), or we may want to wait until Flutter drops support for API levels below 26.
• Using the platform codec for GIF, PNG, and WebP will not currently support animation. We could implement animation support, but will take some effort.
• Looking at using the JPEG platform codec compared to the existing built-in JPEG codec, creating ImageDecoder objects is slower. This is undesirable because it happens on the UI thread. But it may be worth considering this slowdown as a tradeoff with the benefit of reducing binary size. The actual speed of decoding is mostly comparable, but using the platform codec has the advantage that it works around Image loading is significantly slower for non-sRGB color space images when Impeller is enabled #177805.