Skip to content

[Web, engine] Decoding a semi-transparent image gives slightly incorrect results #92958

@dkwingsmt

Description

@dkwingsmt

When decoding a bitmap image, the resulting pixels might be slightly off if any of the pixels are partially transparent, i.e. contain alpha between 1 ~ 254. This does not occur if all pixels have alpha of either 0 or 255.

This is observed on HTML renderer. I have not tested if it reproduces on canvaskit, but I suspect not.

Following is the testing code:

  test('Correctly encodes a transparent image', () async {
    // A 2x2 testing image with transparency.
    // Pixel order: Left to right, then top to bottom.
    // Byte order: 0xAABBGGRR (because uint8 is placed in little endian.)
    final Uint8List sourceImage = Uint8List.sublistView(Uint32List.fromList(
      <int>[ /* To be filled later */ ],
    ));
    final ImageDescriptor descriptor = ImageDescriptor.raw(
      await ImmutableBuffer.fromUint8List(sourceImage),
      width: 2,
      height: 2,
      pixelFormat: PixelFormat.rgba8888,
    );
    final Image encoded = (await (await descriptor.instantiateCodec()).getNextFrame()).image;
    final Uint8List actualPixels  = Uint8List.sublistView(
        (await encoded.toByteData(format: ImageByteFormat.rawStraightRgba))!);
    // expect(actualPixels, listEqual(actualPixels)); // Broken
  });

In the code above, descriptor contains a BMP representation of the source image, encoded is an HtmlImage, and encoded.toByteData decodes the data back into raw pixels.

If any pixel is partially transparent, then all pixels might be slightly off, including the alpha channel:

// Byte order: 0xAABBGGRR
source: 0x030201FF, 0x0605FE04, 0x09FD0807, 0xFC0C0B0A
actual: 0x030000FF, 0x0600FF00, 0x09FF0000, 0xFC0C0B0A

In comparison, if all pixels have either 0 or 255 on alpha, the accurate pixels are decoded:

// Byte order: 0xAABBGGRR
source: 0xFF0201FF, 0xFF05FE04, 0xFFFD0807, 0x000C0B0A
actual: 0xFF0201FF, 0xFF05FE04, 0xFFFD0807, 0x00000000

Note that the fully transparent pixel is turned all 0, which is fine since it's transparent anyway.

The reason is unknown, but I suspect it's because HtmlImage decides image data using canvas.drawImage, which might blend semi-transparent image using a different algorithm that introduces rounding errors. Note that the deviation also depends on browser. Running on Firefox gives a different value from Chrome, neither correct.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work lista: imagesLoading, displaying, rendering imagese: web_htmlHTML rendering backend for Webengineflutter/engine related. See also e: labels.platform-webWeb applications specificallyteam-webOwned by Web platform teamtriaged-webTriaged by Web platform team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions