Note: This is a somewhat ambigious title for a longer open ended project.

Background

Flutter records canvas operations (draw rect, save layer) on the UI thread and plays these operations back on the raster thread. These operations are recorded in the display list format (See https://github.com/flutter/engine/tree/main/display_list ). To support the Skia -> Impeller transition, the format is mostly backend agnostic and is converted to appropriate Skia/Impeller objects on the fly during the raster workload. For Impeller at least, this conversion can actually be quite costly. Furthermore, we have ended up with two implementations of various raster tasks such as bounds compuation and opacity peephole, which is not sustainable long term.

For reference, here are some traces of display list -> impeller conversions on a moderately complex project. The Impeller API requires multiple heap allocations per drawing command (entity, color source, geometry) which adds some extra inefficiency. We've noticed on android that allocation / deallocation is much slower on iOS.

If we zoom in on this we can see its due to the excessive heap allocations in the Impeller API.

Overview

At a high level, there are two problems with the display_list -> Impeller interop that need to be solved:

1. (easy) DisplayList itself contains mostly Skia objects.

For the most part this isn't really a problem. The conversion of types like SkRect/SkPoint/SkColor don't appear in any of our traces or profiles. The conversion of SkPaths and TextBlobs are extremely expensive though, the latter has already been moved into display list via some hacks that I added. We also rely on the SkPath for correct/fast bounds computation (Impeller paths have slow bounds computations) and for the convexity check that is critical for tessellation performance.

2. (harder) EntityPass references display list + offsets instead of constructing new rendering tree

The most expensive part of the interop usually has to do with Impeller heap allocating a geometry/color source/entity for every single drawing command in the display list. I think the best approach involves changing entity pass itself to traverse the display list, and giving display list the appropriate APIs. In theory, this would allow almost no additional allocation in entity pass.

3. (cleanup) Impeller should be defering to display list bounds compuation.

Rather than try to fix Impeller's coverage calculations, we should have display list provide local/global bounds for all drawing commands that it has computed them for. We can then have a single means of computing coverage that we can ensure is correct.

Long term, we can provide display_list as the API for interacting with Flutter and delete aiks.

Alternatives Considered

• Fix the Impeller API to perform fewer heap allocations.

We could flatten the impeller API into a display list like format. Why have two display list formats though?

• Make the Impeller API the recording format / remove display list.

@dnfield tried this in: ___ . Our evaluation at the time was that it would take a lot of work to fix all of the bugs in Impeller's APIs to bring it up to display list quality. Another realization was that to achieve the performance we'd want, we'd need to flatten the recording structure into something that is display list like anyway.