Skip to content

OpacityLayer and ColorFilterLayer should detect if their children can be drawn without a saveLayer #92874

Closed

Description

@dnfield

For example, in trivial cases where there is a single DisplayList child that has a single drawPath operation, it should just apply the opacity/color filter directly to that path instead of incurring a render target switch.

/cc @flar @goderbauer @chinmaygarde

Activity

  1. added
    engineflutter/engine related. See also e: labels.
    c: performanceRelates to speed or footprint issues (see "perf:" labels)
    perf: speedPerformance issues related to (mostly rendering) speed
    P2Important issues not at the top of the work list
    on Nov 2, 2021
  2. dnfield commented on Nov 2, 2021

    @dnfield
    ContributorAuthor

    We could potentially use the RTree implementation in the engine to make this more sophisticated so that it can detect cases where the bounding boxes of multiple draw operations do not overlap.

  3. zanderso commented on Nov 2, 2021

    @zanderso
    Member

    This was already on the list for #85737, but didn't have an issue filed for it yet, so I added this there.

  4. dnfield commented on Nov 2, 2021

    @dnfield
    ContributorAuthor

    If it helps, the concrete use case that prompted this was a custom widget that fades text in and out using a FadeTransition. The text is always a simple drawTextBlob, and it's not too hard to refactor the widget so that it applies the color directly to the text but it's hard for developers to realize that will be better than using FadeTransition.

  5. goderbauer commented on Nov 2, 2021

    @goderbauer
    Member

    Related internal bug: b/204887652

  6. flar commented on Nov 2, 2021

    @flar
    Contributor

    In the case of OpacityLayer and FadeTransition, the typical execution of those mechanisms does not use saveLayer. The path that uses saveLayer is the backup path. The typical path caches the child and does a blit-with-alpha that should be about as fast as rendering the child with an alpha color. The primary savings for that main path would be to avoid the extra memory for the cache, as described in #75697

    Instrumenting OpacityLayer to perform this optimization is on the plan for DisplayList. Right now we could instrument OpacityLayer to bypass the cache (or backup saveLayer) if the child is an image. We can't do that if the child is an SkPicture regardless of the content of the SkPicture because there is no "draw this picture with alpha" mechanism. Or, well, there is such an argument to drawPicture, but internally it is performed with a saveLayer. DisplayList could provide such a mechanism, though, and open that new rendering path to simple rendered children.

  7. Hixie commented on Nov 2, 2021

    @Hixie
    Contributor

    In theory, the render objects only use layer if layers are needed. That's what the compositing bit is all about.

  8. flar commented on Nov 2, 2021

    @flar
    Contributor

    Unfortunately, the churn in the Pictures generated when a compositing layer appears and disappears can be worse than the saveLayer overhead, especially because the saveLayer can be avoided at the lower levels without require a change in the tree at the higher levels.

    With the right RepaintBoundary the Pictures might not churn, but then adding/removing the OpacityLayer can churn the caching that makes the opacity itself fast leading to a blip when the opacity hits 1.0. If 1.0 is the endpoint then the blip isn't very noticeable, but if it is part of a longer animation then we get a re-cache blip every time we get a 1.0 frame.

  9. dnfield commented on Nov 2, 2021

    @dnfield
    ContributorAuthor

    So saveLayer is really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?

  10. dnfield commented on Nov 2, 2021

    @dnfield
    ContributorAuthor

    @Hixie at the RO level, there's no good way to tell if you're drawing something that overlaps with some other RO in the same layer/picture. This is both because the Picture is opaque, and because you really don't have a good sense about whether you're the sole-painter or not.

  11. flar commented on Nov 2, 2021

    @flar
    Contributor

    So saveLayer is really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?

    Yes, the cache entry becomes just a source image to blit with modified opacity which is a cheap operation (and no saveLayer or target switches).

  12. flar commented on Nov 2, 2021

    @flar
    Contributor

    Currently FadeTransition always uses a layer and doesn't check compositing bits afaict. If it avoided the layer and tried to use saveLayer itself then we wouldn't be able to optimize that out at the engine OpacityLayer level and you'd end up with the issue that Dan mentions above - it is hard to tell how complex a picture is at the framework level. We could expose some info about that eventually, but we'd have to switch over to DisplayList completely for that.

    Meanwhile, the engine opacity layer has other tricks up its sleeve that apply even when the child is complex. For that case it is using the raster cache in a way that is kind of like "saveLayer + keep the layer around for next frame" to amortize its layer creation cost.

  13. 1 remaining item

  14. ds84182 commented on Nov 8, 2021

    @ds84182
    Contributor

    Related to the RTree mention above, I'm not sure if the extra complexity added by SkRTree is worth the cost for a small number of ops.

    I think an initial implementation could manage a stack of the last N draw rects. N would be relatively small (16, 32?) so you can do a simple linear search over the rects. If there are more than N draw commands in a single display list, there's already a fairly large probability that at least one overlaps. If you encounter a single overlap, then you stop checking for future overlaps.

    For a worst case scenario, imagine an OpacityLayer containing an animated DisplayListLayer of a bunch of bouncing balls that can overlap. In the cases where two balls collide, we lose our OpacityLayer optimization and then quickly regain it. For a low number of bouncing balls (N < 32) you'd expect a low number of collisions. For a higher number (N > 32), the probability of collisions increases.

    And in most cases, where an app is fading in text, or an image, this initial implementation would be fast.

  15. flar commented on Dec 16, 2021

    @flar
    Contributor

    The OpacityLayer half of this was fixed by flutter-team-archive/engine#29775

  16. wangying3426 commented on Apr 24, 2022

    @wangying3426
    Contributor

    So saveLayer is really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?

    Yes, the cache entry becomes just a source image to blit with modified opacity which is a cheap operation (and no saveLayer or target switches).

    @flar If so, can we replace most of the layer's saveLayer with raster cache? Such as the ClipPathLayer ClipRectLayer/ClipRRectLayer which used Clip::antiAliasWithSaveLayer. It seems that we can reduce the raster time but with no increase in memory.

  17. moved this to To Triage in Display Liston Jun 28, 2022
  18. moved this from To Triage to Performance in Display Liston Jun 28, 2022
  19. jonahwilliams commented on Sep 8, 2024

    @jonahwilliams
    Contributor

    The engine does this via opacity peephole optimizations. While we don't do this for color filters, no-op color filters are much less common.

  20. moved this from ⚡ Performance to ✅ Done in Display Liston Sep 8, 2024
  21. github-actions commented on Sep 23, 2024

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  22. locked as resolved and limited conversation to collaborators on Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work listc: performanceRelates to speed or footprint issues (see "perf:" labels)engineflutter/engine related. See also e: labels.perf: speedPerformance issues related to (mostly rendering) speedteam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions