You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
OpacityLayer and ColorFilterLayer should detect if their children can be drawn without a saveLayer #92874
For example, in trivial cases where there is a single DisplayList child that has a single drawPath operation, it should just apply the opacity/color filter directly to that path instead of incurring a render target switch.
We could potentially use the RTree implementation in the engine to make this more sophisticated so that it can detect cases where the bounding boxes of multiple draw operations do not overlap.
If it helps, the concrete use case that prompted this was a custom widget that fades text in and out using a FadeTransition. The text is always a simple drawTextBlob, and it's not too hard to refactor the widget so that it applies the color directly to the text but it's hard for developers to realize that will be better than using FadeTransition.
In the case of OpacityLayer and FadeTransition, the typical execution of those mechanisms does not use saveLayer. The path that uses saveLayer is the backup path. The typical path caches the child and does a blit-with-alpha that should be about as fast as rendering the child with an alpha color. The primary savings for that main path would be to avoid the extra memory for the cache, as described in #75697
Instrumenting OpacityLayer to perform this optimization is on the plan for DisplayList. Right now we could instrument OpacityLayer to bypass the cache (or backup saveLayer) if the child is an image. We can't do that if the child is an SkPicture regardless of the content of the SkPicture because there is no "draw this picture with alpha" mechanism. Or, well, there is such an argument to drawPicture, but internally it is performed with a saveLayer. DisplayList could provide such a mechanism, though, and open that new rendering path to simple rendered children.
Unfortunately, the churn in the Pictures generated when a compositing layer appears and disappears can be worse than the saveLayer overhead, especially because the saveLayer can be avoided at the lower levels without require a change in the tree at the higher levels.
With the right RepaintBoundary the Pictures might not churn, but then adding/removing the OpacityLayer can churn the caching that makes the opacity itself fast leading to a blip when the opacity hits 1.0. If 1.0 is the endpoint then the blip isn't very noticeable, but if it is part of a longer animation then we get a re-cache blip every time we get a 1.0 frame.
So saveLayer is really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?
@Hixie at the RO level, there's no good way to tell if you're drawing something that overlaps with some other RO in the same layer/picture. This is both because the Picture is opaque, and because you really don't have a good sense about whether you're the sole-painter or not.
So saveLayer is really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?
Yes, the cache entry becomes just a source image to blit with modified opacity which is a cheap operation (and no saveLayer or target switches).
Currently FadeTransition always uses a layer and doesn't check compositing bits afaict. If it avoided the layer and tried to use saveLayer itself then we wouldn't be able to optimize that out at the engine OpacityLayer level and you'd end up with the issue that Dan mentions above - it is hard to tell how complex a picture is at the framework level. We could expose some info about that eventually, but we'd have to switch over to DisplayList completely for that.
Meanwhile, the engine opacity layer has other tricks up its sleeve that apply even when the child is complex. For that case it is using the raster cache in a way that is kind of like "saveLayer + keep the layer around for next frame" to amortize its layer creation cost.
Related to the RTree mention above, I'm not sure if the extra complexity added by SkRTree is worth the cost for a small number of ops.
I think an initial implementation could manage a stack of the last N draw rects. N would be relatively small (16, 32?) so you can do a simple linear search over the rects. If there are more than N draw commands in a single display list, there's already a fairly large probability that at least one overlaps. If you encounter a single overlap, then you stop checking for future overlaps.
For a worst case scenario, imagine an OpacityLayer containing an animated DisplayListLayer of a bunch of bouncing balls that can overlap. In the cases where two balls collide, we lose our OpacityLayer optimization and then quickly regain it. For a low number of bouncing balls (N < 32) you'd expect a low number of collisions. For a higher number (N > 32), the probability of collisions increases.
And in most cases, where an app is fading in text, or an image, this initial implementation would be fast.
So saveLayer is really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?
Yes, the cache entry becomes just a source image to blit with modified opacity which is a cheap operation (and no saveLayer or target switches).
@flar If so, can we replace most of the layer's saveLayer with raster cache? Such as the ClipPathLayer ClipRectLayer/ClipRRectLayer which used Clip::antiAliasWithSaveLayer. It seems that we can reduce the raster time but with no increase in memory.
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.
Activity
dnfield commentedon Nov 2, 2021
We could potentially use the RTree implementation in the engine to make this more sophisticated so that it can detect cases where the bounding boxes of multiple draw operations do not overlap.
zanderso commentedon Nov 2, 2021
This was already on the list for #85737, but didn't have an issue filed for it yet, so I added this there.
dnfield commentedon Nov 2, 2021
If it helps, the concrete use case that prompted this was a custom widget that fades text in and out using a
FadeTransition. The text is always a simpledrawTextBlob, and it's not too hard to refactor the widget so that it applies the color directly to the text but it's hard for developers to realize that will be better than usingFadeTransition.goderbauer commentedon Nov 2, 2021
Related internal bug: b/204887652
flar commentedon Nov 2, 2021
In the case of OpacityLayer and FadeTransition, the typical execution of those mechanisms does not use saveLayer. The path that uses saveLayer is the backup path. The typical path caches the child and does a blit-with-alpha that should be about as fast as rendering the child with an alpha color. The primary savings for that main path would be to avoid the extra memory for the cache, as described in #75697
Instrumenting OpacityLayer to perform this optimization is on the plan for DisplayList. Right now we could instrument OpacityLayer to bypass the cache (or backup saveLayer) if the child is an image. We can't do that if the child is an SkPicture regardless of the content of the SkPicture because there is no "draw this picture with alpha" mechanism. Or, well, there is such an argument to
drawPicture, but internally it is performed with a saveLayer. DisplayList could provide such a mechanism, though, and open that new rendering path to simple rendered children.Hixie commentedon Nov 2, 2021
In theory, the render objects only use layer if layers are needed. That's what the compositing bit is all about.
flar commentedon Nov 2, 2021
Unfortunately, the churn in the Pictures generated when a compositing layer appears and disappears can be worse than the saveLayer overhead, especially because the saveLayer can be avoided at the lower levels without require a change in the tree at the higher levels.
With the right RepaintBoundary the Pictures might not churn, but then adding/removing the OpacityLayer can churn the caching that makes the opacity itself fast leading to a blip when the opacity hits 1.0. If 1.0 is the endpoint then the blip isn't very noticeable, but if it is part of a longer animation then we get a re-cache blip every time we get a 1.0 frame.
dnfield commentedon Nov 2, 2021
So
saveLayeris really just bad because it involves a render target switch. @flar, are you saying that when we raster cache the child without a save layer we avoid a render target switch?dnfield commentedon Nov 2, 2021
@Hixie at the RO level, there's no good way to tell if you're drawing something that overlaps with some other RO in the same layer/picture. This is both because the
Pictureis opaque, and because you really don't have a good sense about whether you're the sole-painter or not.flar commentedon Nov 2, 2021
Yes, the cache entry becomes just a source image to blit with modified opacity which is a cheap operation (and no saveLayer or target switches).
flar commentedon Nov 2, 2021
Currently FadeTransition always uses a layer and doesn't check compositing bits afaict. If it avoided the layer and tried to use saveLayer itself then we wouldn't be able to optimize that out at the engine OpacityLayer level and you'd end up with the issue that Dan mentions above - it is hard to tell how complex a picture is at the framework level. We could expose some info about that eventually, but we'd have to switch over to DisplayList completely for that.
Meanwhile, the engine opacity layer has other tricks up its sleeve that apply even when the child is complex. For that case it is using the raster cache in a way that is kind of like "saveLayer + keep the layer around for next frame" to amortize its layer creation cost.
1 remaining item
ds84182 commentedon Nov 8, 2021
Related to the RTree mention above, I'm not sure if the extra complexity added by SkRTree is worth the cost for a small number of ops.
I think an initial implementation could manage a stack of the last N draw rects. N would be relatively small (16, 32?) so you can do a simple linear search over the rects. If there are more than N draw commands in a single display list, there's already a fairly large probability that at least one overlaps. If you encounter a single overlap, then you stop checking for future overlaps.
For a worst case scenario, imagine an OpacityLayer containing an animated DisplayListLayer of a bunch of bouncing balls that can overlap. In the cases where two balls collide, we lose our OpacityLayer optimization and then quickly regain it. For a low number of bouncing balls (N < 32) you'd expect a low number of collisions. For a higher number (N > 32), the probability of collisions increases.
And in most cases, where an app is fading in text, or an image, this initial implementation would be fast.
flar commentedon Dec 16, 2021
The OpacityLayer half of this was fixed by flutter-team-archive/engine#29775
wangying3426 commentedon Apr 24, 2022
@flar If so, can we replace most of the layer's saveLayer with raster cache? Such as the ClipPathLayer ClipRectLayer/ClipRRectLayer which used Clip::antiAliasWithSaveLayer. It seems that we can reduce the raster time but with no increase in memory.
jonahwilliams commentedon Sep 8, 2024
The engine does this via opacity peephole optimizations. While we don't do this for color filters, no-op color filters are much less common.
github-actions commentedon Sep 23, 2024
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.