Forceably caching the child when the child is a simple rendering can be limiting. We discovered this when the same paradigm was used with OpacityLayer and ImageFilterLayer. We're starting to embrace a different form of optimization for things like this as can be seen in the OpacityLayer and the way it checks if it can hand off/distribute the opacity to its children rather than caching them. For an example:

For one thing, this saves a layer creation if the child is just a TextureLayer, or if it is a DisplayListLayer with a single rendering op or a saveLayer around a bunch of rendering ops (though a DL might be cached eventually - but if it is cached then we don't need the RTL to cache as well, and if it is not cached then it is changing on every frame and the manual cache by the RTL is costing memory that holds over between frames when the DL can just absorb the transform naturally).

Is there a way a similar mechanism could be used by ImageFiltered with a matrix transform instead of introducing a new layer?

Note that the memory usage for an OpacityLayer, ImageFilterLayer, or this RTL for the case where the child is changing is double the memory that a saveLayer would cost. The problem is that a saveLayer uses temporary buffers that are reclaimed and reused better. These layer cache entries are held over for at least a frame even if they aren't going to be used again. The problem is that we discover that they aren't used and reclaim their memory at the end of the next frame where they weren't used.

That holdover and late reclamation may be solved by some upcoming work to redistribute the raster cache population work to either its own pass or to the beginning of the Paint pass where the old cache entries could be freed before the new cache entries are populated. I'm working on that right now, but being distracted by other work while I reorganize it. In the meantime, even with the better reclamation that that work would bring, it points out how clunky the "cacheable child" mechanism has been historically. It solves a performance problem, but creates a memory problem while doing so.

I would characterize this solution as a replacement for:

  TransformLayer(
    <animating transform>,
    filterQuality: X,
    child: TransformLayer.scale(<scale>, child: <child>),
  ),

which currently translates into:

  pushImageFilter(IF.Matrix(<animating transform>))
  pushTransform(<scale>)
  addChild(child)
  pop(); pop();

but with implementing it via a drawImage op rather than an ImageFilter.matrix op. That can be accomplished in other ways, ImageFilterLayer can detect a Matrix op and implement the above technique that RTL is doing and Skia is aware of the fact that a MatrixImageFilter could detect the optimization so they should eventually do that on their own. And we could potentially implement it by passing it to the child for better efficiency in some cases.

Right. Main reason why I introduced the scale in this PR was to avoid the need for that secondary TransformLayer in the tree, since the Flutter-side layer tree doesn't work well with layers that produce multiple Scene layers.

If ImageFilterLayer was optimized for transforms it would work too. Just makes it a bit harder to dynamically control the scale of the rastered subtree.

Closing this for now, optimizing ImageFilterLayer seems like a better idea in the short term.