Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Conversation

@flar
Copy link
Contributor

@flar flar commented Apr 4, 2024

The DisplayListMatrixClipTracker class performs the working of maintaining a stack of cull rects and transforms for classes in the engine that require such functionality (such as DisplayListBuilder, DiffContext, and LayerStateStack.

Until now it has been using a mixture of SkMatrix and SkM44 along with an SkRect to maintain this information, but that complexity involves overhead and implementation complexity which complicates future work on these areas.

By reimplementing the class to use Impeller geometry objects, the internal architecture is greatly simplified allowing for easier future maintenance.

@flar
Copy link
Contributor Author

flar commented Apr 4, 2024

This change introduces small performance regressions at the level of the DisplayListBuilder microbenchmarks, but those should not be evident over the entire course of a running app.

There is also currently a big performance improvement in one of those benchmarks which represents a potential bug - the Impeller rect transform code does not clip the transformed rectangle against the near clipping plane when a perspective matrix is used. I will be investigating solutions to that issue and how it affects the accuracy of the results next and hope to have some conclusions before this PR lands.

Since there were unit tests failing due to the difference in behavior between Skia's mapRect and Impeller's TransformBounds methods due to the Skia implementation implementing clipping against the near plane in the perspective case, I implemented a new TransformAndClipBounds method in Impeller to make the behaviors match. The first set of numbers below are from the original implementation here which used the faster, but less accurate, non-clipping bounds method and the second set are while using the new more accurate (but a little slower) perspective-clipping bounds method.

Here are the representative numbers:

Benchmark cpu time with Skia (us) cpu time with Impeller (us) % change cpu time with fixed Bounds (us) % change
BM_DisplayListBuilderDefault 4.24 5.09 20.05% 5.29 24.76%
BM_DisplayListBuilderWithScaleAndTranslate 4.36 5.18 18.81% 5.38 23.39%
BM_DisplayListBuilderWithPerspective 38.2 8.97 -76.52% 9.31 -75.63%
BM_DisplayListBuilderWithClipRect 4.27 5.1 19.44% 5.34 25.06%
BM_DisplayListBuilderWithGlobalSaveLayer 5.15 6.04 17.28% 6.53 26.80%
BM_DisplayListBuilderWithSaveLayer 21.4 24.2 13.08% 22.6 5.61%
BM_DisplayListBuilderWithSaveLayerAndImageFilter 26.8 26.6 -0.75% 26.7 -0.37%

@chinmaygarde chinmaygarde self-requested a review April 4, 2024 20:13
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't fully looked through all of the math yet, though I trust it will be good :)

Do we still need to do an additional heap allocation for the DisplayListMatrixClipState? Or can we store that in the vector as is

@jonahwilliams
Copy link
Contributor

I think the perspective bug is flutter/flutter#131445 .

@flar
Copy link
Contributor Author

flar commented Apr 4, 2024

I haven't fully looked through all of the math yet, though I trust it will be good :)

Do we still need to do an additional heap allocation for the DisplayListMatrixClipState? Or can we store that in the vector as is

Good catch. That is a remnant of when there were 2 kinds of state and I didn't think to remove that, I was focused on the change to impeller structures and making the individual State structures public. I'll fix that up [Done].

Eventually, though, Tracker will go away and the Builder, which already has a state stack of its own, and other similar classes will just inline the State into their own state stacks. If anyone is using Tracker that doesn't already have their own vector/stack then it might stay around, but there is no reason I can't make it more efficient by eliminating the redirection in the Tracker vector.

Also, at one point I thought "maybe tracker should just have an operator-> indirect operator to forward all of the State methods to the State object", but I wanted to avoid impact on other files at least for now. It would remove a lot of redirection methods, though, so I can try it and see if it impacts the benchmarks. If Tracker goes away eventually, though, this "semantic trick" will be deleted...

@flar
Copy link
Contributor Author

flar commented Apr 5, 2024

The DiffContext failure definitely seems to be due to trying to transform a rectangle with a perspective matrix that produces negative weights. I may have to solve that compatibility issue before this can be pushed. I'd like to do that separately from this PR...

@flar
Copy link
Contributor Author

flar commented Apr 7, 2024

This PR now includes the new impeller::Rect::TransformAndClipBounds method which is only used by this code at the moment. It makes us compatible with Skia's implementation that clips the rect bounds against the near clipping plane (represented by w < 0 during a perspective transform) and thus maintains compatibility. After some settling time we can consider whether to make it the default behavior for impeller::Rect::TransformBounds.

If this fixes all of the outstanding test failures, I'll still want to add new targeted unit tests for the couple of new methods I've added to Impeller.

@flar flar force-pushed the dl-matrix-clip-tracker-on-impeller branch from 9687c89 to 3c83209 Compare April 8, 2024 09:32
@flar
Copy link
Contributor Author

flar commented Apr 8, 2024

Once this passes tests Now that this PR is passing all CI tests, I can create another PR that attempts to convert the Impeller "NumberNear" function to use floating point "ULP" processing to check for "near" numbers. I don't want to make a global change like that for this PR, though.

See flutter/flutter#146455

Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

We have a few places where we stop culling due to the old transform issue. If the new implementation is good we should make sure we update those.

@flar
Copy link
Contributor Author

flar commented Apr 9, 2024

LGTM!

We have a few places where we stop culling due to the old transform issue. If the new implementation is good we should make sure we update those.

What about just merging them? We will see the comparative trends in the benchmark data in a few days.

@flar flar added the autosubmit Merge PR when tree becomes green via auto submit App label Apr 9, 2024
@auto-submit auto-submit bot merged commit 0226114 into flutter:main Apr 9, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Apr 9, 2024
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Apr 9, 2024
gilnobrega pushed a commit to gilnobrega/flutter that referenced this pull request Apr 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

autosubmit Merge PR when tree becomes green via auto submit App e: impeller

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants