Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Conversation

@flar
Copy link
Contributor

@flar flar commented Sep 25, 2021

Fixes flutter/flutter#82955

I did some renaming (and reformatting) of the transform parameters to maintain a consistent terminology all the way through.

One minor change in behavior I discovered during testing is that SkCanvas::concat(SkMatrix::Identity) would be NOPed by the call and not stored in an SkPicture (or passed along to the DisplayList), but now that I'm providing the full 16 matrix entries in a concat(SkM44) call, that check is not done. At least for DisplayList, I do the check anyway, so this would only affect the (soon to be legacy) SkPicture usages.

The fix has its own tests in this change, but it was also tested using the test case in the Github Issue on MacOS and Android (both with and without DisplayList), and Chrome/CanvasKit. The problem didn't originally happen with the HTML/DOM renderer and is still working fine.

@google-cla google-cla bot added the cla: yes label Sep 25, 2021
@flutter-dashboard flutter-dashboard bot added the platform-web Code specifically for the web engine label Sep 25, 2021
Copy link
Member

@chinmaygarde chinmaygarde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote would be to simplify this even further and get rid of SkMatrix altogether and stick to SkM44 everywhere in the engine.

void transform3x3(SkScalar mxx, SkScalar mxy, SkScalar mxt,
SkScalar myx, SkScalar myy, SkScalar myt,
SkScalar mwx, SkScalar mwy, SkScalar mwt) override;
void transform4x4(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: In the documentation, can you mention this is the same as column-major storage. I know the arguments are somewhat descriptive but a quick note here would make it easier to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually row-major, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I'll add some reminder comments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh, you're right. The updated comment is clear enough.

SkMatrix matrix_;
std::vector<SkMatrix> saved_;
SkM44 matrix_;
SkMatrix matrix33_;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am probably misremembering the action items from our conversation the other day. But, shouldn't we just avoid using SkMatrix altogether? It brings nothing to the table except minor storage gains (relative to everything else in the display list) but causes subtle issues in corner cases. Let's just use a single consistent POD type for matrices. It ought to simplify a whole heck of a lot of code and cause less confusion in the process.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation is there on the accessor methods in the base class (SkMatrixSource). For a single transform operation, SkMatrix works fine. If you work out what happens when you transform (x, y, 0, 1) and only need the x,y from the result, SkMatrix has all of the entries you need.

The important thing is that you accumulate the 4x4 for all transform mutations, but then you can use the SkMatrix for transforming points and rects. Also, on a more practical note, that SkM44 doesn't have any mapping methods.

I'll update the block comment in the Dispatcher block comment with the specifics of the math. And maybe sprinkle in some reminder comments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With regards to storage, 99.9% of the matrices we deal with are 2x3. Also, for the code reading and analyzing a DisplayList it helps to know up front if the matrix operations are all 2D affine. Once a transform4x4() method is dispatched it begs the question of whether or not 3D operations are happening without examining the values. For now, the characterization of the matrix into 1 of 3 categories happens in the Builder so the DisplayList streams can trust the categorization without any redundant characterization code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an argument to be made for eliminating the transform3x3. If someone introduces a perspective transform into the mix then a 3x3 may be enough for leaf transform operations, but it is not enough for further composition. Just as the engine, over time, came to trust an SkMatrix for its matrix operations even though it cannot compose multiple non-2D-affine transforms, having that dispatch method may come to have the same effect on future uses of the DisplayList - "hey they take a 3x3 and it has perspective entries, so it must be something that I should use!". Better to set the precedent now to only deal with 2x3 (extremely common 2D affine) or 4x4 so that if you even think of embracing perspective then you do so with the more responsible 4x4.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good on the arguments. About SkM44 not offering the mapping methods, perhaps we should also file issues for Skia to implement them?

Copy link
Contributor Author

@flar flar Sep 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to add that request, but also to think of suggestions for how the API docs and positioning of the APIs can be enhanced to discourage the kind of misuse we've had. There are still a number of places in SkCanvas where you can helpfully hand in an SkMatrix (i.e. 3x3) that will be composed with the canvas matrix, but does it really have enough precision? I see they've deprecated getTotalMatrix() which returns a subset SkMatrix and the new methods require you to deliberately request a 3x3. I would go a step further and not have get...AsM33() and force the caller to get...().asM33() so it is even more deliberate that they are subsetting the answer (though it would require more copying of data). If SkM44 provided mapping methods, it is unlikely that method would be needed.

Another option is to (would have been to) have a wrapper class SkTransform which provides the utility methods and does not indicate whether it is 2x3, or 4x4. Underneath it will maintain whatever precision is needed for its history of operations. Then you can get the efficiency of 2x3 implementations if your (very common) app is all 2D affine and only pay the price of 4x4 when it is needed.

Also, the commit that eliminates the 3x3's has just been pushed.

@flar flar requested a review from chinmaygarde September 26, 2021 00:41
@flar
Copy link
Contributor Author

flar commented Sep 26, 2021

One thought regarding exposure of SkMatrix - at one point I was going to have the MatrixDispatchHelper provide a mapping method for rectangles to hide the internal use of SkMatrix to do the dirty work, but when we go after the Preroll/Paint method chains we will be handing along an SkM44 and many of the layers will need to do transforms and so will need to use asM33 to get an object that can do that for them.

Maybe this will all go away if we follow through on the umbrella task item to get rid of all SkObjects in the DisplayList API and create our own matrix object, but until then I figure it is reasonable to provide SkMatrix for leaf operations as long as we don't use them to accumulate transforms.

void transform3x3(SkScalar mxx, SkScalar mxy, SkScalar mxt,
SkScalar myx, SkScalar myy, SkScalar myt,
SkScalar mwx, SkScalar mwy, SkScalar mwt) override;
void transform4x4(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh, you're right. The updated comment is clear enough.

SkMatrix matrix_;
std::vector<SkMatrix> saved_;
SkM44 matrix_;
SkMatrix matrix33_;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good on the arguments. About SkM44 not offering the mapping methods, perhaps we should also file issues for Skia to implement them?

@flar flar added the waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land. label Sep 27, 2021
@flar
Copy link
Contributor Author

flar commented Sep 27, 2021

Tagging @yjbanov to look at what I did for CanvasKit.

@flar flar requested a review from yjbanov September 27, 2021 21:39
@flar flar added waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land. and removed waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land. labels Sep 27, 2021
@fluttergithubbot fluttergithubbot merged commit a84b2a7 into flutter:master Sep 28, 2021
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 28, 2021
dnfield pushed a commit to dnfield/engine that referenced this pull request Oct 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla: yes platform-web Code specifically for the web engine waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Canvas takes a 4x4 matrix but does not store the full 4x4 matrix

3 participants