Skip to content

Conversation

@knopp
Copy link
Member

@knopp knopp commented Feb 27, 2025

Fixes #164320

If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@flutter-dashboard

This comment was marked as outdated.

@github-actions github-actions bot added the framework flutter/packages/flutter repository. See also f: labels. label Feb 27, 2025
@knopp
Copy link
Member Author

knopp commented Feb 27, 2025

@goderbauer, does this seems like it's going good direction? I haven't had much time to play with it, but on the surface it seems to fix the issue:

multiview-fixed.mov

@knopp knopp changed the title WIP: Only repaint views need to be painted WIP: Only repaint views that need to be painted Feb 27, 2025
Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original direct for loop was added in one of @goderbauer's original multiview PRs:

https://github.com/flutter/flutter/pull/125003/files#diff-7a19e9274d8892fe8800bbbbcafa158116cac3a558f42222786b004810b17876L544-R638

Looking at those lines, it seems reasonable that it was indeed just an oversight I think?

@knopp knopp force-pushed the multiview_repaint branch 2 times, most recently from 705e9c1 to 8a85763 Compare February 28, 2025 22:18
@knopp knopp changed the title WIP: Only repaint views that need to be painted Only repaint views that need to be painted Feb 28, 2025
@knopp knopp requested a review from matanlurey as a code owner February 28, 2025 22:37
@knopp knopp changed the title Only repaint views that need to be painted WIP: Only repaint views that need to be painted Feb 28, 2025
@knopp knopp force-pushed the multiview_repaint branch from d8ac4bc to 8fda595 Compare February 28, 2025 23:06
@knopp knopp changed the title WIP: Only repaint views that need to be painted Only repaint views that need to be painted Feb 28, 2025
@knopp knopp requested review from goderbauer and justinmc February 28, 2025 23:35
@flutter-dashboard

This comment was marked as outdated.

@flutter-dashboard flutter-dashboard bot added the will affect goldens Changes to golden files label Mar 1, 2025
@matanlurey
Copy link
Contributor

It looks like this PR breaks both external texture rendering and platform view rendering on Android:

image

image

If you want instructions on running/debugging these locally:

https://github.com/flutter/flutter/tree/master/dev/integration_tests/android_engine_test

@knopp knopp force-pushed the multiview_repaint branch from 8fda595 to fa48498 Compare March 1, 2025 11:01
@flutter-dashboard
Copy link

Golden file changes are available for triage from new commit, Click here to view.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

Changes reported for pull request #164322 at sha cc5e1c0

@knopp knopp changed the title Only repaint views that need to be painted WIP: Only repaint views that need to be painted Mar 1, 2025
@knopp
Copy link
Member Author

knopp commented Mar 1, 2025

This needs additional work to support external textures.

  • Marking new external texture available without regenerating trees will unconditionally repaint all views. This would not be a blocker for this PR (it is not a regression).

  • Marking new external texture available may not repaint view with texture. This is definitely a blocker. Framework currently does not know that the texture is dirty and if there is no other render object in the view that needs to be repainted the view is skipped from compositing. This may explain failing goldens on android. A frame is scheduled, but the framework has no idea that the platform view texture is dirty. I'm also not sure if the embedder marks new texture available on change, but even if it did it wouldn't be enough.

@knopp
Copy link
Member Author

knopp commented Mar 1, 2025

I think we'll need to wire MarkTextureFrameAvailable to framework to ensure that the view with dirty texture is always recomposited when regenerating layer trees.

We'll also need to make sure that Android embedder calls MarkTextureFrameAvailable for platform view textures. This might be already the case, I'll need to check. @matanlurey, any thoughts?

Stretch goal: Rasterizer::DrawLastLayerTrees should only repaint trees with dirty textures. Assuming that's the only reason right now to redraw layer tree without regenerating. Is it?

@matanlurey matanlurey requested a review from jonahwilliams March 1, 2025 16:55
@jonahwilliams
Copy link
Contributor

Its sort of a design goal of MarkTextureFrameAvailable that it bypasses the UI thread if possible, to reduce the framerate and workload for android applications with animating platform views. So making that roundtrip back to UI seems like a step backwards.

I'm not actually sure why skipping the repaint is causing a black screen though. the engine should happily enough use the previous layer tree.

@knopp
Copy link
Member Author

knopp commented Mar 3, 2025

I think I didn't explain it well. MarkTextureFrameAvailable bypasses the UI thread, and that works as expected, the problem happens when both MarkTextureFrameAvailable() and ScheduleFrame are called. Now we can no longer bypass the UI thread (because frame is scheduled), but as far as Flutter is concerned, nothing in RenderObject tree is dirty, so nothing gets composited (with this PR).

This seems to be the situation with platform views on Android. Instead of just rendering previous layer tree, a frame is scheduled by the embedder.

@jonahwilliams
Copy link
Contributor

hmmmmmmm. I guess we could post messages to both raster and UI task but that feels ... hacky. (Honestly the whole external texture interop feels hacky anyway...)

@knopp
Copy link
Member Author

knopp commented Mar 4, 2025

Consider the following example with multiview:

View1 has external texture that has new frame but no dirty render objects
View2 has some state change and dirty render objects

With this PR, we would only recompose View2, since there are no dirty render objects in View1.
To fix this, MarkTextureFrame available will likely need to call into UI thread to mark the texture render object as dirty. This should not schedule new frame, that would kill the point of external textures, but the frame is already scheduled because of View2 so framework needs to know that View1 needs recomposing.

@jonahwilliams
Copy link
Contributor

I guess post thread merge, the texture is actually going to be marked dirty on the combined platform/ui thread, so calling back into UI doesn't seem that bad

@flutter-dashboard
Copy link

This pull request executed golden file tests, but it has not been updated in a while (20+ days). Test results from Gold expire after as many days, so this pull request will need to be updated with a fresh commit in order to get results from Gold.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@matanlurey matanlurey removed their request for review April 9, 2025 18:56
@Piinks
Copy link
Contributor

Piinks commented Jun 10, 2025

Greetings from stale PR triage! 👋
What is that status of this PR?

@knopp
Copy link
Member Author

knopp commented Jun 11, 2025

There are issues with it. I'll close it for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

framework flutter/packages/flutter repository. See also f: labels. will affect goldens Changes to golden files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Multiview] Flutter re-renders all views during each update

5 participants