Repository navigation
[Android] Fix HCPP frame synchronization when multiple frames run before a Choreographer traversal - #194591
Conversation
PlatformViewsController2.publishTransaction() was appending every raster SurfaceControl.Transaction into a single flat pendingRasterTransactions list across frames, and swapTransactions() on the platform thread drained the entire list at once. Because SubmitFlutterView() posts its platform-thread task asynchronously after submitting the overlay and main SurfaceFrames, the raster thread can begin or complete frame N+1 before frame N's platform task runs swapTransactions(). Draining the flat list caused frame N's platform task to merge frame N+1's overlay (or overlay and main) transaction into frame N's onEndFrame(), pairing frame N+1's Flutter surface buffer(s) with frame N's platform view layout and leaving frame N+1's platform task short. Add sealFrameTransactions() across PlatformViewAndroidJNI, FlutterJNI, and PlatformViewsController2, and call it from AndroidExternalViewEmbedder2::SubmitFlutterView() on the raster thread after submitting the frame's surfaces and before posting the platform task. PlatformViewsController2 queues each sealed frame batch into pendingRasterFrames so swapTransactions() pops one frame's raster transactions per frame.
- PlatformViewsController2Test: verify that swapTransactions() pops one sealed frame batch at a time, leaving an in-progress next frame's published overlay transaction unmerged until that frame seals and keeping back-to-back sealed frames separated across swaps. - AndroidExternalViewEmbedder2: verify that SubmitFlutterView() seals each frame's raster transactions synchronously on the raster thread before the posted platform-thread task runs swapTransaction() and onEndFrame2().
…otonic present_id
There was a problem hiding this comment.
Code Review
This pull request introduces monotonic present ID tracking in AHBSwapchainImplVK to prevent superseded callbacks from displacing displayed textures, and updates PlatformViewsController2 to queue transactions per raster frame and defer frames arriving before the current Choreographer traversal completes. The review feedback focuses on reducing garbage collection churn by optimizing pendingRasterFrames to use List and Collections.emptyList() when empty, as well as simplifying the handler logic by eagerly initializing mainHandler to eliminate redundant null checks.
mboetger
left a comment
There was a problem hiding this comment.
Looks like the new method caused google3 issues and needs a fix.
gemini suggests its an issue with how we're resolving engine artifacts for google testing and rebasing will fix 🤷 🙏 |
…ore a Choreographer traversal (flutter#194591) Fixes a few related synchronization issues in HCPP when the raster thread gets ahead of the platform thread, or when two platform-thread frame tasks run before Android's `Choreographer.doFrame()` has drawn the first one. Together, this effectively moves HCPP presentation from mailbox semantics (where a second frame arriving before `doFrame()` overwrote the pending frame in `applyTransactionOnDraw`) to FIFO, aligning with core Flutter. The end result is that scrolling with multiple platform views looks a lot, lot smoother. 1. Previously, `PlatformViewsController2.publishTransaction()` appended every raster `SurfaceControl.Transaction` into a single flat `pendingRasterTransactions` list, and `swapTransactions()` on the platform thread drained the entire list at once. Because `SubmitFlutterView()` posts its platform-thread task asynchronously after submitting the overlay and main `SurfaceFrame`s, the raster thread can start or finish frame `N+1` before frame `N`'s platform task runs `swapTransactions()`. Draining the flat list caused frame `N` to merge frame `N+1`'s transactions into its `onEndFrame()`, pairing frame `N+1`'s Flutter surface buffers with frame `N`'s platform view layout (and leaving frame `N+1`'s platform task without its transactions). This adds `sealFrameTransactions()` and calls it on the raster thread at the end of `SubmitFlutterView()` so `swapTransactions()` pops one sealed frame batch at a time. 2. Similarly, on the platform thread, `onEndFrame()` calls `flutterView.invalidate()` and `rootSurfaceControl.applyTransactionOnDraw(tx)`, which defers drawing the view hierarchy and applying the transaction until the upcoming `Choreographer` traversal. If a second frame's platform task runs before that traversal happens, it would immediately mutate the platform view layout params and merge a second transaction into `applyTransactionOnDraw`, overwriting the first frame before it was ever drawn. `PlatformViewsController2` now defers a second frame's view operations and transactions into a `QueuedFrame` while a frame is already staged, and replays/commits it once the traversal finishes (using a `Choreographer.FrameCallback` that posts an async message to the front of the main `Handler` queue, plus a `flushCompletedTraversalIfNeeded()` check in case `MessageLoopAndroid` runs inside `MessageQueue.nativePollOnce()` right after `doFrame()` before that `Handler` message is dispatched). 3. When two transactions on the same `SurfaceControl` do get merged before a single draw (for example if 3+ frames arrive within one VSync and coalesce in `QueuedFrame`), the later transaction's buffer overwrites the earlier one, but Android's `SurfaceComposerClient` retains both transaction completion callbacks in an `unordered_set` and can invoke the newer callback before the older, superseded callback. `AHBSwapchainImplVK` now tracks a monotonic `present_id` so a superseded callback recycles its own dropped texture into `pool_` instead of displacing `currently_displayed_texture_`. --------- Co-authored-by: Gray Mackall <[email protected]>
…ore a Choreographer traversal (flutter#194591) Fixes a few related synchronization issues in HCPP when the raster thread gets ahead of the platform thread, or when two platform-thread frame tasks run before Android's `Choreographer.doFrame()` has drawn the first one. Together, this effectively moves HCPP presentation from mailbox semantics (where a second frame arriving before `doFrame()` overwrote the pending frame in `applyTransactionOnDraw`) to FIFO, aligning with core Flutter. The end result is that scrolling with multiple platform views looks a lot, lot smoother. 1. Previously, `PlatformViewsController2.publishTransaction()` appended every raster `SurfaceControl.Transaction` into a single flat `pendingRasterTransactions` list, and `swapTransactions()` on the platform thread drained the entire list at once. Because `SubmitFlutterView()` posts its platform-thread task asynchronously after submitting the overlay and main `SurfaceFrame`s, the raster thread can start or finish frame `N+1` before frame `N`'s platform task runs `swapTransactions()`. Draining the flat list caused frame `N` to merge frame `N+1`'s transactions into its `onEndFrame()`, pairing frame `N+1`'s Flutter surface buffers with frame `N`'s platform view layout (and leaving frame `N+1`'s platform task without its transactions). This adds `sealFrameTransactions()` and calls it on the raster thread at the end of `SubmitFlutterView()` so `swapTransactions()` pops one sealed frame batch at a time. 2. Similarly, on the platform thread, `onEndFrame()` calls `flutterView.invalidate()` and `rootSurfaceControl.applyTransactionOnDraw(tx)`, which defers drawing the view hierarchy and applying the transaction until the upcoming `Choreographer` traversal. If a second frame's platform task runs before that traversal happens, it would immediately mutate the platform view layout params and merge a second transaction into `applyTransactionOnDraw`, overwriting the first frame before it was ever drawn. `PlatformViewsController2` now defers a second frame's view operations and transactions into a `QueuedFrame` while a frame is already staged, and replays/commits it once the traversal finishes (using a `Choreographer.FrameCallback` that posts an async message to the front of the main `Handler` queue, plus a `flushCompletedTraversalIfNeeded()` check in case `MessageLoopAndroid` runs inside `MessageQueue.nativePollOnce()` right after `doFrame()` before that `Handler` message is dispatched). 3. When two transactions on the same `SurfaceControl` do get merged before a single draw (for example if 3+ frames arrive within one VSync and coalesce in `QueuedFrame`), the later transaction's buffer overwrites the earlier one, but Android's `SurfaceComposerClient` retains both transaction completion callbacks in an `unordered_set` and can invoke the newer callback before the older, superseded callback. `AHBSwapchainImplVK` now tracks a monotonic `present_id` so a superseded callback recycles its own dropped texture into `pool_` instead of displacing `currently_displayed_texture_`. --------- Co-authored-by: Gray Mackall <[email protected]>
Fixes a few related synchronization issues in HCPP when the raster thread gets ahead of the platform thread, or when two platform-thread frame tasks run before Android's
Choreographer.doFrame()has drawn the first one. Together, this effectively moves HCPP presentation from mailbox semantics (where a second frame arriving beforedoFrame()overwrote the pending frame inapplyTransactionOnDraw) to FIFO, aligning with core Flutter.The end result is that scrolling with multiple platform views looks a lot, lot smoother.
PlatformViewsController2.publishTransaction()appended every rasterSurfaceControl.Transactioninto a single flatpendingRasterTransactionslist, andswapTransactions()on the platform thread drained the entire list at once. BecauseSubmitFlutterView()posts its platform-thread task asynchronously after submitting the overlay and mainSurfaceFrames, the raster thread can start or finish frameN+1before frameN's platform task runsswapTransactions(). Draining the flat list caused frameNto merge frameN+1's transactions into itsonEndFrame(), pairing frameN+1's Flutter surface buffers with frameN's platform view layout (and leaving frameN+1's platform task without its transactions). This addssealFrameTransactions()and calls it on the raster thread at the end ofSubmitFlutterView()soswapTransactions()pops one sealed frame batch at a time.onEndFrame()callsflutterView.invalidate()androotSurfaceControl.applyTransactionOnDraw(tx), which defers drawing the view hierarchy and applying the transaction until the upcomingChoreographertraversal. If a second frame's platform task runs before that traversal happens, it would immediately mutate the platform view layout params and merge a second transaction intoapplyTransactionOnDraw, overwriting the first frame before it was ever drawn.PlatformViewsController2now defers a second frame's view operations and transactions into aQueuedFramewhile a frame is already staged, and replays/commits it once the traversal finishes (using aChoreographer.FrameCallbackthat posts an async message to the front of the mainHandlerqueue, plus aflushCompletedTraversalIfNeeded()check in caseMessageLoopAndroidruns insideMessageQueue.nativePollOnce()right afterdoFrame()before thatHandlermessage is dispatched).SurfaceControldo get merged before a single draw (for example if 3+ frames arrive within one VSync and coalesce inQueuedFrame), the later transaction's buffer overwrites the earlier one, but Android'sSurfaceComposerClientretains both transaction completion callbacks in anunordered_setand can invoke the newer callback before the older, superseded callback.AHBSwapchainImplVKnow tracks a monotonicpresent_idso a superseded callback recycles its own dropped texture intopool_instead of displacingcurrently_displayed_texture_.