Re-prioritize pipeline compile jobs and perform them eagerly instead of waiting. by chinmaygarde · Pull Request #180022 · flutter/flutter

chinmaygarde · 2025-12-17T21:36:51Z

Also, removes "lazy shader mode" and re-enables eager PSO pre-flight.

Impeller eagerly sets up the entire set of pipeline state objects (PSOs) it may need during renderer setup. This works fine on mid to high-end devices as the setup completes well before the first frame is rendered.

However on low-end devices, not all PSOs may be ready before the first frame is rendered. Low-end device usually don't have as much available concurrency and are slower to boot. On these devices, the rendering thread had to wait for the background compile job to be completed. It was also observed that the relatively higher priority render thread was waiting on a background thread to finish a PSO compile job. The PSO compile job could also be stuck behind another job that was not immediately needed. This made the situation on low end devices less than ideal.

To ameliorate this situation, a stop-gap was introduced called "lazy shader mode". This disabled PSO pre-flight entirely and only dispatched jobs when they were needed. This ameliorated somewhat the issue of unneeded jobs blocking the eagerly needed job but the other issues remained. It also meant that one could expect jank the first time a new PSO was needed. This mode was not widely used or advertised.

In this patch, the lazy shader mode has been entirely removed and eager PSO pre-flight is the default in all configurations. Pipeline compile jobs are now handled by a separate compile queue. This queue operates like a transparent concurrent worker pool job dispatcher in the usual case. However, when the pipeline for a specific descriptor is needed, instead of waiting for the job to complete, the queue allows the job for that descriptor to skip to the head of the queue and be performed on the calling thread. This effectively make the calling thread do the job instead of waiting. Another nice side effect of this behavior is that the higher priority thread is now doing the job it was previously waiting on.

This fixes all issues on lower end devices except one. During eager pre-flight, it would still be better if PSO compile jobs for that were somehow divined to be needed first were towards the front of the pre-flight queue. After all, only a handful of rendering features account for the majority of rendering operations. A future augmentation to the compile queue would be to allow higher prioritization of more widely used pipeline compile jobs as instrumented in #176660. This task is tracked in #176665.

This patch only wires up the compile queue for Vulkan. #176657 tracks doing the same for OpenGL.

The efficacy of the re-prioritizations can be traced using the
PrioritiesElevated trace counter. On higher end devices, there should
be zero of these.

Fixes #176656
Fixes #176663
Fixes #176658

flutter-dashboard · 2025-12-17T21:36:55Z

It looks like this pull request may not have tests. Please make sure to add tests or get an explicit test exemption before merging.

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. If you believe this PR qualifies for a test exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

gemini-code-assist

Code Review

This pull request introduces a PipelineCompileQueue to re-prioritize pipeline compilation jobs, allowing them to be performed eagerly instead of waiting. This change removes the old "lazy shader mode" and aims to improve performance on low-end devices by reducing jank caused by waiting for Pipeline State Objects (PSOs). The implementation is currently wired up for Vulkan. The overall approach is sound and the new queue is well-encapsulated. I've provided a few suggestions to improve the implementation of the new PipelineCompileQueue class, focusing on C++ best practices and efficiency.

gemini-code-assist · 2025-12-17T21:40:00Z

+std::shared_ptr<PipelineCompileQueue> PipelineCompileQueue::Create(
+    std::shared_ptr<fml::ConcurrentTaskRunner> worker_task_runner) {
+  return std::shared_ptr<PipelineCompileQueue>(
+      new PipelineCompileQueue(std::move(worker_task_runner)));
+}


The Google C++ Style Guide, which this repository follows, recommends using std::make_shared instead of std::shared_ptr(new ...) for better performance and exception safety. std::make_shared performs a single memory allocation for both the object and its control block.

To adopt this, you can make the PipelineCompileQueue constructor public. The static Create method will still serve as the designated factory function.

return std::make_shared<PipelineCompileQueue>(std::move(worker_task_runner));

References

C++ code should follow the Google C++ Style Guide. ^(link)

Can't do gem gem. That method is private.

And the suggestion of making the ctor public is a no-go because std::enable_shared_from_this cannot be enforced if someone decides to create a non-shared-pointer owned instance of the queue. The weak_from_this will crash.

gemini-code-assist · 2025-12-17T21:40:00Z

+  if (!insertion_result.second) {
+    // This bit is being extremely conservative. If insertion did not take
+    // place, someone gave the compile queue a job for the same description.
+    // This is highly unusual but technically not impossible. Just run the job
+    // eagerly.
+    FML_LOG(ERROR) << "Got multiple compile jobs for the same descriptor. "
+                      "Running eagerly.";
+    // Don't invoke the job here has there are we have currently acquired a
+    // mutex.
+    worker_task_runner_->PostTask(job);
+    return true;
+  }


The current logic for handling a duplicate job descriptor logs an error and then posts the job again. This could lead to redundant work if the original job is already in the queue. Since PipelineLibraryVK should prevent duplicate job submissions for the same descriptor, this case is rare. However, if it does occur, it would be safer and more efficient to simply ignore the duplicate request. The original job will eventually complete, and the caller will receive the result via the shared future.

if (!insertion_result.second) { // This bit is being extremely conservative. If insertion did not take // place, someone gave the compile queue a job for the same description. // This is highly unusual but technically not impossible. The job is // already in the queue, so we can ignore this request. return true; }

then posts the job again

Yes, but not to the queue.

it would be safer and more efficient to simply ignore the duplicate request

More efficient yes. But safer, not really. What if the two different futures are awaited upon for the same descriptor with different jobs? As mentioned, this is a rare edge case. But this will limit the damage (and log) in such cases instead of causing a deadlock.

gemini-code-assist · 2025-12-17T21:40:00Z

+void PipelineCompileQueue::FinishAllJobs() {
+  // This doesn't have to be fast. Just ensures the task queue is flushed when
+  // the compile queue is shutting down with jobs still in it.
+  while (true) {
+    bool has_jobs = false;
+    {
+      Lock lock(pending_jobs_mutex_);
+      has_jobs = !pending_jobs_.empty();
+    }
+    if (!has_jobs) {
+      return;
+    }
+    DoOneJob();
+  }
+}


The current implementation of FinishAllJobs can be inefficient as it repeatedly locks and unlocks the mutex for each job in the queue. A more efficient approach would be to lock the mutex once, move all pending jobs to a local container using std::swap, and then execute them after releasing the lock. This minimizes lock contention, especially if there are many jobs to process during shutdown.

void PipelineCompileQueue::FinishAllJobs() { // This doesn't have to be fast. Just ensures the task queue is flushed when // the compile queue is shutting down with jobs still in it. decltype(pending_jobs_) jobs_to_run; { Lock lock(pending_jobs_mutex_); jobs_to_run.swap(pending_jobs_); } for (const auto& job : jobs_to_run) { job.second(); } }

My reasoning for doing it this way was that any background task runners still running would still be able to participate in job selection. I agree that from the perspective of holding onto the mutex, doing it one shot is better.

Add a comment noting that worker threads may still be running jobs during and after the execution of FinishAllJobs.

jason-simmons

Overall LGTM

jason-simmons · 2025-12-17T23:21:01Z

+    return false;
+  }
+
+  Lock lock(pending_jobs_mutex_);


Reduce the section that holds the lock to just the pending_jobs_.insert call

jason-simmons · 2025-12-17T23:23:40Z

+  if (pending_jobs_.empty()) {
+    return nullptr;
+  }
+  auto job_iterator = pending_jobs_.begin();


Should PipelineCompileQueue track the insertion order of the jobs?

The Queue naming and the mention of prioritization imply that jobs will be executed in the order they were posted. But TakeNextJob obtains the job from a std::unordered_map iterator that will select an arbitrary job.

My sense was that it doesn't matter. At least not till we implement #176665 which I have left as a follow up task.

jason-simmons · 2025-12-17T23:26:56Z

+void PipelineCompileQueue::FinishAllJobs() {
+  // This doesn't have to be fast. Just ensures the task queue is flushed when
+  // the compile queue is shutting down with jobs still in it.
+  while (true) {
+    bool has_jobs = false;
+    {
+      Lock lock(pending_jobs_mutex_);
+      has_jobs = !pending_jobs_.empty();
+    }
+    if (!has_jobs) {
+      return;
+    }
+    DoOneJob();
+  }
+}


Add a comment noting that worker threads may still be running jobs during and after the execution of FinishAllJobs.

zanderso · 2025-12-19T21:20:55Z

Looks like some Impeller/Vulkan unit tests are unhappy with the changes to pipeline lifetime?

…of waiting. Also, removes "lazy shader mode" and re-enables eager PSO pre-flight. Impeller eagerly sets up the entire set of pipeline state objects (PSOs) it may need during renderer setup. This works fine on mid to high-end devices as the setup completes well before the first frame is rendered. However on low-end devices, not all PSOs may be ready before the first frame is rendered. Low-end device usually don't have as much available concurrency and are slower to boot. On these devices, the rendering thread had to wait for the background compile job to be completed. It was also observed that the relatively higher priority render thread was waiting on a background thread to finish a PSO compile job. The PSO compile job could also be stuck behind another job that was not immediately needed. This made the situation on low end devices less than ideal. To ameliorate this situation, a stop-gap was introduced called "lazy shader mode". This disabled PSO pre-flight entirely and only dispatched jobs when they were needed. This ameliorated somewhat the issue of unneeded jobs blocking the eagerly needed job but the other issues remained. It also meant that one could expect jank the first time a new PSO was needed. This mode was not widely used or advertised. In this patch, the lazy shader mode has been entirely removed and eager PSO pre-flight is the default in all configurations. Pipeline compile jobs are now handled by a separate compile queue. This queue operates like a transparent concurrent worker pool job dispatcher in the usual case. However, when the pipeline for a specific descriptor is needed, instead of waiting for the job to complete, the queue allows the job for that descriptor to skip to the head of the queue and be performed on the calling thread. This effectively make the calling thread do the job instead of waiting. Another nice side effect of this behavior is that the higher priority thread is now doing the job it was previously waiting on. This fixes all issues on lower end devices except one. During eager pre-flight, it would still be better if PSO compile jobs for that were somehow divined to be needed first were towards the front of the pre-flight queue. After all, only a handful of rendering features account for the majority of rendering operations. A future augmentation to the compile queue would be to allow higher prioritization of more widely used pipeline compile jobs as instrumented in flutter#176660. This task is tracked in flutter#176665. This patch only wires up the compile queue for Vulkan. flutter#176657 tracks doing the same for OpenGL. Fixes flutter#176656 Fixes flutter#176663 Fixes flutter#176658

chinmaygarde · 2026-01-09T21:19:20Z

Okie dokie. All tests are passing now. The failures were due to the extra logging. When we tore down the renderer without all compile jobs finishing, the aborted jobs would log an error. Before this patch, that was an error because shutdown could only cleanly happen after a clean startup, and a clean startup depended on all compile jobs finishing. This is no longer the case. The upside is also that tests that startup and shutdown the renderer quickly should not finish faster.

chinmaygarde · 2026-01-09T21:47:45Z

The one Google testing failure doesn't seem related. Some sort of a bluetooth scan failing with a timeout. I think this is good to go.

…instead of waiting. (flutter/flutter#180022)

Roll Flutter from 01d37bc to 73769a2 (65 revisions) flutter/flutter@01d37bc...73769a2 2026-01-10 [email protected] Roll Dart SDK from 5e855c2bb3ef to 87fbfd5381b6 (1 revision) (flutter/flutter#180800) 2026-01-10 [email protected] Roll Skia from b2b109f0e980 to f39cc645b1dd (2 revisions) (flutter/flutter#180796) 2026-01-10 [email protected] Roll Dart SDK from b7963905e6a2 to 5e855c2bb3ef (2 revisions) (flutter/flutter#180794) 2026-01-10 [email protected] Make sure that a CupertinoTabScaffold doesn't crash in 0x0 environment (flutter/flutter#179824) 2026-01-10 [email protected] Roll Dart SDK from d25ad331b7ea to b7963905e6a2 (2 revisions) (flutter/flutter#180783) 2026-01-10 [email protected] Make sure that a Container doesn't crash in 0x0 environment (flutter/flutter#180350) 2026-01-10 [email protected] Make sure that an Expansible doesn't crash in 0x0 environment (flutter/flutter#180478) 2026-01-10 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Enabled some disabled impeller fragment shader dart tests (#180759)" (flutter/flutter#180785) 2026-01-09 [email protected] Merge `widget_tester_leaks_free_test.dart` into `widget_tester_test.dart` (flutter/flutter#180600) 2026-01-09 [email protected] fix: there are no riscv fuchsia artifacts (flutter/flutter#180779) 2026-01-09 [email protected] Roll Skia from 7386219151e6 to b2b109f0e980 (1 revision) (flutter/flutter#180771) 2026-01-09 [email protected] Re-prioritize pipeline compile jobs and perform them eagerly instead of waiting. (flutter/flutter#180022) 2026-01-09 [email protected] Enabled some disabled impeller fragment shader dart tests (flutter/flutter#180759) 2026-01-09 [email protected] Roll Fuchsia Linux SDK from rxeg-6UB678HKJ4UQ... to 83Favz_zzMzdVuOHg... (flutter/flutter#180765) 2026-01-09 [email protected] [A11y ] Add `clearSemantics`in table (flutter/flutter#180665) 2026-01-09 [email protected] Update CODEOWNERS to remove chinmaygarde. (flutter/flutter#180703) 2026-01-09 [email protected] [ Tool ] Add support for linux riscv64 architecture (flutter/flutter#178711) 2026-01-09 [email protected] Roll Skia from e9b3264ade0c to 7386219151e6 (12 revisions) (flutter/flutter#180754) 2026-01-09 [email protected] Roll Dart SDK from fe2ba2c5dd50 to d25ad331b7ea (10 revisions) (flutter/flutter#180741) 2026-01-09 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Unpin google_mobile_ads (#180573)" (flutter/flutter#180761) 2026-01-09 [email protected] Fix typo in dropdown_menu.dart (flutter/flutter#180172) 2026-01-09 [email protected] Roll Packages from 039a026 to 51fe1d9 (1 revision) (flutter/flutter#180742) 2026-01-09 [email protected] Unpin google_mobile_ads (flutter/flutter#180573) 2026-01-09 [email protected] Update flutter changelog for 3.38.6 (flutter/flutter#180708) 2026-01-08 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Fix iOS xattr removal to clear all extended attributes (#180355)" (flutter/flutter#180709) 2026-01-08 [email protected] Make sure that an EditableText doesn't crash in 0x0 environment (flutter/flutter#180457) 2026-01-08 [email protected] Implementation of tooltip windows for win32 (flutter/flutter#179147) 2026-01-08 [email protected] [web] Don't serve files outside of project (flutter/flutter#180699) 2026-01-08 [email protected] Roll Skia from 837be28dd218 to e9b3264ade0c (1 revision) (flutter/flutter#180702) 2026-01-08 [email protected] Add new motion accessibility features to iOS. (flutter/flutter#178102) 2026-01-08 [email protected] Fix iOS xattr removal to clear all extended attributes (flutter/flutter#180355) 2026-01-08 [email protected] [ Widget Preview ] Move widget_preview_scaffold tests to `dev/integration_tests/widget_preview_scaffold` (flutter/flutter#180658) 2026-01-08 [email protected] De-interleaves engine dart test output (flutter/flutter#180651) 2026-01-08 [email protected] [web] Fix SemanticsService.announce not working inside dialogs (flutter/flutter#179958) 2026-01-08 [email protected] Roll Skia from 42233226ac56 to 837be28dd218 (2 revisions) (flutter/flutter#180693) 2026-01-08 [email protected] Roll Dart SDK to 3.11.0-296.2.beta (flutter/flutter#180685) 2026-01-08 [email protected] Improve code quality in `AndroidTouchProcessorTest.java` (flutter/flutter#180583) 2026-01-08 [email protected] Revert "Reverts "[reland] Unify canvas creation and Surface code in S…kwasm and CanvasKit (#179473)" (#180152)" (flutter/flutter#180610) 2026-01-08 [email protected] Roll Skia from a0c407bce408 to 42233226ac56 (4 revisions) (flutter/flutter#180688) 2026-01-08 [email protected] [ Tool ] Fix flake in overall_experience_test.dart (flutter/flutter#180655) 2026-01-08 [email protected] Roll Packages from 9705815 to 039a026 (6 revisions) (flutter/flutter#180684) 2026-01-08 [email protected] flutter_tools: Auto-generate ExportOptions.plist for manual iOS code signing (flutter/flutter#177888) 2026-01-08 [email protected] Roll Skia from 58837e160874 to a0c407bce408 (2 revisions) (flutter/flutter#180679) 2026-01-08 [email protected] Roll Skia from 1e3266fdba86 to 58837e160874 (1 revision) (flutter/flutter#180677) 2026-01-08 [email protected] Roll Skia from 3c47ea10638f to 1e3266fdba86 (4 revisions) (flutter/flutter#180675) 2026-01-08 [email protected] Roll Fuchsia Linux SDK from dTvN_JVSCfGFRasvH... to rxeg-6UB678HKJ4UQ... (flutter/flutter#180673) ...

…of waiting. (flutter#180022) Also, removes "lazy shader mode" and re-enables eager PSO pre-flight. Impeller eagerly sets up the entire set of pipeline state objects (PSOs) it may need during renderer setup. This works fine on mid to high-end devices as the setup completes well before the first frame is rendered. However on low-end devices, not all PSOs may be ready before the first frame is rendered. Low-end device usually don't have as much available concurrency and are slower to boot. On these devices, the rendering thread had to wait for the background compile job to be completed. It was also observed that the relatively higher priority render thread was waiting on a background thread to finish a PSO compile job. The PSO compile job could also be stuck behind another job that was not immediately needed. This made the situation on low end devices less than ideal. To ameliorate this situation, a stop-gap was introduced called "lazy shader mode". This disabled PSO pre-flight entirely and only dispatched jobs when they were needed. This ameliorated somewhat the issue of unneeded jobs blocking the eagerly needed job but the other issues remained. It also meant that one could expect jank the first time a new PSO was needed. This mode was not widely used or advertised. In this patch, the lazy shader mode has been entirely removed and eager PSO pre-flight is the default in all configurations. Pipeline compile jobs are now handled by a separate compile queue. This queue operates like a transparent concurrent worker pool job dispatcher in the usual case. However, when the pipeline for a specific descriptor is needed, instead of waiting for the job to complete, the queue allows the job for that descriptor to skip to the head of the queue and be performed on the calling thread. This effectively make the calling thread do the job instead of waiting. Another nice side effect of this behavior is that the higher priority thread is now doing the job it was previously waiting on. This fixes all issues on lower end devices except one. During eager pre-flight, it would still be better if PSO compile jobs for that were somehow divined to be needed first were towards the front of the pre-flight queue. After all, only a handful of rendering features account for the majority of rendering operations. A future augmentation to the compile queue would be to allow higher prioritization of more widely used pipeline compile jobs as instrumented in flutter#176660. This task is tracked in flutter#176665. This patch only wires up the compile queue for Vulkan. flutter#176657 tracks doing the same for OpenGL. The efficacy of the re-prioritizations can be traced using the PrioritiesElevated trace counter. On higher end devices, there should be zero of these. Fixes flutter#176656 Fixes flutter#176663 Fixes flutter#176658

chinmaygarde requested review from jason-simmons and walley892 December 17, 2025 21:36

github-actions Bot added engine flutter/engine related. See also e: labels. e: impeller Impeller rendering backend issues and features requests labels Dec 17, 2025

gemini-code-assist Bot reviewed Dec 17, 2025

View reviewed changes

chinmaygarde force-pushed the parco branch from 95066f1 to d8e01b1 Compare December 17, 2025 21:41

jason-simmons reviewed Dec 17, 2025

View reviewed changes

chinmaygarde force-pushed the parco branch 2 times, most recently from e72f115 to 335b78a Compare December 18, 2025 22:41

chinmaygarde requested a review from a team as a code owner December 18, 2025 22:41

github-actions Bot added platform-android Android applications specifically team-android Owned by Android platform team labels Dec 18, 2025

jason-simmons approved these changes Dec 18, 2025

View reviewed changes

chinmaygarde force-pushed the parco branch from 335b78a to e52bf53 Compare January 9, 2026 19:14

chinmaygarde added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 9, 2026

auto-submit Bot added this pull request to the merge queue Jan 9, 2026

Merged via the queue into flutter:master with commit 07485a2 Jan 9, 2026
181 checks passed

flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 9, 2026

engine-flutter-autoroll mentioned this pull request Jan 10, 2026

Roll Flutter from 01d37bc80fd5 to 0b7596b3f269 (64 revisions) flutter/packages#10768

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 10, 2026

07485a2 Re-prioritize pipeline compile jobs and perform them eagerly …

a5510f4

…instead of waiting. (flutter/flutter#180022)

engine-flutter-autoroll mentioned this pull request Jan 10, 2026

Roll Flutter from 01d37bc80fd5 to 73769a220639 (65 revisions) flutter/packages#10769

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 10, 2026

07485a2 Re-prioritize pipeline compile jobs and perform them eagerly …

8c937cf

…instead of waiting. (flutter/flutter#180022)

engine-flutter-autoroll mentioned this pull request Jan 10, 2026

Roll Flutter from 01d37bc80fd5 to 73769a220639 (65 revisions) flutter/packages#10770

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 10, 2026

07485a2 Re-prioritize pipeline compile jobs and perform them eagerly …

3b592b3

…instead of waiting. (flutter/flutter#180022)

xiaowei-guan mentioned this pull request May 21, 2026

[Impeller] Use the IO context for OpenGL program setup flutter-tizen/flutter#35

Open

Conversation

chinmaygarde commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flutter-dashboard Bot commented Dec 17, 2025

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

chinmaygarde Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jason-simmons left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanderso commented Dec 19, 2025

Uh oh!

chinmaygarde commented Jan 9, 2026

Uh oh!

chinmaygarde commented Jan 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chinmaygarde commented Dec 17, 2025 •

edited

Loading

chinmaygarde Dec 17, 2025 •

edited

Loading