Skip to content

Commit b1ea363

Browse files
authored
[android]: prewarms shaders when the context is created (flutter#196069)
This preemptively starts compiling shaders as soon as the Context is created instead of waiting for the surface to be created. This has been measured to reduce startup time for mokey by 160ms. One gotcha is that on vulkan the pixel format is compiled into the shaders, getting it wrong is very wasteful. Today we only use r8g8b8a8, but in the future that may change when we support wide gamut. I've added a debug assert to make sure what we warm up is what we use, which i'm pretty sure will be correct. See the linked issue for more info ( flutter#195920 ) fixes flutter#195920 ### before <img src="https://private-user-images.githubusercontent.com/30870216/696606728-5df99815-84ba-4767-8652-f62b1868deab.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3OTQyNzExNTAsIm5iZiI6MTc5NDI3MDg1MCwicGF0aCI6Ii8zMDg3MDIxNi82OTY2MDY3MjgtNWRmOTk4MTUtODRiYS00NzY3LTg2NTItZjYyYjE4NjhkZWFiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjExMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYxMTEwVDAwMzQxMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTEwOTU4ZmY2ODY4M2U2MGM2NWYwODdmMTVmNzVkMjBlZmE2NGVhOWViYmNjNDE1NjE0NzE1ZjdmOTEyOWNhZGMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.Pby0uXVIz5VeI6D2yIjY7esMaH_YFrXLyxvRm6C2UgY"> ### after <img src="https://private-user-images.githubusercontent.com/30870216/696783031-e835a389-c85a-43c4-8b82-2624073a351e.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3OTQyNzExNTEsIm5iZiI6MTc5NDI3MDg1MSwicGF0aCI6Ii8zMDg3MDIxNi82OTY3ODMwMzEtZTgzNWEzODktYzg1YS00M2M0LThiODItMjYyNDA3M2EzNTFlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjExMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYxMTEwVDAwMzQxMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU2M2FmNWFmOTkzZGFjOTM3Mzg1MjU5NzEzNmViZmZmNTBiYWM4MWYyOTI0ZjU5NGZhNTFiZWZhNzU1YjRjNzYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.OYVGbEZQScWwQ4R1z8vA_2HqSbfvIH5W_2qRE4w4GuA"> ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent a07bf69 commit b1ea363

9 files changed

Lines changed: 680 additions & 224 deletions

File tree

‎engine/src/flutter/impeller/entity/BUILD.gn‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -286,6 +286,7 @@ impeller_component("entity_unittests") {
286286

287287
sources = [
288288
"clip_stack_unittests.cc",
289+
"contents/content_context_unittests.cc",
289290
"contents/filters/blend_filter_contents_unittests.cc",
290291
"contents/filters/gaussian_blur_filter_contents_unittests.cc",
291292
"contents/filters/inputs/filter_input_unittests.cc",

‎engine/src/flutter/impeller/entity/contents/content_context.cc‎

Lines changed: 242 additions & 223 deletions
Large diffs are not rendered by default.

‎engine/src/flutter/impeller/entity/contents/content_context.h‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,26 @@ class ContentContext {
129129

130130
~ContentContext();
131131

132+
//----------------------------------------------------------------------------
133+
/// @brief Requests the default pipelines that every `ContentContext`
134+
/// creates, without creating a `ContentContext`.
135+
///
136+
/// The pipelines are requested asynchronously from the pipeline
137+
/// library of `context`, which keeps them for the life of the
138+
/// context. `ContentContext`s created with `context` afterwards
139+
/// find them there, compiled or in flight, instead of compiling
140+
/// them again. This lets embedders start compiling pipelines as
141+
/// soon as a context is available, before there is a surface to
142+
/// render to.
143+
///
144+
/// This must be called on the thread that the `ContentContext`s
145+
/// for `context` are used on, since some pipeline libraries
146+
/// (OpenGL ES) are not thread-safe.
147+
///
148+
/// @param[in] context The context whose pipeline library to warm.
149+
///
150+
static void WarmDefaultPipelines(const Context& context);
151+
132152
bool IsValid() const;
133153

134154
Tessellator& GetTessellator() const;
@@ -362,6 +382,14 @@ class ContentContext {
362382
struct Pipelines;
363383
std::unique_ptr<Pipelines> pipelines_;
364384

385+
/// Requests the default variant of every pipeline from the pipeline library
386+
/// of `context`.
387+
///
388+
/// @return The pipelines holding the default variants, or nullptr if a
389+
/// default pipeline descriptor could not be created.
390+
static std::unique_ptr<Pipelines> CreateDefaultPipelines(
391+
const Context& context);
392+
365393
bool is_valid_ = false;
366394
std::shared_ptr<Tessellator> tessellator_;
367395
std::shared_ptr<RenderTargetAllocator> render_target_cache_;
Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
// Copyright 2013 The Flutter Authors. All rights reserved.
2+
// Use of this source code is governed by a BSD-style license that can be
3+
// found in the LICENSE file.
4+
5+
#include "impeller/entity/contents/content_context.h"
6+
7+
#include <memory>
8+
#include <string>
9+
#include <string_view>
10+
#include <unordered_set>
11+
#include <vector>
12+
13+
#include "gmock/gmock.h"
14+
#include "gtest/gtest.h"
15+
#include "impeller/base/comparable.h"
16+
#include "impeller/base/promise.h"
17+
#include "impeller/renderer/capabilities.h"
18+
#include "impeller/renderer/pipeline.h"
19+
#include "impeller/renderer/pipeline_descriptor.h"
20+
#include "impeller/renderer/testing/mocks.h"
21+
22+
namespace impeller {
23+
namespace testing {
24+
25+
namespace {
26+
27+
using ::testing::_;
28+
using ::testing::AtLeast;
29+
using ::testing::Contains;
30+
using ::testing::NiceMock;
31+
using ::testing::Return;
32+
using ::testing::ReturnRef;
33+
34+
PipelineFuture<PipelineDescriptor> MakeRealizedPipelineFuture(
35+
PipelineDescriptor descriptor) {
36+
return {
37+
std::move(descriptor),
38+
RealizedFuture<std::shared_ptr<Pipeline<PipelineDescriptor>>>(nullptr)};
39+
}
40+
41+
/// A context with no GPU behind it. It resolves every shader and hands out a
42+
/// mock pipeline library that records each pipeline requested from it.
43+
class ContentContextTest : public ::testing::Test {
44+
public:
45+
void SetUp() override {
46+
capabilities_ =
47+
CapabilitiesBuilder()
48+
.SetDefaultColorFormat(PixelFormat::kR8G8B8A8UNormInt)
49+
.SetDefaultDepthStencilFormat(PixelFormat::kD24UnormS8Uint)
50+
.SetDefaultGlyphAtlasFormat(PixelFormat::kA8UNormInt)
51+
.Build();
52+
ON_CALL(*context_, IsValid()).WillByDefault(Return(true));
53+
ON_CALL(*context_, GetBackendType())
54+
.WillByDefault(Return(Context::BackendType::kVulkan));
55+
ON_CALL(*context_, GetCapabilities())
56+
.WillByDefault(ReturnRef(capabilities_));
57+
ON_CALL(*context_, GetShaderLibrary())
58+
.WillByDefault(Return(shader_library_));
59+
ON_CALL(*context_, GetPipelineLibrary())
60+
.WillByDefault(Return(pipeline_library_));
61+
// Resolve every entrypoint, as if every shader had been compiled in.
62+
ON_CALL(*shader_library_, GetFunction(_, _))
63+
.WillByDefault([](std::string_view name, ShaderStage stage) {
64+
return std::make_shared<MockShaderFunction>(std::string(name), stage);
65+
});
66+
ON_CALL(*pipeline_library_, GetPipeline(_, _, _))
67+
.WillByDefault([this](const PipelineDescriptor& descriptor,
68+
bool /*async*/, bool /*threadsafe*/) {
69+
requested_.push_back(descriptor);
70+
return MakeRealizedPipelineFuture(descriptor);
71+
});
72+
}
73+
74+
protected:
75+
std::shared_ptr<const Capabilities> capabilities_;
76+
std::shared_ptr<NiceMock<MockShaderLibrary>> shader_library_ =
77+
std::make_shared<NiceMock<MockShaderLibrary>>();
78+
std::shared_ptr<NiceMock<MockPipelineLibrary>> pipeline_library_ =
79+
std::make_shared<NiceMock<MockPipelineLibrary>>();
80+
std::shared_ptr<NiceMock<MockImpellerContext>> context_ =
81+
std::make_shared<NiceMock<MockImpellerContext>>();
82+
/// Every descriptor requested from `pipeline_library_`, in order.
83+
std::vector<PipelineDescriptor> requested_;
84+
};
85+
86+
} // namespace
87+
88+
TEST_F(ContentContextTest, WarmDefaultPipelinesRequestsDefaultPipelines) {
89+
ContentContext::WarmDefaultPipelines(*context_);
90+
91+
std::vector<std::string> labels;
92+
labels.reserve(requested_.size());
93+
for (const PipelineDescriptor& descriptor : requested_) {
94+
labels.emplace_back(descriptor.GetLabel());
95+
}
96+
EXPECT_THAT(labels, Contains("SolidFill Pipeline"));
97+
EXPECT_THAT(labels, Contains("GlyphAtlas Pipeline"));
98+
EXPECT_THAT(labels, Contains("TextureFill Pipeline"));
99+
}
100+
101+
TEST_F(ContentContextTest, WarmDefaultPipelinesRequestsPipelinesAsync) {
102+
// Compiling synchronously would stall the calling thread, and on OpenGL ES
103+
// would need a context to be current on it.
104+
EXPECT_CALL(*pipeline_library_, GetPipeline(_, /*async=*/false, _)).Times(0);
105+
EXPECT_CALL(*pipeline_library_, GetPipeline(_, /*async=*/true, _))
106+
.Times(AtLeast(1));
107+
108+
ContentContext::WarmDefaultPipelines(*context_);
109+
}
110+
111+
TEST_F(ContentContextTest, WarmDefaultPipelinesRequestsEachPipelineOnce) {
112+
ContentContext::WarmDefaultPipelines(*context_);
113+
114+
ASSERT_FALSE(requested_.empty());
115+
std::unordered_set<PipelineDescriptor, ComparableHash<PipelineDescriptor>,
116+
ComparableEqual<PipelineDescriptor>>
117+
unique(requested_.begin(), requested_.end());
118+
EXPECT_EQ(unique.size(), requested_.size());
119+
}
120+
121+
TEST_F(ContentContextTest, WarmDefaultPipelinesDoesNoGpuWork) {
122+
// Embedders call this before there is a surface, when no rendering context
123+
// may be current. It must only request pipelines.
124+
EXPECT_CALL(*context_, GetResourceAllocator()).Times(0);
125+
EXPECT_CALL(*context_, CreateCommandBuffer()).Times(0);
126+
EXPECT_CALL(*context_, GetCommandQueue()).Times(0);
127+
128+
ContentContext::WarmDefaultPipelines(*context_);
129+
130+
EXPECT_FALSE(requested_.empty());
131+
}
132+
133+
TEST_F(ContentContextTest, WarmDefaultPipelinesIgnoresInvalidContext) {
134+
ON_CALL(*context_, IsValid()).WillByDefault(Return(false));
135+
EXPECT_CALL(*pipeline_library_, GetPipeline(_, _, _)).Times(0);
136+
137+
ContentContext::WarmDefaultPipelines(*context_);
138+
}
139+
140+
} // namespace testing
141+
} // namespace impeller

‎engine/src/flutter/impeller/renderer/testing/mocks.h‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@
1919
#include "impeller/renderer/render_pass.h"
2020
#include "impeller/renderer/render_target.h"
2121
#include "impeller/renderer/sampler_library.h"
22+
#include "impeller/renderer/shader_function.h"
23+
#include "impeller/renderer/shader_library.h"
2224

2325
#define FLT_FORWARD(mock, real, method) \
2426
EXPECT_CALL(*mock, method()) \
@@ -343,6 +345,25 @@ class MockSampler : public Sampler {
343345
explicit MockSampler(const SamplerDescriptor& desc) : Sampler(desc) {}
344346
};
345347

348+
class MockShaderFunction : public ShaderFunction {
349+
public:
350+
MockShaderFunction(std::string name, ShaderStage stage)
351+
: ShaderFunction(UniqueID{}, std::move(name), stage) {}
352+
};
353+
354+
class MockShaderLibrary : public ShaderLibrary {
355+
public:
356+
MOCK_METHOD(bool, IsValid, (), (const, override));
357+
MOCK_METHOD(std::shared_ptr<const ShaderFunction>,
358+
GetFunction,
359+
(std::string_view name, ShaderStage stage),
360+
(override));
361+
MOCK_METHOD(void,
362+
UnregisterFunction,
363+
(std::string name, ShaderStage stage),
364+
(override));
365+
};
366+
346367
} // namespace testing
347368
} // namespace impeller
348369

‎engine/src/flutter/shell/platform/android/BUILD.gn‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ executable("flutter_shell_native_unittests") {
6262
deps = [
6363
":flutter_shell_native_src",
6464
"//flutter/fml",
65+
"//flutter/impeller/renderer/testing:mocks",
6566
"//flutter/shell/platform/android/jni:jni_mock",
6667
"//flutter/shell/platform/android/jni:mock_jni_env",
6768
"//flutter/third_party/googletest:gmock",

‎engine/src/flutter/shell/platform/android/platform_view_android.cc‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434

3535
#include "fml/logging.h"
3636
#include "impeller/display_list/aiks_context.h"
37+
#include "impeller/entity/contents/content_context.h"
3738
#if IMPELLER_ENABLE_VULKAN // b/258506856 for why this is behind an if
3839
#include "flutter/shell/platform/android/android_surface_vk_impeller.h"
3940
#include "flutter/shell/platform/android/image_external_texture_vk_impeller.h"
@@ -427,8 +428,34 @@ std::unique_ptr<Surface> PlatformViewAndroid::CreateRenderingSurface() {
427428
if (!android_surface_) {
428429
return nullptr;
429430
}
430-
return android_surface_->CreateGPUSurface(
431+
std::unique_ptr<Surface> surface = android_surface_->CreateGPUSurface(
431432
android_context_->GetMainSkiaContext().get());
433+
// Creating the GPU surface sets up the swapchain, which is what may change
434+
// the context's default color format.
435+
CheckWarmedColorFormat();
436+
return surface;
437+
}
438+
439+
void PlatformViewAndroid::CheckWarmedColorFormat() const {
440+
if (!warmed_color_format_.has_value()) {
441+
return;
442+
}
443+
std::shared_ptr<impeller::Context> impeller_context =
444+
android_context_->GetImpellerContext();
445+
if (!impeller_context) {
446+
return;
447+
}
448+
// Only read by the DCHECK below, which compiles out of release builds.
449+
[[maybe_unused]] const impeller::PixelFormat surface_color_format =
450+
impeller_context->GetCapabilities()->GetDefaultColorFormat();
451+
FML_DCHECK(surface_color_format == warmed_color_format_.value())
452+
<< "The default pipelines were warmed for "
453+
<< impeller::PixelFormatToString(warmed_color_format_.value())
454+
<< " but the surface renders to "
455+
<< impeller::PixelFormatToString(surface_color_format)
456+
<< ", so they will be compiled again. Reevaluate the call to "
457+
"ContentContext::WarmDefaultPipelines in "
458+
"PlatformViewAndroid::SetupImpellerContext.";
432459
}
433460

434461
// |PlatformView|
@@ -567,6 +594,23 @@ bool PlatformViewAndroid::IsSurfaceControlEnabled() const {
567594
void PlatformViewAndroid::SetupImpellerContext() {
568595
android_context_->SetupImpellerContext();
569596
android_surface_->SetupImpellerSurface();
597+
598+
// Start compiling the default pipelines now, on the raster thread where the
599+
// surface's ContentContext will later look them up, instead of waiting for
600+
// the first surface to be created. Compilation happens off this thread (on
601+
// the IO thread for OpenGL ES, on worker threads for Vulkan), so no rendering
602+
// context needs to be current here.
603+
//
604+
// The pipelines are keyed on the context's default color format, which the
605+
// Vulkan swapchain may change once a surface exists. On Android it is always
606+
// R8G8B8A8, so the pipelines warmed here are the ones the surface uses.
607+
// |CheckWarmedColorFormat| asserts that this stays true.
608+
if (std::shared_ptr<impeller::Context> impeller_context =
609+
android_context_->GetImpellerContext()) {
610+
warmed_color_format_ =
611+
impeller_context->GetCapabilities()->GetDefaultColorFormat();
612+
impeller::ContentContext::WarmDefaultPipelines(*impeller_context);
613+
}
570614
}
571615

572616
} // namespace flutter

‎engine/src/flutter/shell/platform/android/platform_view_android.h‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,13 @@
66
#define FLUTTER_SHELL_PLATFORM_ANDROID_PLATFORM_VIEW_ANDROID_H_
77

88
#include <memory>
9+
#include <optional>
910
#include <string>
1011
#include <vector>
1112

1213
#include <android/hardware_buffer_jni.h>
1314
#include "flutter/fml/platform/android/scoped_java_ref.h"
15+
#include "flutter/impeller/core/formats.h"
1416
#include "flutter/lib/ui/window/platform_message.h"
1517
#include "flutter/shell/common/platform_view.h"
1618
#include "flutter/shell/common/snapshot_surface_producer.h"
@@ -137,6 +139,14 @@ class PlatformViewAndroid final : public PlatformView {
137139
std::unique_ptr<AndroidSurface> android_surface_;
138140
std::shared_ptr<PlatformMessageHandlerAndroid> platform_message_handler_;
139141
bool android_meets_hcpp_criteria_ = false;
142+
// The color format that |SetupImpellerContext| warmed the default pipelines
143+
// for, if it warmed them. Raster thread only.
144+
std::optional<impeller::PixelFormat> warmed_color_format_;
145+
146+
// Asserts that the surface renders to the color format that the default
147+
// pipelines were warmed for. Otherwise the warm-up was wasted. Raster thread
148+
// only.
149+
void CheckWarmedColorFormat() const;
140150

141151
// |PlatformView|
142152
void UpdateSemantics(

0 commit comments

Comments
 (0)