Skip to content

Commit 84a3314

Browse files
authored
[impeller:gles] fixes early calls to flatten scenes (flutter#194506)
This fix delays snapshot display_list creation and creates offscreen surface on demand to support flattening scenes before the surface has been created. This makes opengles match the behavior of vulkan and metal. fixes flutter#192365 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
1 parent 095babc commit 84a3314

15 files changed

Lines changed: 587 additions & 77 deletions

‎engine/src/flutter/lib/ui/BUILD.gn‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -308,6 +308,18 @@ if (enable_unittests) {
308308

309309
deps += [ "//flutter/testing:opengl" ]
310310
}
311+
312+
if (impeller_enable_opengles) {
313+
sources += [ "compositing/scene_impeller_gles_unittests.cc" ]
314+
315+
deps += [
316+
"//flutter/impeller/entity:entity_shaders",
317+
"//flutter/impeller/entity:framebuffer_blend_entity_shaders",
318+
"//flutter/impeller/entity:modern_entity_shaders",
319+
"//flutter/impeller/renderer/backend/gles:mock_gles",
320+
"//flutter/shell/gpu:gpu_surface_gl",
321+
]
322+
}
311323
}
312324

313325
source_set("display_list_deferred_image_unittests") {
Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
// Copyright 2013 The Flutter Authors. All rights reserved.
2+
// Use of this source code is governed by a BSD-style license that can be
3+
// found in the LICENSE file.
4+
5+
// Reproduces https://github.com/flutter/flutter/issues/192365 end to end:
6+
// `Scene.toImage` from Dart while the view has no window surface, rendered by
7+
// Impeller's OpenGL ES backend.
8+
//
9+
// The OpenGL ES driver is |MockGLES|, so this runs on any host. EGL is faked
10+
// just enough to model what matters on Android:
11+
// - Which thread, if any, has the context current. That is what decides
12+
// whether |ReactorGLES| may react.
13+
// - A |SnapshotSurfaceProducer| that "makes a pbuffer current" and then builds
14+
// a |GPUSurfaceGLImpeller|, like
15+
// `AndroidSurfaceGLImpeller::CreateSnapshotSurface`.
16+
17+
#include <atomic>
18+
#include <memory>
19+
#include <optional>
20+
#include <string>
21+
#include <thread>
22+
#include <vector>
23+
24+
#include "flutter/fml/mapping.h"
25+
#include "flutter/fml/synchronization/waitable_event.h"
26+
#include "flutter/shell/common/platform_view.h"
27+
#include "flutter/shell/common/shell_test.h"
28+
#include "flutter/shell/common/snapshot_surface_producer.h"
29+
#include "flutter/shell/gpu/gpu_surface_gl_delegate.h"
30+
#include "flutter/shell/gpu/gpu_surface_gl_impeller.h"
31+
#include "flutter/testing/testing.h"
32+
#include "gmock/gmock.h"
33+
#include "gtest/gtest.h"
34+
#include "impeller/entity/gles/entity_shaders_gles.h"
35+
#include "impeller/entity/gles/framebuffer_blend_shaders_gles.h"
36+
#include "impeller/entity/gles/modern_shaders_gles.h"
37+
#include "impeller/renderer/backend/gles/context_gles.h"
38+
#include "impeller/renderer/backend/gles/proc_table_gles.h"
39+
#include "impeller/renderer/backend/gles/reactor_gles.h"
40+
#include "impeller/renderer/backend/gles/test/mock_gles.h"
41+
#include "third_party/tonic/converter/dart_converter.h"
42+
43+
// CREATE_NATIVE_ENTRY is leaky by design
44+
// NOLINTBEGIN(clang-analyzer-core.StackAddressEscape)
45+
46+
namespace flutter {
47+
namespace testing {
48+
49+
namespace {
50+
51+
using ::testing::_;
52+
using ::testing::NiceMock;
53+
using ::testing::Return;
54+
55+
/// Models the one piece of EGL state that matters here: which thread, if any,
56+
/// has the (single) GL context current.
57+
class FakeEGL {
58+
public:
59+
void MakeCurrent() { current_thread_.store(std::this_thread::get_id()); }
60+
61+
void ClearCurrent() { current_thread_.store(std::thread::id()); }
62+
63+
bool IsCurrentOnThisThread() const {
64+
return current_thread_.load() == std::this_thread::get_id();
65+
}
66+
67+
private:
68+
std::atomic<std::thread::id> current_thread_;
69+
};
70+
71+
/// Lets the reactor react only on the thread that has the context current,
72+
/// like the Android embedder's worker in `android_context_gl_impeller.cc`.
73+
class FakeEGLReactorWorker final : public impeller::ReactorGLES::Worker {
74+
public:
75+
explicit FakeEGLReactorWorker(std::shared_ptr<FakeEGL> egl)
76+
: egl_(std::move(egl)) {}
77+
78+
// |ReactorGLES::Worker|
79+
bool CanReactorReactOnCurrentThreadNow(
80+
const impeller::ReactorGLES& reactor) const override {
81+
return egl_->IsCurrentOnThisThread();
82+
}
83+
84+
private:
85+
std::shared_ptr<FakeEGL> egl_;
86+
};
87+
88+
/// Builds a |GPUSurfaceGLImpeller| after making an offscreen ("pbuffer")
89+
/// context current, like `AndroidSurfaceGLImpeller::CreateSnapshotSurface`.
90+
class FakePbufferSnapshotSurfaceProducer final
91+
: public SnapshotSurfaceProducer {
92+
public:
93+
FakePbufferSnapshotSurfaceProducer(GPUSurfaceGLDelegate* delegate,
94+
std::shared_ptr<impeller::Context> context,
95+
std::shared_ptr<FakeEGL> egl)
96+
: delegate_(delegate),
97+
context_(std::move(context)),
98+
egl_(std::move(egl)) {}
99+
100+
// |SnapshotSurfaceProducer|
101+
std::unique_ptr<Surface> CreateSnapshotSurface() override {
102+
// Make the snapshot surface current because constructing a
103+
// GPUSurfaceGLImpeller and its AiksContext may invoke graphics APIs.
104+
egl_->MakeCurrent();
105+
return std::make_unique<GPUSurfaceGLImpeller>(delegate_, context_,
106+
/*render_to_surface=*/true);
107+
}
108+
109+
private:
110+
GPUSurfaceGLDelegate* delegate_;
111+
std::shared_ptr<impeller::Context> context_;
112+
std::shared_ptr<FakeEGL> egl_;
113+
};
114+
115+
/// A platform view with an Impeller OpenGL ES context that never gets a window
116+
/// surface: the test never calls |PlatformViewNotifyCreated|, like an Android
117+
/// app before `SurfaceView.surfaceCreated` (or after `surfaceDestroyed`).
118+
class NoSurfaceGLESPlatformView final : public PlatformView,
119+
public GPUSurfaceGLDelegate {
120+
public:
121+
NoSurfaceGLESPlatformView(PlatformView::Delegate& delegate,
122+
const TaskRunners& task_runners,
123+
std::shared_ptr<impeller::ContextGLES> context,
124+
std::shared_ptr<FakeEGL> egl)
125+
: PlatformView(delegate, task_runners),
126+
context_(std::move(context)),
127+
egl_(std::move(egl)) {}
128+
129+
// |PlatformView|
130+
std::shared_ptr<impeller::Context> GetImpellerContext() const override {
131+
return context_;
132+
}
133+
134+
// |PlatformView|
135+
std::unique_ptr<SnapshotSurfaceProducer> CreateSnapshotSurfaceProducer()
136+
override {
137+
return std::make_unique<FakePbufferSnapshotSurfaceProducer>(this, context_,
138+
egl_);
139+
}
140+
141+
// |GPUSurfaceGLDelegate|
142+
std::unique_ptr<GLContextResult> GLContextMakeCurrent() override {
143+
egl_->MakeCurrent();
144+
return std::make_unique<GLContextDefaultResult>(true);
145+
}
146+
147+
// |GPUSurfaceGLDelegate|
148+
bool GLContextClearCurrent() override {
149+
egl_->ClearCurrent();
150+
return true;
151+
}
152+
153+
// |GPUSurfaceGLDelegate|
154+
bool GLContextPresent(const GLPresentInfo& present_info) override {
155+
return true;
156+
}
157+
158+
// |GPUSurfaceGLDelegate|
159+
GLFBOInfo GLContextFBO(GLFrameInfo frame_info) const override {
160+
return GLFBOInfo{0, std::nullopt};
161+
}
162+
163+
private:
164+
std::shared_ptr<impeller::ContextGLES> context_;
165+
std::shared_ptr<FakeEGL> egl_;
166+
};
167+
168+
std::vector<std::shared_ptr<fml::Mapping>> ShaderLibraryMappings() {
169+
return {
170+
std::make_shared<fml::NonOwnedMapping>(
171+
impeller_entity_shaders_gles_data,
172+
impeller_entity_shaders_gles_length),
173+
std::make_shared<fml::NonOwnedMapping>(
174+
impeller_modern_shaders_gles_data,
175+
impeller_modern_shaders_gles_length),
176+
std::make_shared<fml::NonOwnedMapping>(
177+
impeller_framebuffer_blend_shaders_gles_data,
178+
impeller_framebuffer_blend_shaders_gles_length),
179+
};
180+
}
181+
182+
/// Gives the mock driver just enough behavior for pipelines to compile and
183+
/// offscreen render targets to be complete.
184+
void ConfigureMockDriver(NiceMock<impeller::testing::MockGLESImpl>& gl) {
185+
ON_CALL(gl, CreateShader(_)).WillByDefault(Return(1));
186+
ON_CALL(gl, CreateProgram()).WillByDefault(Return(1));
187+
ON_CALL(gl, IsProgram(_)).WillByDefault(Return(GL_TRUE));
188+
ON_CALL(gl, GetShaderiv(_, _, _))
189+
.WillByDefault([](GLuint shader, GLenum pname, GLint* params) {
190+
*params = pname == GL_COMPILE_STATUS ? GL_TRUE : 0;
191+
});
192+
ON_CALL(gl, GetProgramiv(_, _, _))
193+
.WillByDefault([](GLuint program, GLenum pname, GLint* params) {
194+
*params = pname == GL_LINK_STATUS ? GL_TRUE : 0;
195+
});
196+
ON_CALL(gl, GetIntegerv(_, _)).WillByDefault([](GLenum name, GLint* value) {
197+
if (name == GL_MAX_TEXTURE_IMAGE_UNITS ||
198+
name == GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS) {
199+
*value = 8;
200+
}
201+
});
202+
ON_CALL(gl, GenTextures(_, _)).WillByDefault([](GLsizei n, GLuint* names) {
203+
static std::atomic<GLuint> next_name = 1;
204+
for (GLsizei i = 0; i < n; i++) {
205+
names[i] = next_name++;
206+
}
207+
});
208+
ON_CALL(gl, GenFramebuffers(_, _))
209+
.WillByDefault([](GLsizei n, GLuint* names) {
210+
static std::atomic<GLuint> next_name = 1;
211+
for (GLsizei i = 0; i < n; i++) {
212+
names[i] = next_name++;
213+
}
214+
});
215+
ON_CALL(gl, IsTexture(_)).WillByDefault(Return(GL_TRUE));
216+
ON_CALL(gl, CheckFramebufferStatus(_))
217+
.WillByDefault(Return(GL_FRAMEBUFFER_COMPLETE));
218+
}
219+
220+
} // namespace
221+
222+
// In unoptimized builds this stops first at the FML_DCHECK in
223+
// ReactorGLES::CreateUntrackedHandle, when the throwaway AiksContext is built
224+
// with no context current. In optimized builds it crashes with the
225+
// use-after-free in DeviceBufferGLES::OnGetContents from the issue.
226+
TEST_F(ShellTest, SceneToImageWithoutSurfaceOnImpellerGLES) {
227+
auto mock_gles_impl =
228+
std::make_unique<NiceMock<impeller::testing::MockGLESImpl>>();
229+
ConfigureMockDriver(*mock_gles_impl);
230+
std::shared_ptr<impeller::testing::MockGLES> mock_gles =
231+
impeller::testing::MockGLES::Init(std::move(mock_gles_impl));
232+
233+
auto egl = std::make_shared<FakeEGL>();
234+
std::shared_ptr<impeller::ContextGLES> context =
235+
impeller::ContextGLES::Create(impeller::Flags{},
236+
std::make_unique<impeller::ProcTableGLES>(
237+
impeller::testing::kMockResolverGLES),
238+
ShaderLibraryMappings(),
239+
/*enable_gpu_tracing=*/false);
240+
ASSERT_TRUE(context);
241+
ASSERT_TRUE(std::static_pointer_cast<impeller::Context>(context)->IsValid());
242+
auto worker = std::make_shared<FakeEGLReactorWorker>(egl);
243+
context->AddReactorWorker(worker);
244+
245+
Settings settings = CreateSettingsForFixture();
246+
settings.enable_impeller = true;
247+
TaskRunners task_runners("test", // label
248+
GetCurrentTaskRunner(), // platform
249+
CreateNewThread(), // raster
250+
CreateNewThread(), // ui
251+
CreateNewThread() // io
252+
);
253+
254+
std::unique_ptr<Shell> shell = CreateShell({
255+
.settings = settings,
256+
.task_runners = task_runners,
257+
.platform_view_create_callback =
258+
[&context, &egl](Shell& shell) -> std::unique_ptr<PlatformView> {
259+
return std::make_unique<NoSurfaceGLESPlatformView>(
260+
shell, shell.GetTaskRunners(), context, egl);
261+
},
262+
});
263+
ASSERT_TRUE(shell->IsSetup());
264+
// Deliberately no PlatformViewNotifyCreated(): the rasterizer has no
265+
// surface, so no context is current on the raster thread.
266+
267+
fml::AutoResetWaitableEvent latch;
268+
int64_t width = 0;
269+
int64_t height = 0;
270+
std::optional<std::string> error;
271+
AddNativeCallback(
272+
"SceneToImageResult", CREATE_NATIVE_ENTRY([&](Dart_NativeArguments args) {
273+
width = tonic::DartConverter<int64_t>::FromDart(
274+
Dart_GetNativeArgument(args, 0));
275+
height = tonic::DartConverter<int64_t>::FromDart(
276+
Dart_GetNativeArgument(args, 1));
277+
Dart_Handle error_handle = Dart_GetNativeArgument(args, 2);
278+
if (!Dart_IsNull(error_handle)) {
279+
error = tonic::DartConverter<std::string>::FromDart(error_handle);
280+
}
281+
latch.Signal();
282+
}));
283+
284+
auto configuration = RunConfiguration::InferFromSettings(settings);
285+
configuration.SetEntrypoint("sceneToImageWithoutSurface");
286+
shell->RunEngine(std::move(configuration), [](auto result) {
287+
ASSERT_EQ(result, Engine::RunStatus::Success);
288+
});
289+
290+
latch.Wait();
291+
EXPECT_EQ(error, std::nullopt);
292+
EXPECT_EQ(width, 8);
293+
EXPECT_EQ(height, 8);
294+
295+
DestroyShell(std::move(shell), task_runners);
296+
}
297+
298+
} // namespace testing
299+
} // namespace flutter
300+
301+
// NOLINTEND(clang-analyzer-core.StackAddressEscape)

‎engine/src/flutter/lib/ui/fixtures/ui_test.dart‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -694,6 +694,31 @@ Future<void> toByteDataRetryOverflows() async {
694694
_validateNotNull(result);
695695
}
696696

697+
@pragma('vm:external-name', 'SceneToImageResult')
698+
external void _sceneToImageResult(int width, int height, String? error);
699+
700+
/// Reproduction from https://github.com/flutter/flutter/issues/192365: a
701+
/// `Scene.toImage` while the view has no window surface (on Android, before
702+
/// `SurfaceView.surfaceCreated`).
703+
@pragma('vm:entry-point')
704+
Future<void> sceneToImageWithoutSurface() async {
705+
final PictureRecorder recorder = PictureRecorder();
706+
Canvas(recorder)
707+
.drawRect(const Rect.fromLTWH(0, 0, 8, 8), Paint()..color = const Color(0xFF2ADEC3));
708+
final Picture picture = recorder.endRecording();
709+
final Scene scene = (SceneBuilder()..addPicture(Offset.zero, picture)).build();
710+
try {
711+
final Image image = await scene.toImage(8, 8);
712+
_sceneToImageResult(image.width, image.height, null);
713+
image.dispose();
714+
} catch (error) {
715+
_sceneToImageResult(0, 0, error.toString());
716+
} finally {
717+
scene.dispose();
718+
picture.dispose();
719+
}
720+
}
721+
697722
@pragma('vm:entry-point')
698723
Future<void> toImageRetries() async {
699724
final PictureRecorder pictureRecorder = PictureRecorder();

‎engine/src/flutter/lib/ui/painting/display_list_deferred_image_gpu_impeller.cc‎

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -168,23 +168,31 @@ void DlDeferredImageGPUImpeller::ImageWrapper::SnapshotDisplayList(
168168
return;
169169
}
170170

171-
sk_sp<DisplayList> display_list;
172-
171+
SnapshotDelegate::SnapshotDisplayListProvider provider;
173172
if (std::holds_alternative<sk_sp<DisplayList>>(content)) {
174-
display_list = std::get<sk_sp<DisplayList>>(std::move(content));
173+
provider = [display_list =
174+
std::get<sk_sp<DisplayList>>(std::move(content))](
175+
impeller::AiksContext*) { return display_list; };
175176
} else if (std::holds_alternative<std::unique_ptr<LayerTree>>(
176177
content)) {
177-
std::unique_ptr<LayerTree> layer_tree =
178-
std::get<std::unique_ptr<LayerTree>>(std::move(content));
179-
auto aiks_context = snapshot_delegate->GetAiksContext();
180-
display_list = layer_tree->Flatten(
181-
DlRect::MakeWH(wrapper->size_.width, wrapper->size_.height),
182-
snapshot_delegate->GetTextureRegistry(),
183-
/*gr_context=*/nullptr, aiks_context.get());
178+
// Flatten only once the snapshot controller has made a rendering
179+
// context current, and with the AiksContext that renders the
180+
// snapshot. https://github.com/flutter/flutter/issues/192365
181+
provider = fml::MakeCopyable(
182+
[layer_tree =
183+
std::get<std::unique_ptr<LayerTree>>(std::move(content)),
184+
texture_registry = snapshot_delegate->GetTextureRegistry(),
185+
bounds =
186+
DlRect::MakeWH(wrapper->size_.width, wrapper->size_.height)](
187+
impeller::AiksContext* aiks_context) mutable {
188+
return layer_tree->Flatten(bounds, texture_registry,
189+
/*gr_context=*/nullptr,
190+
aiks_context);
191+
});
184192
}
185193

186194
auto texture = snapshot_delegate->MakeImpellerSnapshotSync(
187-
display_list, wrapper->size_, wrapper->pixel_format_);
195+
std::move(provider), wrapper->size_, wrapper->pixel_format_);
188196
if (!texture) {
189197
std::scoped_lock lock(wrapper->error_mutex_);
190198
wrapper->error_ = "Failed to create snapshot.";

0 commit comments

Comments
 (0)