Skip to content

Conversation

@knopp
Copy link
Member

@knopp knopp commented Feb 12, 2025

Fixes #163104

The core issue is that EmbedderTaskRunner can schedule things in advance, but there is no checking if the task is stale when executing it. It is possible that FlutterEngineRunTask will attempt to run the task on engine that is non null but already being deallocated.

This in not a problem for raster and platform task runners, because raster task runner shouldn't have any scheduled tasks after shutdown and platform task runner executes the shutdown process, so the pointer is always either valid or null, but it is an issue for custom ui_task_runner, because it may be calling FlutterEngineRunTask with engine pointer that is not yet null but already shutting down.

The proposed solution is to assign a unique identifier for each EmbedderTaskRunner, use this identifier as the runner field inside FlutterTask instead of casting the address of the runner, and verify that this identifier is valid inside FlutterEngineRunTask before calling anything on the engine.

Special care needs to be done to not invalidate the unique identifier while ui_task_runner is running a task as it may lead to raciness. See EmbedderEngine::CollectThreadHost().

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@flutter-dashboard

This comment was marked as outdated.

@github-actions github-actions bot added the engine flutter/engine related. See also e: labels. label Feb 12, 2025
@knopp knopp changed the title [Embedder] Detect and invalidate stale task runner tasks [Embedder] Detect and ignore stale task runner tasks Feb 12, 2025
@knopp knopp force-pushed the embedder_detect_stale_tasks branch from 59ffdfa to 7577468 Compare February 12, 2025 14:06
@knopp
Copy link
Member Author

knopp commented Feb 12, 2025

Added a test that isolates the issue (always crashes) without the fix.

@knopp
Copy link
Member Author

knopp commented Feb 12, 2025

Looking at the test, it is now obvious that the reason why this wasn't a problem before:

Engine shutdown happens on platform thread, and thus checking the engine being non null before calling FlutterEngineRunTask() for platform tasks is not racy because it happens on same thread.

The engine is never invalid when calling FlutterEngineRunTask for raster thread because there shouldn't be any pending raster thread tasks after shutdown.

For UI thread the situation is different. We can't block UI thread while shutting down the engine, because during engine shutdown tasks are scheduled on UI thread. So there is a possibility for UI thread to call FlutterEngineRunTask on engine pointer that is in the process of shutting down but the pointer is still non-null.

This would only be a problem in practice if embedder specifies custom UI task runner that is not same as platform task runner. This doesn't seem a likely scenario, but the API allows it so we should make sure it works correctly.

@matanlurey
Copy link
Contributor

Thanks for the quick response @knopp. I'm going to defer to @jonahwilliams for review.

@matanlurey matanlurey removed their request for review February 12, 2025 17:02
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@knopp knopp force-pushed the embedder_detect_stale_tasks branch from 5524f63 to 876f970 Compare February 12, 2025 17:06
@knopp knopp enabled auto-merge February 12, 2025 19:16
@knopp knopp added this pull request to the merge queue Feb 12, 2025
Merged via the queue into flutter:master with commit 1023664 Feb 12, 2025
178 checks passed
@knopp knopp deleted the embedder_detect_stale_tasks branch February 12, 2025 20:21
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 15, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 16, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 17, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 18, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 19, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Feb 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine flutter/engine related. See also e: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

macOS host engine tests are flaky since #162944

3 participants