Repository navigation
[Impeller] SIGABRT in vkDestroyFramebuffer after long background dwell (cross-GPU: Mali + Adreno) #191168
Description
UPDATE 2026-08-17: deeper forensics across what are now 9 occurrences changed the reproduction model and corrected the toolchain info. The crashes fire while the app is backgrounded, during background FlutterEngine create/destroy churn (Workmanager tasks), not on resume. The crashing binaries were built on Flutter 3.44.8 / engine
0cd610717bde, not 3.47.0 as this body originally said. Full data, corrections, and an MRE harness: comment. The body below has been corrected in place.
Steps to reproduce
- Flutter app using Impeller (Vulkan backend) on Android, with
workmanagerbackground tasks (each Android task execution spawns a freshFlutterEnginewith its own Impeller Vulkan context and destroys it on completion). - Use the app normally, then background it (HOME, not swipe-away).
- Leave it backgrounded for hours (8h to 19h measured across our occurrences) while the background tasks keep running on their schedule.
- A background task execution eventually crashes the process: the main (platform) thread runs
vkDestroyFramebufferfrom insideLooper::pollOnceagainst driver state that is already destroyed.
The crash is silent for the user (the app is backgrounded; it is simply gone from recents afterwards). One measured case crashed 5 minutes after the app's self-scheduled 00:10-local background task fired, 8h11m after backgrounding. Crashed processes show engine churn in their thread lists: e.g. only 3.raster alive (engines 1 and 2, including the original UI engine, already destroyed), or 1.raster/2.raster/6.raster alive (engines 3 to 5 came and went).
An MRE harness that compresses this cadence from hours to seconds (self-rescheduling workmanager one-off every ~20s, plus forced job runs and am send-trim-memory RUNNING_CRITICAL): https://github.com/vedamayank/vk_engine_churn_repro
Crash signature
FORTIFY: pthread_mutex_lock called on a destroyed mutex (0x...)
Fatal signal SIGABRT (6), SI_QUEUE (-1)
Mechanism: Tombstone (native, unhandled). On two devices the same shape manifests as SIGSEGV (SEGV_MAPERR) instead (FORTIFY catches the freed mutex before the wild dereference on some builds, not on others).
Stack trace (Qualcomm/Adreno device)
abort
__fortify_fatal
HandleUsingDestroyedMutex
pthread_mutex_lock
<driver-internal frame> (module hash tracks the device model, e.g. f6b5df5188 on CPH2707)
<driver-internal frame>
<driver-internal frame>
qglinternal::vkDestroyFramebuffer
<libflutter.so frame> (~12 unsymbolicated Impeller/engine frames)
...
android::Looper::pollOnce
android::android_os_MessageQueue_nativePollOnce
android.os.Looper.loopOnce
android.os.Looper.loop
android.app.ActivityThread.main
Correction from the original report: the three hashed frames between pthread_mutex_lock and vkDestroyFramebuffer are driver-internal (their module hash differs per device model), not libflutter. The libflutter.so (Impeller) frames are the unsymbolicated ones between Looper::pollOnce and vkDestroyFramebuffer. Mali devices show the same shape with the driver frames unsymbolicated (the Mali driver exports no symbols).
Affected devices (9 events, 9 users, Aug 11-16 2026)
| Date (UTC) | Device | GPU | Signal |
|---|---|---|---|
| Aug 11 18:05 | realme RMX5003 | Mali | SIGABRT |
| Aug 14 12:04 | OnePlus CPH2707 | Adreno | SIGABRT |
| Aug 14 18:59 | OnePlus CPH2487 | Adreno | SIGSEGV (SEGV_MAPERR) |
| Aug 14 23:02 | realme RMX3999 | Mali | SIGABRT |
| Aug 15 04:48 | realme RMX5033 | Mali | SIGABRT |
| Aug 15 05:15 | OnePlus CPH2707 | Adreno | SIGABRT |
| Aug 15 12:24 | Samsung SM-S906U (S22+) | Adreno | SIGSEGV (SEGV_MAPERR) |
| Aug 15 14:22 | Samsung SM-A146P (A14 5G) | Mali | SIGABRT |
| Aug 16 13:58 | realme RMX5003 | Mali | SIGABRT |
Android 15 and 16, all arm64, all Play installs. Crosses GPU vendors, so the bug is in Impeller's Vulkan backend teardown ordering, not one driver.
Relationship to #187237 / PR #187477
PR #187477 added WaitIdle() in AHBSwapchainImplVK::~AHBSwapchainImplVK before destroying swapchain-owned Vulkan objects. That fix IS present in our build (Flutter 3.44.8 stable, via cherry-pick #188531, commit 97f70d2e9ef), and the crash still occurs.
This crash is in vkDestroyFramebuffer, not in swapchain teardown. The framebuffer destruction path does not appear to go through AHBSwapchainImplVK, so the WaitIdle guard from #187477 does not protect it. Same class of bug (destroying a Vulkan object whose device-side state is already gone), different site.
Additional context
- Every engine in the crashed processes, headless background ones included, has its own
IplrVkFenceWait/IplrVkResMgrthread pair, i.e. a full Impeller Vulkan context per engine. - The background tasks also rasterize widgets offscreen through Flutter (
HomeWidget.renderFlutterWidget), so background engines create real framebuffers. - No Dart-level breadcrumbs survive because the crash fires below the framework layer while the app is backgrounded.
- Our crash capture changed in the same app release (tombstone-based native capture is new in it), so absence of this signature on earlier releases is not evidence of a regression range.
Flutter version
Crashing production binaries (correction; the original report mistakenly quoted the dev machine's 3.47.0 / 59d54a2b2896):
Flutter 3.44.8, channel stable
Engine revision 0cd610717bde95fd88343c64f81c11ba4e5c0010
android-arm64-release libflutter.so GNU BuildID 889a1df37c58955a78bb76391e165822b866bf48
Our next release is built on Flutter 3.47.0 (engine 5f77625673); we will report whether the crash reproduces there.
Activity
puneetkukreja98 commentedon Aug 17, 2026
@vedamayank Could you share an easier or alternative way to reproduce this with an MRE.? Also, could you provide the o/p of your flutter doctor -v?
vedamayank commentedon Aug 17, 2026
@puneetkukreja98 Thanks for picking this up. Three parts: (1) the
flutter doctor -vyou asked for, (2) corrections to the original report after deeper forensics, (3) a much more specific reproduction model plus an MRE harness, based on cross-referencing what are now 9 occurrences with our product analytics and full thread dumps.1. flutter doctor -v
2. Corrections to the original report
Version correction. The doctor output above is today's dev machine (3.47.0). The production binaries behind these crashes were built on Flutter 3.44.8 stable, engine
0cd610717bde95fd88343c64f81c11ba4e5c0010. The original report incorrectly quoted 3.47.0 /59d54a2b2896; that was the dev machine at filing time, sorry for the confusion. For symbol matching: theandroid-arm64-releaselibflutter.sofor that engine has GNU BuildID889a1df37c58955a78bb76391e165822b866bf48.The
AHBSwapchainImplVK::WaitIdle()fix is present at that tag (cherry-picked via #188531, commit97f70d2e9ef), so the original point stands: this crash is in a path that fix does not cover.Stack annotation correction. The three hashed frames between
pthread_mutex_lockandqglinternal::vkDestroyFramebufferare driver-internal, not libflutter. Their module hash tracks the device model, not our binary:f6b5df5188on both OnePlus CPH2707 events,2e251f7ab9on the Galaxy S22+,0c393b63cfon the CPH2487. Thelibflutter.so(Impeller) frames are the ~12 unsymbolicated frames betweenLooper::pollOnceandvkDestroyFramebuffer. We are also wiring the matching engine symbols into our Sentry org so future occurrences arrive with those frames named, and we can share the raw tombstone event JSON for any of the 9 events if that helps.3. Updated scale and a revised reproduction model
9 events, 9 users, 7 device models in 6 days (Aug 11-16 UTC). 8 of the 9 carry release metadata, and all 8 are the same app release; the Aug 11 event reported no version metadata:
Android 15 and 16, all arm64, all installed from Play. Adreno is confirmed by the
qglinternal::frames; Mali bymali-event-hand/mali-mem-purgethreads in the dumps (the Mali driver exports no symbols, so those stacks show the same shape with the driver frames unsymbolicated). We read SIGABRT and SIGSEGV as one bug: identical crashed-thread shape (main thread, insideLooper::pollOnce, destroying a framebuffer); FORTIFY just catches the freed mutex before the wild dereference on some builds.One caveat we owe you: this app release also changed our crash capture (tombstone-based native capture is new in it), so "zero occurrences on earlier releases" is not evidence the bug is new. Please do not treat it as a regression range.
What analytics cross-referencing shows. We matched each crash against our product analytics (lifecycle events per user). This changes the reproduction model from the original report:
Application Backgrounded8h to 19h earlier, and nothing foreground follows until a later cold start. Users never see the crash; the app is simply gone from recents.FlutterEngineand destroys it on completion, and the task rasterizes widget images offscreen through Flutter (HomeWidget.renderFlutterWidget).3.raster/3.ioalive: engines 1 and 2, including the original UI engine, were already destroyed. The S22+ process has1.raster,2.raster, and6.rasteralive, so engines 3 to 5 had come and gone. Every engine, headless ones included, has its ownIplrVkFenceWait/IplrVkResMgrpair, i.e. a full Impeller Vulkan context.So our best current model: hours after backgrounding (with the OS trimming/reclaiming in between), background FlutterEngine create/destroy churn eventually runs a deferred
vkDestroyFramebufferon the platform thread against a VkDevice that engine teardown already destroyed. The long dwell seems to matter as accumulation plus OS reclamation, not because resume is involved.MRE
Minimal harness: vk_engine_churn_repro. It compresses the production cadence from hours to seconds:
workmanagerone-off spawns and destroys a background engine every ~20s while the app is backgrounded (same plugin and pattern as production, 0.10.x).churn.shforces the queued job to run each cycle and issuesam send-trim-memory RUNNING_CRITICAL, replacing the overnight dwell.Honest status: we have not yet caught the crash on demand ourselves; production needed hours of dwell at a 6h task cadence, and we are now running the compressed harness on the affected models. Sharing it already because the conditions above are narrow enough to point at the multi-engine teardown path, and your team can likely get there faster with engine internals in view.
Happy to run instrumented or validation builds on affected models, share raw tombstones, or test a patch. This is a recurring fatal for us (9 users in 6 days on the current release) and we believe it contributes to crash complaints in our store reviews. Our next release is built on 3.47.0 (engine
5f77625673), and we will report whether it reproduces there.puneetkukreja98 commentedon Aug 18, 2026
@vedamayank Thanks for the response, I noticed that you are using the workmanager package in your MRE.
Could you provide a MRE without third-party packages?
vedamayank commentedon Aug 19, 2026
@puneetkukreja98 Done, the MRE is now plugin-free: vk_engine_churn_repro (same link) has zero pub dependencies and zero added Gradle dependencies. Framework
JobSchedulerplus ~100 lines of Kotlin do exactly what the plugin did:new FlutterEngine(context)->executeDartEntrypoint(offscreentoImagerenders, pure dart:ui) ->jobFinished->engine.destroy()posted to a later main-looper turn. The workmanager version stays at tagv1-workmanager. One note from the README: it must be plainFlutterEngine, notFlutterEngineGroup; grouped engines share oneContextVK/VkDevice, which masks the bug.Since the last comment we also received a fully symbolicated production occurrence (10th event, 8th device model; in-process signal-handler capture instead of a tombstone; same release on Flutter 3.44.8, engine
0cd610717bde, Adreno, Android 16). Innermost frames first, libc++ internals elided:That names the mechanism end to end: a message-loop task from
Picture::DoRasterizeToImage's task chain (picture.cc:244) is destroyed duringFlushTasks, releasing the last reference to aDlImageImpeller;~TextureSourceVKthen destroys its cachedframe_data_(FramebufferAndRenderPass), whoseSharedHandleVK<vk::Framebuffer>destroys through its rawVkDevice, which the already-destroyed engine no longer backs.~TextureSourceVKis defaulted, with no device-liveness guard, on 3.44 and on master; #189666 added exactly that guard to the AHB subclass members and its comment notes the destructor can run afterContextVK/VkDevicedestruction, but the base-classframe_data_teardown was not covered. Same bug class as #187703 (FenceWaiter) and #186458 (CommandPoolVK), both already fixed one object at a time.The harness reproduces this on demand. OnePlus NE2211 (Adreno, Android 16), Flutter 3.47.0 stable, debug build with the stock validation layer (
EnableVulkanValidation): crashed within 5 churn cycles, on both process generations tried. The timestamped sequence from one occurrence:The leaked pair the layer names (framebuffer + render pass) is exactly what
frame_data_caches. Symbolized against the engine's own debug symbols (BuildID-verified), the crash chain on 3.47.0 is frame-for-frame the production stack above:FlushTasks-> destruction of an undeliveredPicture::DoRasterizeToImageresult lambda (picture.cc:266; capturesshared_ptr<impeller::Texture>) ->~TextureVK->frame_data_teardown (std::vector<CachedFrameDataEntry>there,std::array<FramebufferAndRenderPass,2>at 3.44.8) ->SharedHandleVKrelease through the dead device. Full tombstones, validation output, and symbolized stacks are in the repo underevidence/, with the run protocol inRESULTS.md. The release build survived 60 freeze-stress cycles before this, so the validation layer is what makes it deterministic; production waits for the driver to trip instead.Suggested labels:
engine,e: impeller,platform-android,c: fatal crash,a: production,found in release: 3.44,customer: crowd.puneetkukreja98 commentedon Aug 19, 2026
@vedamayank Thanks for the updated MRE, routing this to team-engine for further investigation.
phpirate commentedon Sep 22, 2026
Second production app hitting this, moving our data over from #193182 which @mbcorona closed as a duplicate. The analysis above is well ahead of anything I could add to, so only what might be useful.
Same shape as @vedamayank's app: Impeller default, offscreen widget rasterisation via
HomeWidget.renderFlutterWidget()from short-lived background isolates. Flutter 3.44.6 stable, engined3a3293399, so just behind the 3.44.8 above and without the #187477 cherry-pick.Prevalence: 8 crashes from 8 distinct users in 6 days, against a Play install base of about 26 devices. Two vendors (Adreno and Mali), 5 device models, Android 15 and 16, two of our releases. Seven are teardown aborts nobody sees; one hit with the app in the foreground.
One symbolicated Mali stack, in case it is useful that the Mali side is not just shape-matched. Caught in-process by a signal handler rather than read from a tombstone, on a Xiaomi
25028RN03Y, Android 15 (A15.0.2.0.VGWMIXM):That is the
~TextureSourceVK→frame_data_teardown you named, reached through the Mali blob instead of Adreno, which fits the unguarded destructor being the common factor rather than anything vendor-specific.Happy to run the harness on the affected Mali models or test a patched engine build. Raw tombstones and breadcrumbs for any of the 8 are available if they would help.
phpirate commentedon Oct 2, 2026
Prevalence update from the second app, 10 days on.
When I posted on 22 Sep it was 8 crashes from 8 distinct users. The largest of our four Sentry groups for this signature is now at 116 events from 79 distinct users, still climbing — 10 new users in the last 26 hours alone. Spread across 5 of our releases, both vendors, Android 13 through 16, and 20+ device models. No release of ours has ever been free of it.
One mechanism detail I left out last time, which probably explains why we see it so heavily: our
workmanagertask is registered unconditionally at every launch on a 15-minute period, and each execution callsHomeWidget.renderFlutterWidget(). So every install churns a backgroundFlutterEnginefour times an hour regardless of whether the user has actually placed a widget — the cadence in the MRE, running continuously in production.Field data above is still Flutter 3.44.6 (engine
d3a3293399); our shipped build has not moved yet. I did check 3.47.5 source:TextureSourceVK::~TextureSourceVKis still= default, andframe_data_is now astd::vector<CachedFrameDataEntry>rather than a singleFramebufferAndRenderPass, so teardown destroys N framebuffers through the dead device instead of one.Offer stands to test a patched engine build on affected hardware, Adreno or Mali.
wazeemlabs commentedon Oct 12, 2026
A third production app with this abort, and a second way to get the extra lifetime: no second FlutterEngine and no workmanager, only ordinary background isolates (
Isolate.spawnandIsolate.run) in the root isolate's group.Build: Flutter 3.47.2 stable, engine a804b26, Dart 3.13.2. Impeller default (no
EnableImpellermeta-data). OneFlutterFragmentActivity, a new engine per activity, noFlutterEngineGroup, no cached engine, no home-screen widget rendering.Prevalence, 28 days to 2026-10-12 (Play Console reporting API): 15 reports from 14 devices on three of our releases. Mali 11 (Honor, Samsung Galaxy A16 5G, Redmi 13C, Redmi 14C, Redmi Note 14, Xiaomi 14T, moto g67, ZTE 9030 twice, Oppo A5 Pro 5G, one unnamed), Adreno Vulkan 4 (Galaxy S24 Ultra, OnePlus 13R, motorola edge 50 fusion, motorola signature). Android 11 to 16.
Two backtrace shapes, both on a Dart thread-pool thread, never on the UI or raster thread:
EngineLayer::~EngineLayerup, reached from:The Dart frames below the GC are our background isolate's own work (image decoding through FFI, a TFLite model running over image tiles), never UI code.
What the frames show, and what is inferred: the root isolate runs on the UI thread, so a
Dart_ShutdownIsolateon a thread-pool worker is a spawned isolate, here running group-wide finalizers ofEngineLayers the framework never disposed. From that we infer the sequence: the activity is destroyed while a background isolate is still busy (on Android 12+ that is the task swiped from recents), the engine's Vulkan device goes with it, and the background isolate then finalizes the UI's layers against the dead device, in a GC or when it exits as the group's last isolate. We have not reproduced it on a device; the inference is from the backtraces alone. If it holds, it is the same extra lifetime this issue's second engine provides, from inside one engine.The 15 backtraces, frames only, can be attached as a text file if that helps.
TonyDowney commentedon Oct 23, 2026
One more trigger for this abort, on Mali with Vulkan: an ordinary
FlutterActivityrecreate after a configuration change. No background engine, no workmanager, no home-widget rendering. The app was in the foreground.Build: Flutter 3.47.3, engine
0e228ec8c8d2abc9fcf1d053e8a40665bb859ec7, Dart 3.13.3. Impeller default. Profile build. OneFlutterActivitythat owns its engine (no cached engine, noFlutterEngineGroup).Device: Pixel 8 (Tensor G3, Mali), Android 16.
Steps:
adb shell cmd overlay enable-exclusive --category com.android.internal.systemui.navbar.threebutton. Android swaps a resource overlay, so the activity is destroyed and recreated.FlutterActivitydestroys its engine with the old activity and creates a new engine for the new one, in the same process.About 100 ms after the new engine attached, logcat showed
F libc: FORTIFY: pthread_mutex_lock called on a destroyed mutex. The aborting thread was in a Vulkan destroy call fromlibflutter.so. The frames below it have no symbols, so "the old engine's teardown" is my reading of the timing, not something the trace proves:It was
vkDestroyRenderPasshere, notvkDestroyFramebuffer. The shape is the same as the teardown cases above: a destroy call into the driver after the driver's state is gone.Rate: 1 abort in 2 tries on the first day. It did not come back on demand after that: 0 in 28 more recreates on the same build. On every recreate the old engine detached and a new Impeller context was created (6 of 6 that were still in the log buffer).
What made it worse for us: the app also runs Sentry with the native (NDK) integration. Sentry's signal handler caught the abort on that thread and blocked. At the same moment, the new engine's Dart
main()re-initialised Sentry on the platform thread, which closes the previous native SDK and waits for it. The two waited on each other: no tombstone, the UI froze on its last frame, and an ANR followed 80 s later. I am reporting that half to Sentry separately; it only matters here because it turns this abort into a silent hang.Workaround that removed the teardown on this path: keep the engine across a configuration-change recreate only. In the activity:
shouldDestroyEngineWithHost()returns!isChangingConfigurations;onDestroy()keepsflutterEngine(andbackCallbackState) in a static slot whenisChangingConfigurationsis true;provideFlutterEngine()hands it to the next instance and callssetFrameworkHandlesBack(...)with the saved value. After that: 30 recreates, 0 engine teardowns, 0 aborts. Two things a retained engine loses with the old activity, in case anyone copies this: the back-callback registration (Back leaves the app on Android 16 without thesetFrameworkHandlesBackcall) and theSystemChromeUI mode and preferred orientations, which need to be sent again. Do not use one process-wide engine for this: aFLAG_ACTIVITY_CLEAR_TASKlaunch then throwsAssertionError ... has been attached to by another activity.This only avoids the teardown on a recreate. Finishing the activity still destroys an engine, so the race is still there.