[windows] Add an Impeller Vulkan backend presented through DirectComposition#189586
Draft
sero583 wants to merge 5 commits into
Draft
[windows] Add an Impeller Vulkan backend presented through DirectComposition#189586sero583 wants to merge 5 commits into
sero583 wants to merge 5 commits into
Conversation
…urce exhaustion Stability fixes for the Vulkan backend discovered through sustained stress testing on AMD RDNA 2 hardware and Mesa dzn (Vulkan on D3D12 in WSL2), first published as part of flutter#183382 and rebased onto the current FenceWaiterVK submission architecture. Device loss handling: - ContextVK gains an atomic device_lost_ flag. CreateCommandBuffer() and CommandQueueVK::Submit() short-circuit once the device is lost, since any Vulkan call on a driver that entered a corrupted OOM state can access-fault inside the ICD. - On vkQueueSubmit failure, tracked objects are abandoned without making Vulkan calls (AbandonForDriverCrash on command pools and descriptor pools), the device is marked lost, and per-heap memory budgets are logged for diagnosis. - FenceWaiterVK releases rather than destroys the fence when the submit callback fails; some drivers partially track the fence even though the submission failed, and vkDestroyFence then crashes (VUID-vkDestroyFence-fence-01120 observed on AMD). Submission backpressure: - CommandQueueVK caps concurrent in-flight submissions (3) with a condition variable, releasing the slot from the fence completion callback. Prevents unbounded host memory growth when the CPU outpaces the GPU. - ContextVK::FlushCommandBuffers() submits in chunks of 64 to keep per-submit driver allocations bounded on frames that accumulate very large command buffer counts. - GPUSurfaceVulkanImpeller (embedder-managed image path) adds a two-entry frame fence ring mirroring the KHR swapchain's backpressure, with a bounded wait that skips the frame under extreme GPU pressure. Image views for embedder images are now cached instead of being allocated (and leaked) per frame, and in-flight frames are drained before views are destroyed on resize or teardown. Command pool lifecycle: - BackgroundCommandPoolVK releases its handles without Vulkan calls when the recycler is gone; the device may already be destroyed at that point. This is the same class of teardown race as the CommandPoolVK destructor fix that landed in flutter#186749. - The recycled pool list is capped at 8 entries, evicting oldest first. - unused_command_buffers_ is annotated IPLR_GUARDED_BY(pool_mutex_). Descriptor pools: - Reset via vkResetDescriptorPool on reclaim. Reclaimed pools previously stayed at capacity and produced VK_ERROR_OUT_OF_POOL_MEMORY on every reuse. Barrier and render pass correctness: - Six barrier fixes in the blit pass: pre-copy barriers use transfer stage and access flags, post-copy barriers flush transfer writes, and mipmap generation barriers match the actual current layout. Fixes corrupted glyph atlases and SYNC-HAZARD-WRITE-AFTER-WRITE reported by the synchronization validation layer on AMD RDNA 2. - RenderPassBuilderVK moves subpass dependencies to a counted builder, drops eByRegion from external (VK_SUBPASS_EXTERNAL) dependencies where it has no meaning per Vulkan spec section 7.1, and gains SetFramebufferFetchEnabled() so input attachment references and self-dependencies are omitted on drivers without framebuffer fetch support. The flag is wired through pipeline and render pass creation and the compat render pass used for PSO construction. Driver workarounds: - DriverInfoVK exposes GetDriverID() (Vulkan 1.2) and detects Mesa dzn. - New skip_sub_region_buffer_to_image_copy workaround: Mesa dzn reports minImageTransferGranularity of (0,0,0) and corrupts textures on sub-region buffer-to-image copies; the blit pass stages such copies through a full-size temporary texture. Memory management: - The VMA buffer pool is capped at 256 blocks (about 1 GB); the pool otherwise grows without bound when high-water-mark demand spikes and fragmentation prevents block reuse. - Image allocation failure paths no longer leak the VMA allocation when image view creation fails. - On Windows, the working set is periodically trimmed when Resizable BAR mappings inflate RSS far beyond actual VMA usage (gated by cooldown, RSS threshold, and RSS-to-VMA ratio). KHR swapchain robustness: - The synchronizer fence is re-signaled with an empty submit on every early return after a failed or unusable acquireNextImageKHR (including the surface-lost path added in flutter#183338). The fence was reset by WaitForFence, and without a new signal the next wrap-around to the same synchronizer deadlocks in waitForFences with an infinite timeout. - AcquireNextDrawable short-circuits when the device is marked lost. - Zero-extent surfaces (minimized windows) skip the frame instead of attempting swapchain recreation, preserving recovery on restore. Embedder mode: - CapabilitiesVK enumerates instance layers in embedder mode so that validation layers can be detected and enabled when requested.
…ottle tests Adjust the descriptor pool changes to keep the current set-recycling semantics (used sets return to the unused cache on reclaim) instead of resetting pools on reclaim, which would have invalidated the cached set handles that DescriptorsAreRecycled encodes as intended behavior. The safety fixes remain: AbandonForDriverCrash, handle release on teardown when the context or recycler is gone, propagating CreateNewPool failures, and registering a new descriptor set in the cache only after the allocation succeeded (a garbage handle was previously cached when vkAllocateDescriptorSets failed). Guard the DeathRattle test helper against invoking a moved-from callback. WaitForReclaim moves the local instance into a UniqueResourceVKT, and the moved-from local still runs the destructor; calling the empty std::function aborts with bad_function_call. This reproduces on unmodified master when running impeller_unittests on a Windows host and made every CommandPoolRecyclerVKTest case crash there. New tests: - CommandQueueVKTest.SubmitAfterDeviceLostIsCancelled: no submission reaches the queue once the device is marked lost. - CommandQueueVKTest.ThrottleAllowsSustainedSubmissions: in-flight slots are released as submissions complete. - ContextVKTest.CreateCommandBufferShortCircuitsAfterDeviceLost.
An embedder that presents rendered images itself, for example through a platform compositor, supplies a Vulkan device without the surface, WSI, or swapchain extensions. CapabilitiesVK previously rejected such a device because it required VK_KHR_surface and a windowing-system integration extension. Relax those requirements in embedder mode when the surface extension is absent, so presentation becomes the embedder's responsibility. Embedders that do provide the surface extension are unaffected. Adds regression tests covering the surfaceless case and the case where the embedder still provides the surface extension.
This was referenced Jul 16, 2026
Follow-ups to the first review pass on the Vulkan hardening change: - AllocatedTextureSourceVK: destroy the already-created image views before the image when a render target view fails to create partway through the per-subresource view loop. - BlitPassVK: pick the destination barrier's source scope from the texture's tracked layout in OnCopyTextureToTextureCommand, mirroring OnCopyBufferToTextureCommand. The unconditional eTransferWrite scope can trip best-practices validation when the destination was last sampled; the WAW protection for back-to-back transfer writes in the atlas growth path is preserved in the eTransferDstOptimal branch. - AllocatorVK: make the Windows working-set trim cooldown a process-wide atomic instead of thread_local, since EmptyWorkingSet affects the whole process. - GPUSurfaceVulkanImpeller: on a lost device, abandon the cached image views and frame fences instead of waiting on or destroying them, in both the destructor and the resize drain, matching the AbandonForDriverCrash policy. Adds a teardown-after-device-loss test.
sero583
force-pushed
the
impeller-vulkan-windows-dcomp
branch
from
July 17, 2026 11:16
cf1f238 to
7c847d7
Compare
…osition Adds an opt-in Vulkan rendering path to the Windows embedder: Impeller renders with its existing Vulkan backend and the content is presented through DXGI and DirectComposition, so the OS compositor owns presentation. There is no VkSurfaceKHR and no Vulkan swapchain in this path, addressing the review feedback on the original proposal that a KHR swapchain would block compositor integration, resize synchronization, and future platform views. Activation uses engine switches, the same mechanism as the existing Impeller opt-in, and requires both switches: --enable-impeller --impeller-backend=vulkan Nothing changes otherwise; ANGLE remains the default and the fallback. Wiring these switches into the flutter tool's command line is left to a follow-up, so for now they are supplied through an embedder's switch list or the engine switch environment protocol. Components: - VulkanManager: dynamic vulkan-1.dll loading (a missing runtime falls back before engine start), instance and device creation with no windowing extensions. Device selection requires a graphics queue, a valid adapter LUID, and Direct3D 11 texture import support, verified per device via vkGetPhysicalDeviceImageFormatProperties2, so a successful Create() guarantees the interop works. Every rejection logs its reason. - DCompPresenter: a Direct3D 11 device on the LUID-matched adapter allocates the shared render target textures (keyed mutex, NT handle) and presents them by copying into a flip-model composition swapchain rooted in a DirectComposition visual. Content reaches the screen only through the DComp commit, so window chrome and Flutter content resize atomically. - VulkanImportedImage: binds a VkImage to a shared texture's memory (dedicated allocation, VkImportMemoryWin32HandleInfoKHR). The import direction is dictated by the drivers: Direct3D 11 texture memory is reported to Vulkan as import-only (measured on AMD Adrenalin as DEDICATED_ONLY | IMPORTABLE), so the allocation originates on the Direct3D side, the same direction Chromium and Dawn use. - CompositorVulkan: a FlutterCompositor whose backing stores pair a shared texture with its Vulkan import. The engine host-syncs all layers before the present callback, and the Direct3D side brackets its access with the keyed mutex. Platform views are rejected with a clear error; the visual tree architecture is what enables them later. - Impeller Vulkan backing stores: the engine renders each compositor backing store into the imported image with Impeller. The image is wrapped as the resolve target of an MSAA color attachment with a depth-stencil, matching SurfaceVK::WrapSwapchainImage, and reported as a swapchain image so the render pass leaves it in VK_IMAGE_LAYOUT_GENERAL rather than a sampled layout the imported image does not support. GPUSurfaceVulkanImpeller gains a render_to_surface flag: when an external view embedder is present the root surface is a no-op and all content arrives through backing stores, matching the OpenGL Impeller path. - FlutterWindowsEngine: parses --impeller-backend=vulkan next to the existing enable-impeller handling, skips EGL initialization entirely when Vulkan is active, and supplies a kVulkan renderer config whose image callbacks are never invoked because the compositor is supplied. The SDF rendering default is an ANGLE/OpenGL Impeller option and is not applied when the Vulkan backend is active. Tests (hardware-dependent cases skip cleanly without Vulkan): - VulkanManagerTest: loading, create-or-null, LUID validity, required extensions, no windowing extensions ever, proc resolution, teardown. - VulkanImportedImageTest: argument validation. - DCompPresenterTest: unknown LUID rejection, present-before-bind rejection, the full allocate/import/present/evict pipeline against a live DirectComposition tree, in-place swapchain resize, and repeated lifecycle cycles. - FlutterWindowsEngineTest: the backend requires the explicit switch, is ignored without Impeller, either activates with ANGLE skipped or falls back without failing engine construction, and does not enable SDF rendering when Vulkan is active. - GPUSurfaceVulkanImpeller: with render_to_surface disabled the root surface yields a no-op frame and never pulls an image from the delegate.
Contributor
Author
|
oops |
sero583
force-pushed
the
impeller-vulkan-windows-dcomp
branch
from
July 17, 2026 11:24
7c847d7 to
1cd99c3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in Vulkan rendering path to the Windows embedder: Impeller renders with its existing Vulkan backend, and the content is presented through DXGI and DirectComposition, so the OS compositor owns presentation. There is no VkSurfaceKHR and no Vulkan swapchain anywhere in this path, addressing the review feedback on #183382 that a KHR swapchain would block compositor integration, resize synchronization, and future platform views.
Activation uses the engine switch mechanism, in line with the review guidance to gate the backend behind flags rather than public API. The engine accepts
--impeller-backend=vulkannext to the existing--enable-impellerhandling; until the flutter_tools plumbing lands in a follow-up, the switch reaches the engine through the standard FLUTTER_ENGINE_SWITCHES environment protocol (debug and profile only):Without the explicit backend switch, or when no capable device exists, nothing changes: ANGLE remains the default and the fallback. When Vulkan is active, ANGLE is not initialized at all.
How it works
VulkanManagerinitializes the instance and device with dynamic loading (vulkan-1.dll; missing runtimes fall back before engine start) and no windowing extensions. Device selection requires a graphics queue, a valid adapter LUID, and Direct3D 11 texture import support, verified per device throughvkGetPhysicalDeviceImageFormatProperties2before selection, so a successfulCreate()guarantees the interop works. Every rejection path logs its reason for field diagnosis.DCompPresenterowns a Direct3D 11 device on the LUID-matched adapter, allocates the shared textures, and presents them through a composition swapchain rooted in a DirectComposition visual on the window. Because content reaches the screen only through the DComp commit, the window chrome and the Flutter content resize atomically.VulkanImportedImagebinds a VkImage to the shared texture's memory (dedicated allocation,VkImportMemoryWin32HandleInfoKHR).DEDICATED_ONLY | IMPORTABLE), so the allocation originates on the Direct3D side. This matches the direction used by Chromium and Dawn.CompositorVulkanrenders each backing store with Impeller. The imported image is reported as a swapchain image so the render pass leaves it inVK_IMAGE_LAYOUT_GENERALrather than a sampled layout the imported image does not support.GPUSurfaceVulkanImpellergains arender_to_surfaceflag: with an external view embedder present the root surface is a no-op and all content arrives through backing stores, matching the OpenGL Impeller path.Synchronization
The engine performs a host sync for all layers before invoking the compositor present callback (documented in
FlutterVulkanBackingStore), so Vulkan writes are complete before the copy. The Direct3D side still brackets its access with the shared texture's keyed mutex, which performs the cross-device flushes.Depends on
The Impeller hardening PR (#189580), which includes the CapabilitiesVK change that allows an embedder to create a surfaceless Vulkan context (no
VK_KHR_surface/ swapchain), required by this presentation path.Not in this PR
FlutterVulkanRendererConfig,FlutterCompositor, andFlutterVulkanBackingStoresurfaces are used as-is. The engine-sidekFlutterBackingStoreTypeVulkanImpeller path is implemented here (it previously logged "Unimplemented").Testing
Regression tests run in
flutter_windows_unittestsandimpeller_unittests; hardware-dependent tests skip cleanly on machines without Vulkan:VulkanManagerTest(7): dynamic loading, create-or-null, LUID validity, required extensions present, no windowing extensions ever, proc resolution, teardown.VulkanImportedImageTest: argument validation.DCompPresenterTest(5): unknown LUID rejection, present-before-bind rejection, the full pipeline (allocate, import, keyed-mutex present through a live DComp tree, evict), swapchain resize in place, and an 8-cycle allocate/import/evict lifecycle.FlutterWindowsEngineTest: the backend requires the explicit switch, is ignored without Impeller, either activates (ANGLE skipped) or falls back without failing engine construction, and does not enable SDF rendering when Vulkan is active.GPUSurfaceVulkanImpeller.RenderToSurfaceDisabledYieldsNoOpFrame: with an external view embedder the root surface yields a no-op frame and never pulls an image from the delegate.Performance
Release-mode comparison of this Vulkan/DComp path against the default ANGLE/DXGI path (Impeller both sides), AMD Radeon RX 6750 XT, Windows 10, same benchmark app and binary:
Native Vulkan wins substantially on composition/clip/transform-heavy scenes (no ANGLE GL-to-D3D11 translation), and regresses on offscreen-layer-heavy scenes (save layers, shader masks), where the 1%-low frame rate drops sharply. Net a modest geomean gain with a clear optimization target (the offscreen/save-layer path) called out honestly rather than cherry-picked.
Manual verification on AMD Radeon RX 6750 XT (Windows 10): the backend activates, renders the full 9-scene benchmark suite in both debug and release AOT builds with no Vulkan validation errors under VK_LAYER_KHRONOS_validation, and presents through the DirectComposition tree with correct visuals throughout the runs. Window resizing is exercised by DCompPresenterTest.ResizeRecreatesSwapchainBuffers; because presentation goes through a single DComp commit, chrome and content resize atomically by construction.
Part of #181711. Design doc: https://flutter.dev/go/impeller-backend-desktop
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.