Skip to content

[Flutter GPU] ☂️ Compute shaders #188474

Description

@bdero

Goal

A Flutter GPU user can write a compute shader, dispatch it over a 1D, 2D, or 3D workgroup grid, and read and write storage buffers and storage textures from it. Runs on Metal, Vulkan, and desktop OpenGL, with a runtime capability flag so an app can detect support and provide a non-compute fallback where it is unavailable.

This is a tracker for the whole roadmap, linking the work items that have to land together for compute to be usable end to end from Dart. Supersedes the earlier stub #161098.

Background

Impeller already has a compute HAL (ComputePass, ComputePipeline) that runs on Metal and Vulkan, but it was built for an abandoned tessellation experiment, is unused in production, and its dispatch contract is unsuitable for a public API. impellerc already compiles .comp shaders and reflects storage buffers, but the bundle format does not serialize the metadata and the Dart surface exposes none of it. Three gaps block a good API.

  1. The dispatch contract is incoherent. ComputePass::Compute(ISize) is 2D only, means a workgroup count on Vulkan but a total invocation count on Metal, and ignores the shader's local_size, so shared memory and 3D dispatch are impossible.
  2. The GLES cross-compile output is capped at GLSL ES 3.0. Storage buffers, compute, and image load/store require GLSL ES 3.10 or desktop 430. The frontend already compiles to Vulkan SPIR-V so the features are expressible, the wall is purely the output version.
  3. A shader bundle compiles its OpenGL ES output at a single version for the whole bundle, so the GLES floor is all-or-nothing rather than per shader.

Explicit compute barriers already exist in the HAL from #140798. This effort builds on that rather than redoing it.

How other systems approach this

The design below is informed by a survey of cross-platform graphics HALs. The recurring patterns:

System Dispatch units Access declaration Sync model GL / GLES compute
WebGPU / wgpu workgroup counts per-binding read / read_write automatic (usage scopes) gated at GLES 3.1, none on WebGL2, one artifact plus capability flags
SDL_GPU workgroup counts writable set at pass begin automatic at pass boundary no GL backend at all
sokol_gfx workgroup counts reflected from shader readonly automatic (bounded per-pass tracker) GL 4.3 / GLES 3.1, none on WebGL2
bgfx workgroup counts access enum at bind time manual via view ordering GL 4.3 / GLES 3.1
Unity workgroup counts HLSL type plus RenderGraph access flags automatic in RenderGraph gated by supportsComputeShaders, none on WebGL
Godot RenderingDevice workgroup counts uniform sets explicit barriers GL renderer has no compute
Impeller today mixed per backend none explicit only disabled

Takeaways that shaped the decisions below. Workgroup-count dispatch is universal. The friendlier APIs declare resource access and insert barriers automatically. The GLES compute floor is GLES 3.1 / desktop GL 4.3 everywhere, WebGL2 has no compute anywhere, and the missing-capability fallback is always the app author's responsibility, never synthesized by the engine.

Design summary

Rendering stays at the broad floor, down to GLES 2.0 / GLSL ES 1.00. Compute and storage become an additive, runtime-gated tier whose floor is GLES 3.1 / desktop GL 4.3. Shaders are tiered by the features they use rather than by stage, so a fragment shader that reads a storage buffer is also a 3.1 shader. Below the tier, compute and storage shaders are absent, the app checks supportsCompute and branches to a non-compute path, and loading a shader whose tier the device cannot meet fails loudly rather than silently.

Key decisions.

  • Dispatch takes workgroup counts in 3D. The shader declares local_size, the compiler reflects it, it compiles into the SPIR-V and GLSL, and it is passed to Metal as threadsPerThreadgroup. This replaces the current per-backend guesswork.
  • Resource access is declared in the shader through the readonly qualifier on storage buffers and the access qualifier on storage images, captured by reflection, and normalized internally to a per-binding read or read_write access used to drive barriers.
  • Synchronization is automatic by default with an explicit escape hatch. Impeller inserts a coarse compute memory barrier between dispatches where a resource written by an earlier dispatch is read by a later one, using the global Vulkan memory barrier it already emits, a single glMemoryBarrier on GL, and encoder hazard tracking on Metal. The existing AddBufferMemoryBarrier and AddTextureMemoryBarrier remain as the advanced manual override.
  • Each shader compiles to one GLES version, the lowest that supports its features. A compute or storage shader compiles to GLES 3.1, a plain render shader to a lower version, and the runtime gates on the recorded minimum. There is no per-shader multi-variant packaging, it is never needed, a shader either needs a high-tier feature and only has the high-tier form, or it does not and the low form runs everywhere.
  • Desktop GL 4.3 is the OpenGL compute target. Windows and Linux desktop GL is where a GL compute path adds unique reach. Mobile GLES 3.1 compute is deprioritized as largely redundant with Vulkan, iOS is Metal, web is WebGPU, and macOS GL caps at 4.1.

The work splits into five groups.

Compiler and bundle format

  • impellerc compiles each shader to the lowest GLES version that supports the features it uses, GLES 3.1 / GL 4.3 for compute or storage and lower for plain render shaders.
  • The bundle flatbuffer carries workgroup size, storage-buffer descriptors, reflected access, and each shader's minimum GLES tier so the runtime can gate or fall back.
  • The reflector emits workgroup size and storage-buffer bindings into generated headers.

HAL correctness

  • Replace Compute(ISize) with a 3D workgroup-count dispatch that honors the reflected local_size, dropping the Vulkan specialization-constant override and the Metal halving heuristic.
  • Add automatic hazard tracking with a bounded per-pass write set, keeping the explicit barrier methods as the override.
  • Add storage texture (image load/store) bindings with read and read_write access.
  • Add indirect dispatch.

OpenGL backend

  • A desktop GL 4.3 compute pass over glDispatchCompute and glMemoryBarrier, with the GLES capability check returning true at GL 4.3+ and GLES 3.1+. Removes the incorrect "not until GLES 3.2" assumption. Mobile GLES 3.1 is a later follow-up.

Dart API

  • Storage-buffer bindings on DeviceBuffer.
  • ComputePipeline and CommandBuffer.createComputePass, with dispatch(x, y, z) and indirect dispatch.
  • A supportsCompute capability flag and fallback guidance, with a loud error when a shader's tier is unmet.

Examples and docs

  • A compute sample and an end-to-end test across Metal, Vulkan, and desktop GL 4.3, plus flutter_gpu_shaders docs for the compute and storage tier.

Status

Already in place

Wave 1, Metal end to end

Wave 2, Vulkan and HAL completeness (to file)

  • complete and validate the Vulkan compute backend for the new dispatch contract
  • automatic compute hazard tracking, with the explicit barriers as the manual override
  • storage texture bindings with read and read_write access
  • indirect dispatch

Wave 3, desktop OpenGL (to file)

  • impellerc emits a GLES 3.1 / GL 4.3 output tier and records each shader's minimum GLES tier in the bundle
  • desktop GL 4.3 compute backend and capability gate

Wave 4, gating and docs (to file)

  • supportsCompute capability flag, loud tier error, and fallback guidance
  • flutter_gpu_shaders docs for the compute and storage tier (downstream)

Sequencing

The plan is Metal first, to get a working end-to-end slice as fast as possible,
then Vulkan, then desktop GL, then gating and docs.

  1. Wave 1, Metal. Bundle metadata ([Flutter GPU] shader bundle carries compute workgroup size and storage buffer bindings #188477) and the dispatch fix ([Impeller] Compute dispatch takes 3D workgroup counts and honors the shader local_size #188478) in
    parallel, then storage buffer bindings ([Flutter GPU] Storage buffer bindings #188479), then the compute API
    ([Flutter GPU] ComputePipeline and ComputePass with dispatch #188480), then the example and test ([Flutter GPU] Compute example and Metal end-to-end test #188481).
  2. Wave 2, Vulkan. Validate the dispatch contract on Vulkan, then automatic
    hazard tracking, storage textures, and indirect dispatch.
  3. Wave 3, desktop GL. The impellerc output tier, then the GL 4.3 backend.
  4. Wave 4. The supportsCompute gate and the package docs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    c: new featureNothing broken; request for a new capabilitye: impellerImpeller rendering backend issues and features requestsengineflutter/engine related. See also e: labels.flutter-gputeam-fluttergpuOwned by Flutter GPU team

    Type

    No type

    Projects

    Status
    💡 Features

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions