Skip to content

[Flutter GPU] Expand vertex attribute formats and gate them by backend capability #186309

Description

@bdero

Related to #145013. Closes #102778 once landed.

The VertexFormat enum shipped in #186310 only covers the 32-bit-per-component scalar types (float32, uint32, sint32 in 1, 2, 3, and 4 component widths). This is enough for hand-rolled vertex buffers and most flutter_gpu samples, but it forces every glTF, KTX2, or other industry-format asset to be expanded to 32-bit-per-component on the CPU before upload. It also prevents storing the dart:ui Color packed-byte representation directly in a vertex stream.

This issue covers both halves of the work, adding the formats and defining how a caller learns which of them the current device can use. The second half matters because the GLES backend targets a GLES 2.0 floor and cannot support all of them, while every other backend can.

Formats to add

Each adds only a new enum entry (no breaking change to existing ones), and each keeps its bytesPerElement and componentCount getters. Grouped by the GLES tier they fall into (see Backend capability gating below).

Ungated, GLES 2.0 core:

  • 8-bit normalized: unorm8, unorm8x2, unorm8x4, snorm8, snorm8x2, snorm8x4.
  • 8-bit packed BGRA: unorm8x4Bgra (the canonical dart:ui Color packed format, swizzled).
  • 16-bit normalized: unorm16, unorm16x2, unorm16x4, snorm16, snorm16x2, snorm16x4.

GLES 3.0 core, or a GLES 2.0 extension:

  • 16-bit half-float: float16, float16x2, float16x4.
  • 10/10/10/2 packed: unorm10_10_10_2 (compact normalized normals and tangents).

GLES 3.0 only (no GLES 2.0 path):

  • 8-bit integer inputs: uint8, uint8x2, uint8x4, sint8, sint8x2, sint8x4.
  • 16-bit integer inputs: uint16, uint16x2, uint16x4, sint16, sint16x2, sint16x4.
  • The already-shipped uint32/sint32 belong to this tier too (see the note under Backend capability gating).

Native only (no GLES backend):

  • 64-bit float: float64, float64x2, float64x3, float64x4.

Validation continues to enforce that the shader's declared scalar type class (float, signed int, unsigned int) matches the supplied format. Component-count substitution rules (missing components fill to (0, 0, 0, 1)) stay backend-default.

Backend capability gating

Among the backends Flutter GPU targets, only GLES constrains the vertex-format set. Metal, Vulkan, and the WebGPU-class web backend carry the full normalized, packed, and half-float set as core. The wider ecosystem treats it the same way, which is why this is GLES-specific.

  • WebGPU makes every GPUVertexFormat mandatory core with no feature flag, in deliberate contrast to texture formats, which have a rich optional tier. Its hardware floor is GLES3/D3D11/Metal-class, so it never needs a vertex-format gate.
  • wgpu gates only the Float64 family behind VERTEX_ATTRIBUTE_64BIT (native only), and has no downlevel flag for any other vertex format, because its GLES baseline is GLES3/WebGL2.
  • Filament expresses vertex types as a flat ElementType plus per-attribute FLAG_NORMALIZED and FLAG_INTEGER_TARGET, and gates by FeatureLevel, where FEATURE_LEVEL_0 is GLES2 and integer-target attributes (glVertexAttribIPointer) require FEATURE_LEVEL_1 (GLES3).

Impeller is the one backend that genuinely targets a GLES 2.0 floor (the GLES BUILD.gn notes that GLES3 features are used only when the driver supports them), so it is the one that needs a real gate. The tiers, and the notable part that the highest-value compression formats are free even on the GLES 2.0 floor:

Tier Formats GLES requirement
Ungated (ES2 core) unorm8/snorm8, unorm16/snorm16, unorm8x4Bgra, float32 glVertexAttribPointer with normalized is core in GLES 2.0
ES3 core or ES2 extension float16, unorm10_10_10_2 GLES 3.0 core, or OES_vertex_half_float / OES_vertex_type_10_10_10_2 on ES2
ES3 required, no ES2 path integer inputs uint8/sint8/uint16/sint16/uint32/sint32 glVertexAttribIPointer, GLES 3.0 only (GLSL ES 1.00 has no integer vertex inputs)
Native only float64 not available on any GLES backend

Note on the already-shipped integer formats. uint32/sint32 from #186310 are in this same ES3-only tier, and the GLES backend currently rejects them (ToVertexAttribType in formats_gles.h returns no GL type for integer scalar types, and there is no glVertexAttribIPointer path). So they are effectively Metal and Vulkan only today, and would fail at pipeline creation on GLES. The capability surface below should cover them, not just the new formats.

Proposed capability surface

Mirror the existing texture-format precedent. GpuContext already exposes supportsTextureFormat(PixelFormat, {renderTarget, shaderWrite}) and supportsTextureCompression(TextureCompressionFamily), backed by Impeller Capabilities. Add the analogous vertex query.

// flutter_gpu (Dart)
bool supportsVertexFormat(VertexFormat format);

backed by Capabilities::SupportsVertexFormat(...) Impeller-side. Per-format is preferred over a single feature-level integer, for the same reason texture formats are per-format. It stays forward-compatible as new formats land, and it hides the backend's internal tiering from callers.

Open question for the team. A single supportsVertexFormat predicate, or a coarser family/feature-level enum like Filament. Per-format reads cleaner for callers, family flags are cheaper to compute and map directly onto the tiers above.

Failure mode. Requesting an unsupported format should fail at pipeline creation with a clear message, consistent with the existing validation in BuildCustomVertexDescriptor. Callers that care branch on supportsVertexFormat first.

HAL dependency

Today ShaderStageIOSlot describes the format as a (ShaderType, bit_width, vec_size, columns, relaxed_precision) tuple, which cannot represent normalization or packed formats and has no way to mark bgra swizzle. Two options.

  1. Add a flat VertexAttributeFormat enum on ShaderStageIOSlot that captures all of the above directly.
  2. Add a normalize bit and a separate packed_format enum on ShaderStageIOSlot to retrofit on top of the current decomposed shape.

Option 1 is cleaner and matches what every HAL surveyed ultimately uses. WebGPU GPUVertexFormat, wgpu VertexFormat, and Filament ElementType are all flat format enums. The capability gate keys on this enum.

Impeller-side work this implies

Grounded in the current source.

  • ShaderStageIOSlot (impeller/core/shader_types.h) gains the flat VertexAttributeFormat (option 1). The format check in BuildCustomVertexDescriptor (lib/gpu/render_pipeline.cc, currently a TODO pointing at this issue) verifies that enum instead of checking type and bit_width.
  • The GLES backend hardcodes normalized = GL_FALSE (buffer_bindings_gles.cc) and ToVertexAttribType (formats_gles.h) returns no GL type for int, half, and double. The tiers above mean setting normalized from the format, calling glVertexAttribIPointer for integer inputs, and mapping HALF_FLOAT and *_2_10_10_10_REV with their OES_* ES2 fallbacks.
  • CapabilitiesGLES already derives per-feature flags from DescriptionGLES via GetGlVersion().major_version >= 3 and HasExtension(...). The vertex-format flags slot in the same way, and the instance-divisor path (core VertexAttribDivisor with a VertexAttribDivisorEXT ES2 fallback) is the exact precedent for the ES3-or-extension tier.
  • Metal and Vulkan return true for the first three tiers unconditionally. float64 stays false on GLES and behind a native-only flag elsewhere.

Implementation plan

Landing as a stack of three PRs. The split is for review surface and bisectability, not code volume, since much of the infrastructure (the VertexLayout plumbing, the Capabilities pattern, the GLES version and extension checks) already exists. PR 2 and PR 3 can be combined if reviewers prefer; PR 1 should stay on its own.

  • PR 1, HAL representation (keystone). Add a flat VertexAttributeFormat enum to ShaderStageIOSlot (option 1 above) and map it in every backend (Metal, Vulkan, GLES). Behavior-preserving, only the existing 32-bit formats are reachable, so it is a pure refactor with no new user-facing formats. Round-trip reflection test. Landed in [Impeller] Add a flat VertexAttributeFormat for vertex inputs #188684.
  • PR 2, GLES backend completion and capability gate. Add Capabilities::SupportsVertexFormat(...) with the tiered CapabilitiesGLES computation (version and extension checks, modeled on the instance-divisor precedent). Finish the GLES binding path, setting normalized from the format, calling glVertexAttribIPointer for integer inputs, and mapping HALF_FLOAT and *_2_10_10_10_REV with their OES_* ES2 fallbacks. Also fixes the existing uint32/sint32-on-GLES rejection noted above. Capability-tier tests.
  • PR 3, Dart surface. Expand the VertexFormat enum with the new values, extend kVertexFormatTable and the BuildCustomVertexDescriptor validation, and expose GpuContext.supportsVertexFormat with its native binding. Dart enum and supportsVertexFormat tests.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    c: new featureNothing broken; request for a new capabilityc: proposalA detailed proposal for a change to Flutterengineflutter/engine related. See also e: labels.flutter-gputeam-fluttergpuOwned by Flutter GPU team

    Type

    No type

    Projects

    Status
    💡 Features

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions