[Impeller][Windows] drawRect with a FragmentShader takes 3 render passes and 2 MSAA resolves: 30 ms against 2.7 ms on Skia #192994

Summary

On Windows, Impeller runs the UberSDF path by default. Canvas::DrawRect sends a rectangle to UberSDF, and UberSDF evaluates only two color sources directly: linear and radial gradients. Every other color source falls back to a three pass sequence: a white SDF coverage mask into an offscreen, the color source into a second offscreen, a SrcIn blend of the two into a third target, and a copy of the result into the main pass.

For N such rectangles this is 3N + 1 render passes, 5N draw commands and 2N MSAA resolves.

Measured on the example below, 140 rectangles covering a 2560x1440 window: raster time is 30.0 ms per frame at 33 FPS, against 2.7 ms at 120 FPS with Skia in the same app on the same machine. A build that keeps runtime effects off the SDF path draws the same scene in 2.5 ms at 120 FPS. The cost scales with the number of rectangles, not with the shaded area.

The same fallback also darkens shared edges that land on a fractional device row, and it drops the last device row at the bottom of the window. Skia leaves a milder seam of its own on those boundaries, so this part is a comparison rather than a clean defect, and it is reported at the end because the single pass builds remove it along with the 27 ms.

Flutter already fixed exactly this pattern for gradients in #192124, which replaced the same multi pass fallback with direct evaluation in UberSDF and took linear gradient raster p50 from 19.094 ms to 5.665 ms. Runtime effects, image shaders, conical and sweep gradients were left on the fallback.

This is in the released 3.47 series, not only on master.

Environment

Every number below compares variants measured on the same machine in the same session.

Minimal reproduction

pubspec.yaml:

flutter:
  uses-material-design: true
  shaders:
    - shaders/effect.frag

shaders/effect.frag:

#include <flutter/runtime_effect.glsl>

#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
#else
precision mediump float;
#endif

out vec4 fragColor;

uniform vec2 uSize;
uniform float uTime;

void main() {
  vec2 p = FlutterFragCoord().xy / max(uSize, vec2(1.0));
  float signal = sin((p.x * 18.0 + p.y * 0.65 + uTime * 0.35) * 6.2831853);
  fragColor = vec4(vec3(0.2, 0.55, 0.95) * (0.72 + 0.28 * signal), 1.0);
}

lib/main.dart:

import 'dart:ui' as ui;

import 'package:flutter/material.dart';
import 'package:flutter/scheduler.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();
  final ui.FragmentProgram program =
      await ui.FragmentProgram.fromAsset('shaders/effect.frag');
  _printFrameTimes();
  runApp(App(program));
}

void _printFrameTimes() {
  final List<int> raster = <int>[];
  final Stopwatch watch = Stopwatch()..start();
  SchedulerBinding.instance.addTimingsCallback((List<FrameTiming> timings) {
    raster.addAll(
      timings.map((FrameTiming t) => t.rasterDuration.inMicroseconds),
    );
    if (watch.elapsedMilliseconds < 1000 || raster.isEmpty) {
      return;
    }
    raster.sort();
    final double median = raster[raster.length ~/ 2] / 1000;
    debugPrint('frames=${raster.length} '
        'fps=${(raster.length * 1000 / watch.elapsedMilliseconds).toStringAsFixed(1)} '
        'raster_median_ms=${median.toStringAsFixed(3)}');
    raster.clear();
    watch.reset();
  });
}

class App extends StatefulWidget {
  const App(this.program, {super.key});

  final ui.FragmentProgram program;

  @override
  State<App> createState() => _AppState();
}

class _AppState extends State<App> with SingleTickerProviderStateMixin {
  late final AnimationController animation = AnimationController(
    vsync: this,
    duration: const Duration(seconds: 8),
  )..repeat();

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      debugShowCheckedModeBanner: false,
      home: ColoredBox(
        color: const Color(0xff07111f),
        child: CustomPaint(
          painter: EffectPainter(widget.program.fragmentShader(), animation),
          child: const SizedBox.expand(),
        ),
      ),
    );
  }
}

class EffectPainter extends CustomPainter {
  EffectPainter(this.shader, this.animation) : super(repaint: animation);

  final ui.FragmentShader shader;
  final Animation<double> animation;
  final Paint fillPaint = Paint();

  @override
  void paint(Canvas canvas, Size size) {
    const int columns = 14;
    const int rows = 10;
    final double width = size.width / columns;
    final double height = size.height / rows;
    for (int index = 0; index < columns * rows; index++) {
      shader
        ..setFloat(0, width)
        ..setFloat(1, height)
        ..setFloat(2, animation.value * 8 + index * 0.013);
      fillPaint.shader = shader;
      canvas.drawRect(
        Rect.fromLTWH(
          (index % columns) * width,
          (index ~/ columns) * height,
          width,
          height,
        ),
        fillPaint,
      );
    }
  }

  @override
  bool shouldRepaint(EffectPainter oldDelegate) => false;
}

Run it maximized:

flutter build windows --release
build\windows\x64\runner\Release\<app>.exe

Compare against Skia by building the same app with Impeller disabled.

The numbers below come from this code with two changes for unattended runs: the print above is replaced by a line with frames, FPS, mean raster and p90 raster once a second, and the grid size comes from two environment variables so one build can measure several rectangle counts. A third variable stops the animation at a fixed phase, which is what makes two engines produce comparable screenshots.

Expected results

A drawRect whose paint carries a FragmentShader should cost about one render pass and one draw command, as it did before Windows moved to the UberSDF path and as it still does when the same rectangle is filled with a solid color. Frame time should follow the shaded area, not the number of rectangles.

Actual results

Each such rectangle costs three extra render passes, four extra draw commands and two MSAA resolves. 140 of them take 30.0 ms of raster time at 33 FPS, against 2.7 ms at 120 FPS with Skia and 2.5 ms at 120 FPS with a single pass Impeller build.

Measured results

Every row is the median of per second samples, first four seconds of each run dropped, medians taken per run and then across runs. Impeller and Skia runs were interleaved.

The window content is 2534x1369 device pixels and the grid always covers all of it, so every row below shades the same pixels with the same shader and only the number of drawRect calls changes. 5 runs per cell.

uSize is the size of the rectangle, so the fragment program runs the same instructions per pixel in every row, only the period of the sine changes. The total area of the intermediate snapshots does not grow either: it is always about two windows.

What the engine does per rectangle

Engine labels for one runtime effect rectangle:

1. PipelineBlend(Dst): one UberSdf Pipeline V#1 command, one MSAA resolve
2. PipelineBlend(Src): one Runtime Stage command, one MSAA resolve
3. Pipeline Blend Filter: two TextureFill commands into a target without MSAA
4. EntityPass Render Pass: one TextureFill command per logical draw

Instrumented release build, 140 rectangles, steady frames:

Medians in microseconds per frame from the same build, nested rows are not additive:

The resolves are half the frame and the command loops of the extra passes are most of the rest. Uniform binding is 4% of it: I also packed the user uniforms of a six shader production workload, which cut active glUniform* calls per frame from 859 to 580, and raster time moved by 1.16%. This is not a uniform binding problem.

Where it comes from

Canvas::DrawRect sends every antialiased rectangle to UberSDF (canvas.cc:1071), and Canvas::AddRenderSDFEntityToCurrentPass decides how the color source is evaluated (canvas.cc:2300-2335):

  if (!paint.color_source || params.gradient.has_value()) {
    // No color source (solid paint color), or a supported gradient color
    // source.
    ...
  } else {
    // Color source not directly supported by UberSDF (e.g. image, runtime
    // effect, or an unsupported gradient type). Render a solid white mask with
    // UberSDF and blend with ColorSourceContents.
    params.color = Color::White();
    ...
    std::shared_ptr<Contents> final_contents = ColorFilterContents::MakeBlend(
        BlendMode::kSrcIn, {FilterInput::Make(std::move(uber_sdf_contents)),
                            FilterInput::Make(color_source_contents)});

CreateUberSDFGradientParameters returns parameters only for linear and radial gradients, and only when the gradient matrix is affine, with the extra condition of uniform scale for radial gradients. Everything else reaches the else branch.

Scope

Color sources on the fallback: runtime effects, ImageShader, conical gradients, sweep gradients, and linear or radial gradients with a non affine matrix or non uniform scale.

Draw calls that go through UberSDF and therefore through the same fallback: DrawLine, DrawRect, DrawOval, DrawRoundRect, DrawRoundSuperellipse, DrawCircle.

A second app draws the same 140 rectangles of 181x137 and only changes the color source. Impeller on master, 3 runs each, medians:

The three fallback sources cost 30 to 37 ms, and a solid color over the same pixels costs 2.7 ms.

One caveat on the gradient rows: this app allocates a new Gradient per rectangle per frame, so those two numbers include building a gradient color ramp texture 140 times per frame, which is what #192948 and #192962 are about. Their pass count is already one per rectangle after #192124, which is why their p90 spikes come from allocation rather than from resolves.

import 'dart:io';
import 'dart:ui' as ui;

import 'package:flutter/material.dart';

final String source = Platform.environment['BW_SOURCE'] ?? 'shader';

class SourcePainter extends CustomPainter {
  SourcePainter(this.shader, this.texture, this.animation)
      : super(repaint: animation);

  final ui.FragmentShader shader;
  final ui.Image texture;
  final Animation<double> animation;
  final Paint fillPaint = Paint();

  static const List<Color> _colors = <Color>[
    Color(0xff3399ee),
    Color(0xff22ddaa),
    Color(0xffffcc55),
  ];
  static const List<double> _stops = <double>[0, 0.5, 1];

  @override
  void paint(Canvas canvas, Size size) {
    const int columns = 14;
    const int rows = 10;
    final double width = size.width / columns;
    final double height = size.height / rows;
    for (int index = 0; index < columns * rows; index++) {
      final Rect rect = Rect.fromLTWH(
        (index % columns) * width,
        (index ~/ columns) * height,
        width,
        height,
      );
      switch (source) {
        case 'solid':
          fillPaint.shader = null;
          fillPaint.color = _colors[index % _colors.length];
        case 'linear':
          fillPaint.shader = ui.Gradient.linear(
              rect.topLeft, rect.bottomRight, _colors, _stops);
        case 'radial':
          fillPaint.shader = ui.Gradient.radial(
              rect.center, rect.shortestSide / 2, _colors, _stops);
        case 'sweep':
          fillPaint.shader = ui.Gradient.sweep(rect.center, _colors, _stops);
        case 'image':
          fillPaint.shader = ui.ImageShader(texture, TileMode.repeated,
              TileMode.repeated, Matrix4.identity().storage);
        default:
          shader
            ..setFloat(0, width)
            ..setFloat(1, height)
            ..setFloat(2, animation.value * 8 + index * 0.013);
          fillPaint.shader = shader;
      }
      canvas.drawRect(rect, fillPaint);
    }
  }

  @override
  bool shouldRepaint(SourcePainter oldDelegate) => false;
}

When it started, and who is affected

#184090 documented the cost when it landed: "This adds gradient rendering to the SDF renders by performing gradients as a multi rendering pass. The shape is rendered and the gradient is rendered then they are combined with a porter duff blend. This is suboptimal to performing the gradients directly in the fragment shader. Since we are working on desktop computers there is not a lot of pressure to implement that directly since it comes at a higher maintenance cost. We can revisit that decision once the ubersdf renderer has all the shapes."

#187877 is an ancestor of the 3.47.0 tag, and Impeller is the default on Windows in that series (enable_impeller = true in flutter_windows_engine.cc), so on Windows this arrived for users with 3.47.

Platform defaults today: Windows pushes --impeller-use-sdfs=true itself, macOS and iOS read an opt in flag, Android and Linux never enable it. So the bug is a Windows default and an opt in elsewhere.

Measured on the same machine with the same Dart code, 3 runs each: stable 3.47.4 renders the 140 rectangle grid at 33.6 FPS and 29.726 ms, master at 33.2 FPS and 30.133 ms. The frozen frames of the two are identical, 0 differing pixels out of 2560x1440, seam included. The release behaves exactly like master.

Side effect of the same path: edges on fractional rows

This part is secondary to the frame time, and it is not a clean Impeller only defect, so here is what the pixels say.

The window content is 2534x1369 device pixels. With 14 columns and 10 rows, 2534 / 14 = 181.0 exactly and 1369 / 10 = 136.9, so vertical edges of neighbouring rectangles land on whole pixels and horizontal edges do not. On the vertical edges all three renderers agree to within one unit per channel. On the horizontal ones they do not.

Seam depth per interior boundary, measured on frozen frames of the same animation phase as the brightness of the shared row against the mean of the rows three pixels above and below, averaged over 2200 columns:

Impeller leaves three visible seams and the deepest is 23% below its neighbours, Skia leaves a shallow one on every boundary and never passes 11%, and the single pass Impeller build leaves none. Pixel values on the worst boundary, same column: master (28, 76, 131), Skia (37, 101, 175), single pass (48, 131, 227), with neighbours near (50, 137, 236) and (46, 126, 217) and the background at (7, 17, 31).

The two single pass Impeller builds, the one that keeps runtime effects off UberSDF and the one with UberSDF switched off, are identical to each other, and stable 3.47.4 is identical to master.

One row is unambiguous. At the bottom edge of the window the fallback drops the last device row of the bottom rectangles and shows the background instead:

Hypothesis for both, not yet proven: the mechanism of #192980, where a blend subpass texture is truncated to an integer size and composited back at a fractional origin with nearest sampling. The fallback goes through the same ColorFilterContents::MakeBlend machinery, and the deviation appears only on the axis whose boundary is fractional. Whatever the cause, the single pass builds remove it together with the 27 ms, which is why it is reported here rather than separately.

The attached image is a 4x crop of one boundary, master above and the single pass build below.

What I tried

Six builds, one executable, the dll swapped between runs, 3 rounds interleaved, medians:

The official artifact and the local build agree, so the local toolchain is not a factor.

1. Keep runtime effects off the SDF path. The complete diff that was measured:

--- a/engine/src/flutter/impeller/display_list/canvas.cc
+++ b/engine/src/flutter/impeller/display_list/canvas.cc
@@ -1068,7 +1068,10 @@ void Canvas::DrawRect(const Rect& rect, const Paint& paint) {
     }
   }
 
-  if (renderer_.GetContext()->GetFlags().use_sdfs &&
+  if ((!paint.color_source ||
+       paint.color_source->type() !=
+           flutter::DlColorSourceType::kRuntimeEffect) &&
+      renderer_.GetContext()->GetFlags().use_sdfs &&
       IsCompatibleWithSDFRendering(paint, GetCurrentTransform())) {
     Rect effective_rect = rect;
     Color effective_color = paint.color;

It turns 421 passes, 700 commands and 280 resolves into one pass, 140 commands and no resolves, and its output is identical to a build with UberSDF switched off, seam and bottom row included. It is a diagnostic, not a proposed fix: it drops ExpandRectToPixelMinimum, which only exists on the SDF path, so rectangles thinner than one device pixel are no longer expanded, and it replaces analytic SDF coverage with MSAA coverage, which changes edges under rotation and subpixel translation. The goldens it would move include AiksTest.CanRenderClippedRuntimeEffects, AiksTest.CanRenderRuntimeEffectFilter, AiksTest.ClippedBackdropFilterWithShader and AiksTest.RuntimeEffectVectorArray in their MetalSDF and OpenGLESSDF configurations.

2. Drop MSAA from the two intermediate snapshots.

-        BlendMode::kSrcIn, {FilterInput::Make(std::move(uber_sdf_contents)),
-                            FilterInput::Make(color_source_contents)});
+        BlendMode::kSrcIn,
+        {FilterInput::Make(std::move(uber_sdf_contents), false),
+         FilterInput::Make(color_source_contents, false)});

30.240 ms goes to 18.444 ms and the frozen frame is byte identical to master, including the seam. The analytic mask already carries the coverage, so MSAA on these two snapshots buys nothing here. It removes 2N resolves, and it leaves 3N+1 passes and 5N commands, so it cannot reach the single pass result. It is a safe intermediate step, but it needs its own golden run.

3. --impeller-use-sdfs=false does nothing. The Windows embedder checks for that exact string before pushing its own --impeller-use-sdfs=true (flutter_windows_engine.cc:306-311), but the setting is parsed with HasOption (switches.cc:567-568), which only tests whether the flag is present:

  settings.impeller_use_sdfs =
      command_line.HasOption(FlagForSwitch(Switch::kImpellerUseSdFs));

So passing false enables the SDF path. A build that simply does not push the switch was needed to measure it. Worth fixing on its own: today there is no way to turn UberSDF off from an application, and in release builds GetSwitchesFromEnvironment is compiled out as well.

What a production fix should preserve

• antialiasing for subpixel translated and rotated edges
• perspective transforms, fill and stroke behaviour
• rectangles thinner than one device pixel, including ExpandRectToPixelMinimum and the work in UberSDF rect handling for thin (line-like) rectangles #188821 and Thin UberSDF rects are sometimes invisible #188591
• paint alpha, color filters, image filters, mask filters, clips, non default blend modes
• runtime effects with and without image samplers
• wide gamut output, which is what reverted the first gradient attempt in Revert: Handle linear and radial gradients in UberSDF #191913

Preferred direction

Evaluate the color source inside the UberSDF pass, as #192124 did for linear and radial gradients, so coverage masks the color source in one fragment pass. For runtime effects this is harder than for gradients because the user program is dynamic; a generated wrapper around the runtime entry point, or an analytic coverage input in the runtime effect pipeline, are both plausible, and the engine owners are better placed to choose. Image shaders, conical and sweep gradients are on the same fallback and would benefit from the same interface. The gradient work in flight, #192948 and #192962, suggests this direction is already being extended.

Related issues

• [Windows][Impeller] Scrolling basic LinearGradient decorations causes severe raster slowdown #190401: the gradient form of the same problem, fixed by Handle linear and radial gradients in UberSDF (re-implement after fix) #192124. This issue is the remaining form.
• [Impeller][Android] GLES backend re-issues all GL state per draw (no state cache): raster thread CPU-bound, 2–3× slower than Skia GL on Adreno 610 #192147: redundant GLES state per draw command. State caching lowers the cost of each of the 700 commands, it does not remove the extra 420 passes or the 280 resolves.
• [Windows][Impeller] BackdropFilter blur has significantly higher raster cost than Skia and bounded blur loses most grouping benefit #191207 and Windows app significantly slower when impeller enabled (Gone when on skia) #191353: Windows render passes are expensive, and the general report that Windows is slower with Impeller. Those are about the cost of a pass. This issue is about their count, which is backend independent, so the two compose.
• Variety of issues with Impeller drawing thin lines/rects #188590, Thin UberSDF rects are sometimes invisible #188591, UberSDF rect handling for thin (line-like) rectangles #188821: known fidelity differences for thin rectangles, which is why the diagnostic bypass must not be taken as the fix.
• [Impeller] Advanced blend modes wobble: the blend subpass texture is truncated to an integer size and composited back at a fractional origin #192980: the integer truncation of a blend subpass texture, the likely mechanism behind the seam measured here.

Method

Release AOT builds throughout. Frame statistics come from FrameTiming once a second, the first four samples of every run are dropped, each cell is the median of per second values per run and then the median across runs, and variants were interleaved round by round. Engine counters and timers come from a local instrumentation patch with FML_LOG(ERROR), built in release as well. Frozen frames were captured with the animation stopped at a fixed phase so two engines produce comparable images, and compared with ImageMagick.

Zoom in to see the difference:
Impeller stable:

Impeller master:

Skia stable:

Impeller with diagnosis diff tried: