Goldens images changed after upgrade to macOS 26.4

Steps to reproduce

It seems most of our golden test images changed after i updated to macOS 26.4. I'm not 100% sure this is caused by the upgrade, but i can't see what else has changed. What is even stranger i can't really see any changes to the images when looking at them. Maybe the font rendering has slightly changed ? See the header for "News" and "My Bookings", i think one looks slightly thicker.

This was caused by running: flutter test --update-goldens which runs the testGoldens functions.

I can't actually roll back to a previous macOS version to test this.

The goldens changed using both Flutter version 3.38.4 and 3.41.5 (latest).

Expected results

I didn't expect most of the goldens images in our codebase to change.

Actual results

Most of our goldens images changed.

Code sample

I can't really create a reproducible sample as i'm not able to test this on a previous macOS version at the moment and i can't share our proprietary code (which is quite complex anyways).

[Paste your code here]

Screenshots or Video

Screenshots / Video demonstration

Before and after:

Logs

[Paste your logs here]

Flutter Doctor output

Doctor output

[✓] Flutter (Channel stable, 3.41.5, on macOS 26.4 25E246 darwin-arm64, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 36.1.0)
[✓] Xcode - develop for iOS and macOS (Xcode 26.3)
[✓] Chrome - develop for the web
[✓] Connected device (2 available)
[✓] Network resources

Unfortunately, this is a known drawback of golden testing: these tests are highly sensitive to the environment and are generally only reliable when run on a machine with the exact same configuration (OS version, OS settings, etc) as the one used to produce the original image.

In this case, it appears that macOS 26.4 introduced changes to system-level font rendering. This effectively invalidates golden images produced on macOS 26.3 or older if they contain text, particularly when using custom fonts. Because the Flutter team does not control how the underlying operating system renders fonts, this isn't something we can fix in the framework itself.

If your golden files use custom fonts and t…

@erf

@erf Thanks for the report!

To help us investigate, could you please provide:

A minimal reproducible example (small widget with golden test)
Before/after golden images highlighting the differences as we can barely see any difference in the provided images.
Whether the issue reproduces on another machine or CI environment
Whether specific fonts are used (custom vs system fonts)

My colleague was still running macOS version 26.3.1 and helped me create a minimal golden test (I'll include it at the end).

I can confirm that the goldens he produced on 26.3.1 fails when i run it on my machine 26.4. Here is the before and after image and the test output. Notice there is only a 18px diff detected which is small and probably why you can't see the difference.

BTW maybe consider running these golden tests on various platform versions before releasing to stable.

Golden on 26.3.1

Golden on 26.4

Output from running fvm flutter test (fvm is set to run Flutter version 3.38.4, but it should be the same result on later versions)

00:02 +0: Minimal example
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown while running async test code:
Golden "goldens/minimal_example.png": Pixel test failed, 0.00%, 18px diff detected.
Failure feedback can be found at /Users/erlend/projects/membro_medlem_app/test/widgets/failures

When the exception was thrown, this was the stack:
#0      LocalFileComparator.compare (package:flutter_test/src/_goldens_io.dart:108:5)
<asynchronous suspension>
#1      MatchesGoldenFile.matchAsync.<anonymous closure> (package:flutter_test/src/_matchers_io.dart:124:32)
<asynchronous suspension>
<asynchronous suspension>
#3      _expect.<anonymous closure> (package:matcher/src/expect/expect.dart:123:26)
<asynchronous suspension>
<asynchronous suspension>
#5      expectLater.<anonymous closure> (package:flutter_test/src/widget_tester.dart:508:19)
<asynchronous suspension>
#6      main.<anonymous closure> (file:///Users/erlend/projects/membro_medlem_app/test/widgets/minimal_example_test.dart:108:5)
<asynchronous suspension>
#7      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:192:15)
<asynchronous suspension>
#8      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1059:5)
<asynchronous suspension>
<asynchronous suspension>
(elided 3 frames from dart:async and package:stack_trace)

The exception was caught asynchronously.
════════════════════════════════════════════════════════════════════════════════════════════════════
00:02 +0 -1: Minimal example [E]
  Test failed. See exception logs above.
  The test description was: Minimal example


To run this test again: /Users/erlend/fvm/versions/3.38.4/bin/cache/dart-sdk/bin/dart test /Users/erlend/projects/membro_medlem_app/test/widgets/minimal_example_test.dart -p vm --plain-name 'Minimal example'
00:02 +0 -1: Some tests failed.

Here is the full test example:

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('Minimal example', (tester) async {
    WidgetsFlutterBinding.ensureInitialized();
    await tester.pumpWidget(
      MaterialApp(
        theme: ThemeData(fontFamily: 'Open Sans'),
        home: Scaffold(
          appBar: AppBar(title: const Text('A small example')),
          floatingActionButton: FloatingActionButton(
            onPressed: () {},
            child: const Icon(Icons.add),
          ),
          body: const Column(
            spacing: 16,
            children: [
              Text('Title of screen'),
              Card(
                child: ListTile(
                  title: Text('This is an example screen'),
                  subtitle: Text('This is used to create a minimal example.'),
                ),
              ),
              Card(
                color: Colors.blueAccent,
                child: SizedBox(
                  height: 250,
                  child: Center(
                    child: Text(
                      'Large widget test',
                      style: TextStyle(color: Colors.white, fontSize: 32),
                    ),
                  ),
                ),
              ),
            ],
          ),
        ),
      ),
    );
    await expectLater(
      find.byType(Card).first,
      matchesGoldenFile('goldens/minimal_example.png'),
    );
  });
}

@erf

@erf Thanks for following up. I'm having trouble reproducing the issue because the provided code depends on missing files like FontManifest.json and goldens/minimal_example.png

Please provide a fully self-contained example so I can investigate further. Also, confirm whether any specific fonts are used (custom vs system fonts).

We are seeing the same issue

@erf

Hi @erf @saurabh-mirajkar

Simple reproducible sample:
https://github.com/piotrmoskala/golden_compare_sample

Goldens inside are generated on macOS 26.4, we tested it with @timcreatedit who runs macOS 15 and it generated different results.

@piotrmoskala

@erf Thanks for following up. I'm having trouble reproducing the issue because the provided code depends on missing files like FontManifest.json and goldens/minimal_example.png

Please provide a fully self-contained example so I can investigate further. Also, confirm whether any specific fonts are used (custom vs system fonts).

TheFontManifest.json should be included if you create a new project and add the example . The minimal_example.png is the first image included in my original post. We use the custom font "Open Sans" .

If you can't use the standalone example above that @piotrmoskala provided, then we'll try and make something over the weekend.

We're also being blocked by this

Here's the workaround my teams have used in the past to get beyond very minor golden diffs (use with caution and ideally only temporarily):

// ignore_for_file: avoid_print
// This code is pulled from https://rows.com/blog/post/writing-a-localfilecomparator-with-threshold-for-flutter-golden-tests
// and used to account for differences in goldens between developer/ci machines for whatever reasons. Use with caution.

import 'package:flutter/foundation.dart';
import 'package:flutter_test/flutter_test.dart';

/// Works just like [LocalFileComparator] but includes a [threshold] that, when
/// exceeded, marks the test as a failure.
class LocalFileComparatorWithThreshold extends LocalFileComparator {
  LocalFileComparatorWithThreshold(super.testFile, this.threshold) : assert(threshold >= 0 && threshold <= 1);

  /// Threshold above which tests will be marked as failing.
  /// Ranges from 0 to 1, both inclusive.
  final double threshold;

  /// Copy of [LocalFileComparator]'s [compare] method, except for the fact that
  /// it checks if the [ComparisonResult.diffPercent] is not greater than
  /// [threshold] to decide whether this test is successful or a failure.
  @override
  Future<bool> compare(Uint8List imageBytes, Uri golden) async {
    final result = await GoldenFileComparator.compareLists(imageBytes, await getGoldenBytes(golden));

    if (!result.passed && result.diffPercent <= threshold) {
      print(
        'A difference of ${result.diffPercent * 100}% was found, but it is '
        'acceptable since it is not greater than the threshold of '
        '${threshold * 100}%',
      );

      return true;
    }

    if (!result.passed) {
      final error = await generateFailureOutput(result, golden, basedir);
      throw FlutterError(error);
    }
    return result.passed;
  }
}

void setThresholdGoldenComparator(double threshold) {
  if (goldenFileComparator is LocalFileComparator) {
    final testUrl = (goldenFileComparator as LocalFileComparator).basedir;

    goldenFileComparator = LocalFileComparatorWithThreshold(
      // flutter_test's LocalFileComparator expects the test's URI to be passed
      // as an argument, but it only uses it to parse the baseDir in order to
      // obtain the directory where the golden tests will be placed.
      // As such, we use the default `testUrl`, which is only the `baseDir` and
      // append a generically named `test.dart` so that the `baseDir` is
      // properly extracted.
      Uri.parse('$testUrl/test.dart'),
      threshold,
    );
  } else {
    throw Exception(
      'Expected `goldenFileComparator` to be of type `LocalFileComparator`, '
      'but it is of type `${goldenFileComparator.runtimeType}`',
    );
  }
}

Also getting blocked by this. Can we get an ETA if someone is working on a fix?

We ended up just making a new commit for all the goldens changes and build on top of that - there is no way to know when there is a fix ready via the official channels.

@erf

We ended up just making a new commit for all the goldens changes and build on top of that - there is no way to know when there is a fix ready via the official channels.

That's what we are considering, but it would have been nice to get some actual fix for the issue. Thanks for sharing your approach @erf !

Unfortunately, this is a known drawback of golden testing: these tests are highly sensitive to the environment and are generally only reliable when run on a machine with the exact same configuration (OS version, OS settings, etc) as the one used to produce the original image.

In this case, it appears that macOS 26.4 introduced changes to system-level font rendering. This effectively invalidates golden images produced on macOS 26.3 or older if they contain text, particularly when using custom fonts. Because the Flutter team does not control how the underlying operating system renders fonts, this isn't something we can fix in the framework itself.

If your golden files use custom fonts and the changes do not appear to be a real regression, I'd update your goldens to make the macOS 26.4 output your new baseline.

To mitigate these kinds of breakages in the future, I'd recommend:

Prefer integration tests or widget tests over golden tests where possible. These kinds of tests are less sensitive to their environment.
Avoid text in goldens. Where possible, use golden tests to test things like layouts and colors, instead of typography.
Use the Ahem font. If you do need text in your golden test, use the Ahem font in tests to render characters as squares, which is much more resilient to OS-level rendering shifts.

Thanks for the answer and suggestions.

I was a bit surprised by the statement:

Because the Flutter team does not control how the underlying operating system renders fonts

I haven't done any deep dives into the Flutter engine, but i would assume that for a cross platform rendering engine, even if you use a OS specific rendering library like Core Text, you would still be in control of laying out and rendering text. Similar like to how the Ghostty terminal does this on various platforms.

Do you have any idea what exact system level changes or libraries lead to this?

when using OS specific api to render text to achieve native look, you are also dependent on how that api renders text..

... i would assume that for a cross platform rendering engine, even if you use a OS specific rendering library like Core Text, you would still be in control of laying out and rendering text. Similar like to how the Ghostty terminal does this on various platforms.

Both Ghostty and Flutter uses CoreText to turn font data into bitmaps on macOS. Text rendering is hard. But yes, Flutter does use platform-agnostic logic for things like glyph selection (through Harfbuzz) or line breaking (through Skia Paragraph). Here's a fun read if you're interested in learning more: https://faultlore.com/blah/text-hates-you/

@loic-sharma

Since there is nothing actionable for Flutter to do here, closing.

Please refer to @loic-sharma comment for details and mitigations: #184182 (comment)

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Goldens images changed after upgrade to macOS 26.4 #184182

Steps to reproduce

Expected results

Actual results

Code sample

Screenshots or Video

Logs

Flutter Doctor output

13 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Goldens images changed after upgrade to macOS 26.4 #184182

Description

Steps to reproduce

Expected results

Actual results

Code sample

Screenshots or Video

Logs

Flutter Doctor output

Activity

saurabh-mirajkar commented on Mar 26, 2026

erf commented on Mar 26, 2026

saurabh-mirajkar commented on Mar 27, 2026

timcreatedit commented on Mar 27, 2026

piotrmoskala commented on Mar 27, 2026

erf commented on Mar 27, 2026

13 remaining items

CaleyD commented on Mar 31, 2026

ljubenAtPropertyMe commented on Apr 7, 2026

erf commented on Apr 7, 2026

ljubenAtPropertyMe commented on Apr 7, 2026

loic-sharma commented on Apr 8, 2026

erf commented on Apr 9, 2026

mx1up commented on Apr 9, 2026

loic-sharma commented on Apr 9, 2026

vashworth commented on Apr 17, 2026

github-actions commented on May 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions