Fix performance problems of flutter test --experimental-faster-testing

The performance of flutter test --experimental-faster-testing is currently not ideal when running a large number of test files because the Lightweight Engines spawned when using this strategy get blocked from shutting down by open ReceivePorts. This was my latest attempt to ensure that all channels, and thus all ReceivePorts, get closed on the spawned isolates: d560cf4, but this commit did not fix the problem. So, we either have to determine what other changes need to be made in order to ensure that all ReceivePorts get closed on spawned isolates, or alternatively we need to add a new Engine API that can forcibly kill spawned Lightweight Engines.

@a-siva

cc @a-siva

@jonahwilliams

//cc @jonahwilliams would you be the right person to help out Derek here ?

@derekxu16

@derekxu16 are these package:test send/receive ports? if so have you talked to any of the package:test maintainers yet? I can look at it a bit but I'm not very familiar with the work.

There was some discussion with the package:test maintainers here: https://chat.google.com/room/AAAA3-KvHNw/_Ng7rX3hbHk. Nate's last message indicated that package:test currently forcibly kills spawned isolates when they are done running test code, and he doesn't know whether or not the spawned isolates would shut down by themselves if the force-killing logic was removed. When I get the chance, I can check what happens if the force-killing logic is removed from package:test.

and he doesn't know whether or not the spawned isolates would shut down by themselves if the force-killing logic was removed.

They do in the typical case.

After some more thinking, though, I got worried about removing the kill() call. I think it might be in place as a resource optimization. Nothing breaks when I remove it - but if some test patterns are commonly holding resources which keep test isolates running don't know if it could become a noticeable performance issue.

dart-lang/test#2198 (comment)

The culprit of slow performance seems to be the fact that all dart code for all tests in one combined dill file for all concurrently-running dart isolates spawned in flutter_tester runs on a single one and the same platform thread. So effectively we end up running all the tests on a single thread, not taking advantage of possible paralellization.

This is officially dead?

Are plans to stabilize this feature / fix the single-threaded limitation?

Our mobile app has over 2,000 test cases - widget tests, Blocs, accessibility, etc. In other languages it wouldn't really be some crazy amount, but in Flutter they already take almost 3 minutes to run. And that value is quickly growing as we frequently add new features.

Since we run these tests not only in the CI pipeline, but also at the end of each agentic coding session (as part of the LLM verifying if the task was completed successfully - which is done dozens of times a day) it's starting to accumulate and sum up to a very substantial share of extra developer downtime in a week / month.

Improving the paralellization of the testing tools should have always been a priority for the Flutter team (which is explained well in one of my all-time favorite DEV blogposts - the code quality pyramid), but in 2026 this problem just became more pressing than ever.

I've ended-up writing this little dart script to improve the test speed using the same technique that is used by the VGV CLI

#!/usr/bin/env dart

/// Generates a merged test file that imports all test files into a single entrypoint.
/// This replicates the optimization from `very_good test` which significantly speeds up
/// test execution by avoiding VM startup overhead per test file.
///
/// Usage: dart scripts/generate_merged_tests.dart
/// Output: test/.test_optimizer.dart

import 'dart:io';

void main() {
  final testDir = Directory('test');
  if (!testDir.existsSync()) {
    stderr.writeln('Error: test directory not found');
    exit(1);
  }

  final testFiles = testDir
      .listSync(recursive: true)
      .whereType<File>()
      .where((f) => f.path.endsWith('_test.dart'))
      .where((f) => !f.path.contains('.test_optimizer.dart'))
      .map((f) => f.path.replaceFirst('test/', ''))
      .toList()
    ..sort();

  final buffer = StringBuffer();
  buffer.writeln('// GENERATED FILE - DO NOT EDIT');
  buffer.writeln('// Generated by scripts/generate_merged_tests.dart');
  buffer
      .writeln('// This file aggregates all tests into a single entrypoint for faster execution.');
  buffer.writeln();
  buffer.writeln("import 'package:flutter_test/flutter_test.dart';");
  buffer.writeln();

  for (var i = 0; i < testFiles.length; i++) {
    buffer.writeln("import '${testFiles[i]}' as test_$i;");
  }

  buffer.writeln();
  buffer.writeln('void main() {');

  for (var i = 0; i < testFiles.length; i++) {
    final testName = testFiles[i].replaceAll('/', ' > ').replaceAll('_test.dart', '');
    buffer.writeln("  group('$testName', test_$i.main);");
  }

  buffer.writeln('}');

  final outputFile = File('test/.test_optimizer.dart');
  outputFile.writeAsStringSync(buffer.toString());

  stdout.writeln('Generated ${outputFile.path} with ${testFiles.length} test files');
}

and then run it with a make script like this:

.PHONY: test
test:
	@dart scripts/generate_merged_tests.dart
	flutter test test/.test_optimizer.dart --concurrency=$$(sysctl -n hw.ncpu) --no-pub --test-randomize-ordering-seed=random --no-track-widget-creation --fail-fast

Of course there is no reasons why this is not in the framework itself besides the fact that the team has zero effective capacity to take anything. Hopefully this helps as a workaround.

[flutter_tools] Shut down isolates in --experimental-faster-testing w…

[flutter_tools, flutter_test] Deprecate --experimental-faster-testing (…

Navigation Menu

Repository navigation

Fix performance problems of `flutter test --experimental-faster-testing` #146603

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fix performance problems of flutter test --experimental-faster-testing #146603

Description

Activity

derekxu16 commented on Apr 10, 2024

a-siva commented on Apr 10, 2024

jonahwilliams commented on Apr 10, 2024

derekxu16 commented on Apr 11, 2024

natebosch commented on May 2, 2024

aam commented on May 8, 2024

nerder commented on Dec 12, 2025

nielsenko commented on Feb 26, 2026

polczak-itt commented on Feb 26, 2026

nerder commented on Feb 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Fix performance problems of `flutter test --experimental-faster-testing` #146603