Skip to content

Support cross-browser web macrobenchmark runs#188623

Draft
MarlonJD wants to merge 1 commit into
flutter:masterfrom
MarlonJD:web-cross-browser-macrobench-support
Draft

Support cross-browser web macrobenchmark runs#188623
MarlonJD wants to merge 1 commit into
flutter:masterfrom
MarlonJD:web-cross-browser-macrobench-support

Conversation

@MarlonJD

@MarlonJD MarlonJD commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds local cross-browser execution support to the existing web macrobenchmark task:

  • WEB_BENCHMARK_BROWSER=chrome|firefox|safari selects the browser.
  • WEB_BENCHMARKS=bench_a,bench_b optionally limits a run to specific benchmarks.
  • Firefox launches with an isolated temporary profile.
  • Safari launches through safaridriver.
  • Non-Chrome runs skip the Chrome DevTools tracing endpoints while still collecting the app-reported benchmark profile metrics.

This also updates Blink trace parsing to fall back from tdur to dur when newer Chrome trace events do not include thread duration, with a regression test.

Why

Related to #187660 and follow-up measurement work around Skwasm on Safari and Firefox.

The current web macrobenchmark task is Chrome-centered. That makes it hard to compare CanvasKit, normal Skwasm, and single-threaded Skwasm on the browsers where Skwasm policy decisions are most uncertain.

This PR is intentionally measurement infrastructure only. It does not change the default renderer policy and does not enable Skwasm by default on Safari or Firefox.

Relationship to #188614

#188614 is the artifact-proof PR: it lets tests force normal, heavy, or auto Skwasm and verify which Skwasm-family files actually loaded.

This PR is the measurement-helper PR: it lets the existing macrobenchmark runner execute in Chrome, Firefox, or Safari.

The two PRs are complementary but independent. Neither PR is the Safari/WebKit performance solution by itself.

Policy notes

  • Chrome should keep the existing normal Skwasm direction; the local macrobenchmarks still show clear Skwasm wins.
  • Firefox does not currently justify forcing single-threaded Skwasm as default. ST remains diagnostic only.
  • Safari remains the main open problem. The next Safari work should separate artifact policy (normal/heavy/auto) from presentation strategy, especially retained/tiled or small-dirty presentation.

Measurement notes

Chrome keeps the existing DevTools tracing path and can still report totalUiFrame.average.

Firefox and Safari do not have that Chrome tracing path here, so their runs collect the benchmark profile metrics reported by the Flutter app, such as drawFrameDuration.average. This is enough for local A/B comparison, but it is not claiming trace parity with Chrome.

Safari requires WebDriver support to be enabled locally, for example with safaridriver --enable.

Verification

  • dart analyze lib/tasks/web_benchmarks.dart lib/framework/browser.dart test/framework/browser_test.dart
  • dart --packages=/private/tmp/flutter-pr-188614/.dart_tool/package_config.json /private/tmp/flutter-pr-188614-pub-cache/hosted/pub.dev/test-1.31.1/bin/test.dart test/framework/browser_test.dart

flutter test through the wrapper was not used for the final local test run because this isolated checkout tried to refresh Flutter tool dependencies from pub.dev. The test above uses the existing workspace package config and does not contact pub.dev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

platform-web Web applications specifically team-web Owned by Web platform team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants