Support cross-browser web macrobenchmark runs#188623
Draft
MarlonJD wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds local cross-browser execution support to the existing web macrobenchmark task:
WEB_BENCHMARK_BROWSER=chrome|firefox|safariselects the browser.WEB_BENCHMARKS=bench_a,bench_boptionally limits a run to specific benchmarks.safaridriver.This also updates Blink trace parsing to fall back from
tdurtodurwhen newer Chrome trace events do not include thread duration, with a regression test.Why
Related to #187660 and follow-up measurement work around Skwasm on Safari and Firefox.
The current web macrobenchmark task is Chrome-centered. That makes it hard to compare CanvasKit, normal Skwasm, and single-threaded Skwasm on the browsers where Skwasm policy decisions are most uncertain.
This PR is intentionally measurement infrastructure only. It does not change the default renderer policy and does not enable Skwasm by default on Safari or Firefox.
Relationship to #188614
#188614 is the artifact-proof PR: it lets tests force
normal,heavy, orautoSkwasm and verify which Skwasm-family files actually loaded.This PR is the measurement-helper PR: it lets the existing macrobenchmark runner execute in Chrome, Firefox, or Safari.
The two PRs are complementary but independent. Neither PR is the Safari/WebKit performance solution by itself.
Policy notes
normal/heavy/auto) from presentation strategy, especially retained/tiled or small-dirty presentation.Measurement notes
Chrome keeps the existing DevTools tracing path and can still report
totalUiFrame.average.Firefox and Safari do not have that Chrome tracing path here, so their runs collect the benchmark profile metrics reported by the Flutter app, such as
drawFrameDuration.average. This is enough for local A/B comparison, but it is not claiming trace parity with Chrome.Safari requires WebDriver support to be enabled locally, for example with
safaridriver --enable.Verification
dart analyze lib/tasks/web_benchmarks.dart lib/framework/browser.dart test/framework/browser_test.dartdart --packages=/private/tmp/flutter-pr-188614/.dart_tool/package_config.json /private/tmp/flutter-pr-188614-pub-cache/hosted/pub.dev/test-1.31.1/bin/test.dart test/framework/browser_test.dartflutter testthrough the wrapper was not used for the final local test run because this isolated checkout tried to refresh Flutter tool dependencies from pub.dev. The test above uses the existing workspace package config and does not contact pub.dev.