-
Notifications
You must be signed in to change notification settings - Fork 29.7k
[web] Reuse chrome instance to run all flutter tests #174957
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[web] Reuse chrome instance to run all flutter tests #174957
Conversation
|
It looks like this pull request may not have tests. Please make sure to add tests or get an explicit test exemption before merging. If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix? Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. If you believe this PR qualifies for a test exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request aims to speed up web tests by reusing the same Chrome instance across multiple test files. This is achieved by only launching a new browser instance if one isn't already running, and by not shutting down the browser after each test suite completes. My review identifies a potential issue where the closeEphemeral method could close the browser without properly cleaning up state, which might cause subsequent tests to fail. This is a side-effect of the changes made to persist the browser instance.
eyebrowsoffire
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Roll Flutter from 6b18740d5a23 to 87d5b753196c (88 revisions) flutter/flutter@6b18740...87d5b75 2025-09-05 [email protected] [ Device Lab ] Add regression testing for flutter/flutter#174952 (flutter/flutter#174956) 2025-09-05 [email protected] Roll Skia from 0ca53adfc6cc to 845ec125e94c (2 revisions) (flutter/flutter#174978) 2025-09-05 [email protected] [a11y-app] Fix NavigationRail leading and trailing labels (flutter/flutter#174861) 2025-09-05 [email protected] Roll Skia from d7e99be07d5d to 0ca53adfc6cc (5 revisions) (flutter/flutter#174972) 2025-09-05 [email protected] Roll Fuchsia Linux SDK from izfNA3o_2zL4Cjqmy... to xG_uERsxHvUwFHpF2... (flutter/flutter#174970) 2025-09-04 [email protected] Fix IconButton.color overrided by IconButtomTheme (flutter/flutter#174515) 2025-09-04 [email protected] [web] Reuse chrome instance to run all flutter tests (flutter/flutter#174957) 2025-09-04 [email protected] fix(Semantics): Ensure semantics properties take priority over button's (flutter/flutter#174473) 2025-09-04 [email protected] Make every LLDB Init error message actionable (flutter/flutter#174726) 2025-09-04 [email protected] Fix table cell semantics rect alignment issues. (flutter/flutter#174914) 2025-09-04 [email protected] Fix: Use route navigator for CupertinoSheetRoute pop (flutter/flutter#173103) 2025-09-04 [email protected] [ Widget Preview] Add `group` property to `Preview` (flutter/flutter#174849) 2025-09-04 [email protected] Allow OverlayPortal.overlayChildLayoutBuilder to choose root Overlay (flutter/flutter#174239) 2025-09-04 [email protected] Roll Skia from 7c2f502e3304 to d7e99be07d5d (18 revisions) (flutter/flutter#174936) 2025-09-04 [email protected] Remove 'terms of use' wording from web_unicode (flutter/flutter#174939) 2025-09-04 [email protected] [ Tool ] Remove leftover Android x86 deprecation warning constant (flutter/flutter#174941) 2025-09-04 [email protected] Prevent potential crash when accessing window in FlutterSceneDelegate (flutter/flutter#174873) 2025-09-04 [email protected] Roll Packages from 42bb347 to 98580c6 (5 revisions) (flutter/flutter#174943) 2025-09-04 [email protected] [ Device Lab ] Fix wakefulness check to only match log entries with string values (flutter/flutter#174953) 2025-09-04 [email protected] Fix expanded DropdownMenu panel is shorter than text field (flutter/flutter#174443) 2025-09-04 [email protected] Add a `viewController` property to the ios/macOS `FlutterPluginRegistrar` protocol (flutter/flutter#174168) 2025-09-03 [email protected] Roll Fuchsia Linux SDK from J3T_wZqL_57mRfWky... to izfNA3o_2zL4Cjqmy... (flutter/flutter#174908) 2025-09-03 [email protected] Wires up Android API to set section locale (flutter/flutter#173364) 2025-09-03 [email protected] Delete impeller::SPrintF. (flutter/flutter#174900) 2025-09-03 [email protected] Make sure that a DropdownMenuFormField doesn't crash in 0x0 environment (flutter/flutter#174777) 2025-09-03 [email protected] Remove unnecessary `presubmit_max_attempts` from .ci.yaml (flutter/flutter#174885) 2025-09-03 [email protected] Use local canvaskit in `dart_data_asset_test.dart` (flutter/flutter#174891) 2025-09-03 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Mark Linux web_canvaskit_tests_7_last as bringup (#174878)" (flutter/flutter#174897) 2025-09-03 [email protected] Correct intrinsics calculation for CupertinoTextField with placeholder (flutter/flutter#174889) 2025-09-03 [email protected] Considers large title height in CupertinoNavigationBar's preferred size (flutter/flutter#173722) 2025-09-03 [email protected] [A11y] Add semantics for CupertinoExpansionTile (flutter/flutter#174480) 2025-09-03 [email protected] Fix LinearProgressIndicator track painting. (flutter/flutter#173108) 2025-09-03 [email protected] update triage documentation to include team-android (flutter/flutter#174850) 2025-09-03 [email protected] Update `test_timeout_secs` to match `timeout` for `Linux web_skwasm_tests_*` and `Linux web_canvaskit_tests_*` (flutter/flutter#174881) 2025-09-03 [email protected] Roll Packages from 5d785a0 to 42bb347 (10 revisions) (flutter/flutter#174876) 2025-09-03 [email protected] Fixup formatting of gn files in the old buildroot. (flutter/flutter#174852) 2025-09-03 [email protected] Roll Dart SDK to 3.10.0-162.1.beta (flutter/flutter#174834) 2025-09-03 [email protected] Mark Linux web_canvaskit_tests_7_last as bringup (flutter/flutter#174878) 2025-09-02 [email protected] [Impeller] Fix overdraw in DrawRect geometry (flutter/flutter#174735) 2025-09-02 [email protected] Patch .clang-format files to specify C++20. (flutter/flutter#174848) 2025-09-02 [email protected] Add data assets (flutter/flutter#174685) 2025-09-02 [email protected] refactors `FlutterPlugin.kt` to use one line statement in the `into` bloc (flutter/flutter#174759) 2025-09-02 [email protected] Ensures initial semantics state is sent to engine (flutter/flutter#174845) 2025-09-02 [email protected] [Android] Break up plugin_test integration tests (flutter/flutter#174728) 2025-09-02 [email protected] Fix: Assertion failure in RawScrollbarState with dynamic controller assignment (flutter/flutter#173156) 2025-09-02 [email protected] Roll Skia from 359f3d7cc7ed to 7c2f502e3304 (1 revision) (flutter/flutter#174844) ...
Web test shards take a significant time to run in CI compared to their VM counterparts. One theory is that launching and shutting down Chrome repeatedly is a major part of the slowness. This PR makes `flutter test --platform=chrome` reuse the same Chrome instance for all test files instead of launching a new one for each test file. At first glance, it may seem like we aren't shutting down Chrome anymore, but we actually do in `closeEphemeral` and `close` methods. Thanks @eyebrowsoffire for the idea!
Web test shards take a significant time to run in CI compared to their VM counterparts. One theory is that launching and shutting down Chrome repeatedly is a major part of the slowness. This PR makes `flutter test --platform=chrome` reuse the same Chrome instance for all test files instead of launching a new one for each test file. At first glance, it may seem like we aren't shutting down Chrome anymore, but we actually do in `closeEphemeral` and `close` methods. Thanks @eyebrowsoffire for the idea!
…r#9962) Roll Flutter from 6b18740d5a23 to 87d5b753196c (88 revisions) flutter/flutter@6b18740...87d5b75 2025-09-05 [email protected] [ Device Lab ] Add regression testing for flutter/flutter#174952 (flutter/flutter#174956) 2025-09-05 [email protected] Roll Skia from 0ca53adfc6cc to 845ec125e94c (2 revisions) (flutter/flutter#174978) 2025-09-05 [email protected] [a11y-app] Fix NavigationRail leading and trailing labels (flutter/flutter#174861) 2025-09-05 [email protected] Roll Skia from d7e99be07d5d to 0ca53adfc6cc (5 revisions) (flutter/flutter#174972) 2025-09-05 [email protected] Roll Fuchsia Linux SDK from izfNA3o_2zL4Cjqmy... to xG_uERsxHvUwFHpF2... (flutter/flutter#174970) 2025-09-04 [email protected] Fix IconButton.color overrided by IconButtomTheme (flutter/flutter#174515) 2025-09-04 [email protected] [web] Reuse chrome instance to run all flutter tests (flutter/flutter#174957) 2025-09-04 [email protected] fix(Semantics): Ensure semantics properties take priority over button's (flutter/flutter#174473) 2025-09-04 [email protected] Make every LLDB Init error message actionable (flutter/flutter#174726) 2025-09-04 [email protected] Fix table cell semantics rect alignment issues. (flutter/flutter#174914) 2025-09-04 [email protected] Fix: Use route navigator for CupertinoSheetRoute pop (flutter/flutter#173103) 2025-09-04 [email protected] [ Widget Preview] Add `group` property to `Preview` (flutter/flutter#174849) 2025-09-04 [email protected] Allow OverlayPortal.overlayChildLayoutBuilder to choose root Overlay (flutter/flutter#174239) 2025-09-04 [email protected] Roll Skia from 7c2f502e3304 to d7e99be07d5d (18 revisions) (flutter/flutter#174936) 2025-09-04 [email protected] Remove 'terms of use' wording from web_unicode (flutter/flutter#174939) 2025-09-04 [email protected] [ Tool ] Remove leftover Android x86 deprecation warning constant (flutter/flutter#174941) 2025-09-04 [email protected] Prevent potential crash when accessing window in FlutterSceneDelegate (flutter/flutter#174873) 2025-09-04 [email protected] Roll Packages from 42bb347 to 98580c6 (5 revisions) (flutter/flutter#174943) 2025-09-04 [email protected] [ Device Lab ] Fix wakefulness check to only match log entries with string values (flutter/flutter#174953) 2025-09-04 [email protected] Fix expanded DropdownMenu panel is shorter than text field (flutter/flutter#174443) 2025-09-04 [email protected] Add a `viewController` property to the ios/macOS `FlutterPluginRegistrar` protocol (flutter/flutter#174168) 2025-09-03 [email protected] Roll Fuchsia Linux SDK from J3T_wZqL_57mRfWky... to izfNA3o_2zL4Cjqmy... (flutter/flutter#174908) 2025-09-03 [email protected] Wires up Android API to set section locale (flutter/flutter#173364) 2025-09-03 [email protected] Delete impeller::SPrintF. (flutter/flutter#174900) 2025-09-03 [email protected] Make sure that a DropdownMenuFormField doesn't crash in 0x0 environment (flutter/flutter#174777) 2025-09-03 [email protected] Remove unnecessary `presubmit_max_attempts` from .ci.yaml (flutter/flutter#174885) 2025-09-03 [email protected] Use local canvaskit in `dart_data_asset_test.dart` (flutter/flutter#174891) 2025-09-03 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Mark Linux web_canvaskit_tests_7_last as bringup (#174878)" (flutter/flutter#174897) 2025-09-03 [email protected] Correct intrinsics calculation for CupertinoTextField with placeholder (flutter/flutter#174889) 2025-09-03 [email protected] Considers large title height in CupertinoNavigationBar's preferred size (flutter/flutter#173722) 2025-09-03 [email protected] [A11y] Add semantics for CupertinoExpansionTile (flutter/flutter#174480) 2025-09-03 [email protected] Fix LinearProgressIndicator track painting. (flutter/flutter#173108) 2025-09-03 [email protected] update triage documentation to include team-android (flutter/flutter#174850) 2025-09-03 [email protected] Update `test_timeout_secs` to match `timeout` for `Linux web_skwasm_tests_*` and `Linux web_canvaskit_tests_*` (flutter/flutter#174881) 2025-09-03 [email protected] Roll Packages from 5d785a0 to 42bb347 (10 revisions) (flutter/flutter#174876) 2025-09-03 [email protected] Fixup formatting of gn files in the old buildroot. (flutter/flutter#174852) 2025-09-03 [email protected] Roll Dart SDK to 3.10.0-162.1.beta (flutter/flutter#174834) 2025-09-03 [email protected] Mark Linux web_canvaskit_tests_7_last as bringup (flutter/flutter#174878) 2025-09-02 [email protected] [Impeller] Fix overdraw in DrawRect geometry (flutter/flutter#174735) 2025-09-02 [email protected] Patch .clang-format files to specify C++20. (flutter/flutter#174848) 2025-09-02 [email protected] Add data assets (flutter/flutter#174685) 2025-09-02 [email protected] refactors `FlutterPlugin.kt` to use one line statement in the `into` bloc (flutter/flutter#174759) 2025-09-02 [email protected] Ensures initial semantics state is sent to engine (flutter/flutter#174845) 2025-09-02 [email protected] [Android] Break up plugin_test integration tests (flutter/flutter#174728) 2025-09-02 [email protected] Fix: Assertion failure in RawScrollbarState with dynamic controller assignment (flutter/flutter#173156) 2025-09-02 [email protected] Roll Skia from 359f3d7cc7ed to 7c2f502e3304 (1 revision) (flutter/flutter#174844) ...
Web test shards take a significant time to run in CI compared to their VM counterparts. One theory is that launching and shutting down Chrome repeatedly is a major part of the slowness. This PR makes `flutter test --platform=chrome` reuse the same Chrome instance for all test files instead of launching a new one for each test file. At first glance, it may seem like we aren't shutting down Chrome anymore, but we actually do in `closeEphemeral` and `close` methods. Thanks @eyebrowsoffire for the idea!
Web test shards take a significant time to run in CI compared to their VM counterparts. One theory is that launching and shutting down Chrome repeatedly is a major part of the slowness.
This PR makes
flutter test --platform=chromereuse the same Chrome instance for all test files instead of launching a new one for each test file.At first glance, it may seem like we aren't shutting down Chrome anymore, but we actually do in
closeEphemeralandclosemethods.Thanks @eyebrowsoffire for the idea!