Skip to content

Move integration_test.FlutterDeviceScreenshotTest to the framework slow shard #164398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
matanlurey merged 1 commit into from
Feb 28, 2025
Merged

Move integration_test.FlutterDeviceScreenshotTest to the framework slow shard #164398

matanlurey merged 1 commit into from
Feb 28, 2025

Conversation

@matanlurey
Copy link
Contributor

@matanlurey matanlurey commented Feb 28, 2025 •
edited
Loading

Closes #164177.

Every recent failure is integration_test.FlutterDeviceScreenshotTest, which is run using Gradle, and often takes 5m+. This test is already close to hitting its threshold of 30m consistently:

Image

I am proposing moving this test to Linux framework_tests_slow, it does not have a lot of value to run on every platform, and is indeed, slow.

@gmackall
Copy link
Member

gmackall commented Feb 28, 2025

I wonder why this test takes so long on ci, it looks like it takes 2s for me:

mackall-macbookpro:android mackall$ ./gradlew clean

> Configure project :app
WARNING: The option setting 'android.defaults.buildfeatures.buildconfig=true' is deprecated.
The current default is 'false'.
It will be removed in version 9.0 of the Android Gradle plugin.
To keep using this feature, add the following to your module-level build.gradle files:
    android.buildFeatures.buildConfig = true
or from Android Studio, click: `Refactor` > `Migrate BuildConfig to Gradle Build Files`.

BUILD SUCCESSFUL in 638ms
12 actionable tasks: 5 executed, 7 up-to-date


mackall-macbookpro:android mackall$ ./gradlew :integration_test:testDebugUnitTest --tests dev.flutter.plugins.integration_test.FlutterDeviceScreenshotTest

> Configure project :app
WARNING: The option setting 'android.defaults.buildfeatures.buildconfig=true' is deprecated.
The current default is 'false'.
It will be removed in version 9.0 of the Android Gradle plugin.
To keep using this feature, add the following to your module-level build.gradle files:
    android.buildFeatures.buildConfig = true
or from Android Studio, click: `Refactor` > `Migrate BuildConfig to Gradle Build Files`.
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

BUILD SUCCESSFUL in 2s
20 actionable tasks: 15 executed, 5 up-to-date
mackall-macbookpro:android mackall$

But lgtm regardless

@matanlurey
Copy link
Contributor Author

matanlurey commented Feb 28, 2025

It's Windows specifically that is slow 😞

dkwingsmt
dkwingsmt approved these changes Feb 28, 2025
View reviewed changes
Copy link
Contributor

@dkwingsmt dkwingsmt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RSLGTM

@matanlurey matanlurey added emergency Jump the queue; land PR in front of all others; only use for emergencies autosubmit Merge PR when tree becomes green via auto submit App labels Feb 28, 2025
@matanlurey matanlurey added this pull request to the merge queue Feb 28, 2025
Merged via the queue into flutter:master with commit 600ee30 Feb 28, 2025
148 of 149 checks passed
@matanlurey matanlurey deleted the move-FlutterDeviceScreenshotTest-to-runSlow branch February 28, 2025 22:24
@flutter-dashboard flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Feb 28, 2025
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Mar 1, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 1, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
9acf016 
…amework slow shard (flutter/flutter#164398)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Mar 1, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 1, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
1b58552 
…amework slow shard (flutter/flutter#164398)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Mar 2, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 2, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
246d4e8 
…amework slow shard (flutter/flutter#164398)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Mar 2, 2025
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 2, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
10afa80 
…amework slow shard (flutter/flutter#164398)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request Mar 2, 2025
Merged
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 2, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
8e2bada 
…amework slow shard (flutter/flutter#164398)
auto-submit bot pushed a commit to flutter/packages that referenced this pull request Mar 2, 2025
@engine-flutter-autoroll
Roll Flutter from 2e570ca to 842db35 (59 revisions) (#8771)
9e4684e 
Roll Flutter from 2e570ca to 842db35 (59 revisions)

flutter/flutter@2e570ca...842db35

2025-03-02 [email protected] Roll Skia from ad64415050aa to 101eee8fce59 (1 revision) (flutter/flutter#164449)
2025-03-02 [email protected] Roll Fuchsia Linux SDK from ln3joxJfRN2XGhvCv... to AO1KirSDI7-MVYNPN... (flutter/flutter#164440)
2025-03-02 [email protected] android: Clean up gen_snapshot artifact build (flutter/flutter#164418)
2025-03-02 [email protected] Start using `bin/cache/engine.{stamp|realm}` instead of `bin/internal/engine.{realm|version}`. (flutter/flutter#164352)
2025-03-01 [email protected] Add macos/android_debug_unopt to local_engine.json (flutter/flutter#164410)
2025-03-01 [email protected] Delete unused build archive targets (flutter/flutter#164414)
2025-03-01 [email protected] Roll Fuchsia Linux SDK from QMun2itYrV_zUYrvW... to ln3joxJfRN2XGhvCv... (flutter/flutter#164423)
2025-03-01 [email protected] Roll Skia from ac14158663ea to ad64415050aa (1 revision) (flutter/flutter#164413)
2025-03-01 [email protected] Update linux_host_engine.json ci/host_release description (flutter/flutter#164402)
2025-03-01 [email protected] In update_engine_version_test.dart, do not populate the test environment with the host platform environment (flutter/flutter#164395)
2025-03-01 [email protected] Roll-forward #164317: Use `bin/cache/engine.stamp` (flutter/flutter#164401)
2025-02-28 [email protected] Make pressing and moving on CupertinoButton closer to native behavior. (flutter/flutter#161731)
2025-02-28 [email protected] Roll Skia from 4005ba3ca7b6 to ac14158663ea (7 revisions) (flutter/flutter#164404)
2025-02-28 [email protected] [macOS] Prepare FlutterKeyboardManager for multi-view (flutter/flutter#163962)
2025-02-28 [email protected] Add PlatformDispatcher.engineId (flutter/flutter#163476)
2025-02-28 [email protected] Move `integration_test.FlutterDeviceScreenshotTest` to the framework slow shard (flutter/flutter#164398)
2025-02-28 [email protected] Fix: Update DelegatedTransition animation parameter correctly (flutter/flutter#163853)
2025-02-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Write an identical value to `bin/cache/engine.stamp` to prepare for migration (#164317)" (flutter/flutter#164396)
2025-02-28 [email protected] Add action for configuring default action of EditableText.onTapUpOutside (flutter/flutter#162575)
2025-02-28 [email protected] Align jvmTarget usages across codebase, while editing build.gradle files align them with android version documentation (flutter/flutter#164200)
2025-02-28 [email protected] Roll Packages from 01d3d5c to 70b41e1 (5 revisions) (flutter/flutter#164380)
2025-02-28 [email protected] [tool] Allow using archiveName in android bundle build (flutter/flutter#162390)
2025-02-28 [email protected] Fix incorrectly checking for invalid environment variables in the tool (flutter/flutter#164101)
2025-02-28 [email protected] Support forward and back buttons (flutter/flutter#164356)
2025-02-28 [email protected] Fix window creation callback for multi-window (flutter/flutter#164353)
2025-02-28 [email protected] Fix flutter doctor usage of eglinfo in failure cases. (flutter/flutter#164334)
2025-02-28 [email protected]  Refactor writing of package config in tests (flutter/flutter#163734)
2025-02-28 [email protected] Fixed the issue that Slider's secondaryTrackValue is not updated. (flutter/flutter#163996)
2025-02-28 [email protected] Roll pub packages (flutter/flutter#164357)
2025-02-28 [email protected] Remove `Mac mac_unopt` presubmit retry count (flutter/flutter#164350)
2025-02-28 [email protected] Roll Fuchsia Linux SDK from 1elkOxihZuTEiTXzY... to QMun2itYrV_zUYrvW... (flutter/flutter#164351)
2025-02-28 [email protected] Drag handles only need to be tested on mobile platforms. (flutter/flutter#163723)
2025-02-28 [email protected] [Android] Use java for looking up Android API level. (flutter/flutter#163558)
2025-02-28 [email protected] Write an identical value to `bin/cache/engine.stamp` to prepare for migration (flutter/flutter#164317)
2025-02-27 [email protected] Wires up expanded state in web engine (flutter/flutter#164048)
2025-02-27 [email protected] Remove Cheserton's File (flutter/flutter#164340)
2025-02-27 [email protected] Replace update semantics handler with signal. (flutter/flutter#163583)
2025-02-27 [email protected] Split up the conical gradient fragment shader (flutter/flutter#164058)
2025-02-27 [email protected] Roll Skia from c16b145749d4 to 4005ba3ca7b6 (3 revisions) (flutter/flutter#164339)
2025-02-27 [email protected] Delete and update stale documentation regarding engine/engine hash. (flutter/flutter#164324)
2025-02-27 [email protected] Document how `engine.version` (is/will be) computed (flutter/flutter#164335)
2025-02-27 [email protected] Update conductor to write engine.version file (flutter/flutter#163350)
2025-02-27 [email protected] remove last usages of min/compile/target SdkVersion, align sourceCompatibility across repo and update android version documentation (flutter/flutter#164198)
2025-02-27 [email protected] Update links to the `flutter/engine` repository for the monorepo. (flutter/flutter#164328)
2025-02-27 [email protected] Add empty `io.flutter.app.FlutterApplication` to give deprecation notice, and un-break projects that have not migrated (flutter/flutter#164233)
2025-02-27 [email protected] Revert dart sdks that were causing dartaotruntime issues in g3  (flutter/flutter#164307)
...
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
4933591 
…amework slow shard (flutter/flutter#164398)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
24bcc0a 
…amework slow shard (flutter/flutter#164398)
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
@engine-flutter-autoroll
600ee30 Move integration_test.FlutterDeviceScreenshotTest to the fr…
26881b6 
…amework slow shard (flutter/flutter#164398)
androidseb pushed a commit to androidseb/packages that referenced this pull request Jun 8, 2025
@engine-flutter-autoroll @androidseb
Roll Flutter from 2e570ca to 842db35 (59 revisions) (flutter#8771)
2960ae1 
Roll Flutter from 2e570ca to 842db35 (59 revisions)

flutter/flutter@2e570ca...842db35

2025-03-02 [email protected] Roll Skia from ad64415050aa to 101eee8fce59 (1 revision) (flutter/flutter#164449)
2025-03-02 [email protected] Roll Fuchsia Linux SDK from ln3joxJfRN2XGhvCv... to AO1KirSDI7-MVYNPN... (flutter/flutter#164440)
2025-03-02 [email protected] android: Clean up gen_snapshot artifact build (flutter/flutter#164418)
2025-03-02 [email protected] Start using `bin/cache/engine.{stamp|realm}` instead of `bin/internal/engine.{realm|version}`. (flutter/flutter#164352)
2025-03-01 [email protected] Add macos/android_debug_unopt to local_engine.json (flutter/flutter#164410)
2025-03-01 [email protected] Delete unused build archive targets (flutter/flutter#164414)
2025-03-01 [email protected] Roll Fuchsia Linux SDK from QMun2itYrV_zUYrvW... to ln3joxJfRN2XGhvCv... (flutter/flutter#164423)
2025-03-01 [email protected] Roll Skia from ac14158663ea to ad64415050aa (1 revision) (flutter/flutter#164413)
2025-03-01 [email protected] Update linux_host_engine.json ci/host_release description (flutter/flutter#164402)
2025-03-01 [email protected] In update_engine_version_test.dart, do not populate the test environment with the host platform environment (flutter/flutter#164395)
2025-03-01 [email protected] Roll-forward #164317: Use `bin/cache/engine.stamp` (flutter/flutter#164401)
2025-02-28 [email protected] Make pressing and moving on CupertinoButton closer to native behavior. (flutter/flutter#161731)
2025-02-28 [email protected] Roll Skia from 4005ba3ca7b6 to ac14158663ea (7 revisions) (flutter/flutter#164404)
2025-02-28 [email protected] [macOS] Prepare FlutterKeyboardManager for multi-view (flutter/flutter#163962)
2025-02-28 [email protected] Add PlatformDispatcher.engineId (flutter/flutter#163476)
2025-02-28 [email protected] Move `integration_test.FlutterDeviceScreenshotTest` to the framework slow shard (flutter/flutter#164398)
2025-02-28 [email protected] Fix: Update DelegatedTransition animation parameter correctly (flutter/flutter#163853)
2025-02-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Write an identical value to `bin/cache/engine.stamp` to prepare for migration (#164317)" (flutter/flutter#164396)
2025-02-28 [email protected] Add action for configuring default action of EditableText.onTapUpOutside (flutter/flutter#162575)
2025-02-28 [email protected] Align jvmTarget usages across codebase, while editing build.gradle files align them with android version documentation (flutter/flutter#164200)
2025-02-28 [email protected] Roll Packages from 01d3d5c to 70b41e1 (5 revisions) (flutter/flutter#164380)
2025-02-28 [email protected] [tool] Allow using archiveName in android bundle build (flutter/flutter#162390)
2025-02-28 [email protected] Fix incorrectly checking for invalid environment variables in the tool (flutter/flutter#164101)
2025-02-28 [email protected] Support forward and back buttons (flutter/flutter#164356)
2025-02-28 [email protected] Fix window creation callback for multi-window (flutter/flutter#164353)
2025-02-28 [email protected] Fix flutter doctor usage of eglinfo in failure cases. (flutter/flutter#164334)
2025-02-28 [email protected]  Refactor writing of package config in tests (flutter/flutter#163734)
2025-02-28 [email protected] Fixed the issue that Slider's secondaryTrackValue is not updated. (flutter/flutter#163996)
2025-02-28 [email protected] Roll pub packages (flutter/flutter#164357)
2025-02-28 [email protected] Remove `Mac mac_unopt` presubmit retry count (flutter/flutter#164350)
2025-02-28 [email protected] Roll Fuchsia Linux SDK from 1elkOxihZuTEiTXzY... to QMun2itYrV_zUYrvW... (flutter/flutter#164351)
2025-02-28 [email protected] Drag handles only need to be tested on mobile platforms. (flutter/flutter#163723)
2025-02-28 [email protected] [Android] Use java for looking up Android API level. (flutter/flutter#163558)
2025-02-28 [email protected] Write an identical value to `bin/cache/engine.stamp` to prepare for migration (flutter/flutter#164317)
2025-02-27 [email protected] Wires up expanded state in web engine (flutter/flutter#164048)
2025-02-27 [email protected] Remove Cheserton's File (flutter/flutter#164340)
2025-02-27 [email protected] Replace update semantics handler with signal. (flutter/flutter#163583)
2025-02-27 [email protected] Split up the conical gradient fragment shader (flutter/flutter#164058)
2025-02-27 [email protected] Roll Skia from c16b145749d4 to 4005ba3ca7b6 (3 revisions) (flutter/flutter#164339)
2025-02-27 [email protected] Delete and update stale documentation regarding engine/engine hash. (flutter/flutter#164324)
2025-02-27 [email protected] Document how `engine.version` (is/will be) computed (flutter/flutter#164335)
2025-02-27 [email protected] Update conductor to write engine.version file (flutter/flutter#163350)
2025-02-27 [email protected] remove last usages of min/compile/target SdkVersion, align sourceCompatibility across repo and update android version documentation (flutter/flutter#164198)
2025-02-27 [email protected] Update links to the `flutter/engine` repository for the monorepo. (flutter/flutter#164328)
2025-02-27 [email protected] Add empty `io.flutter.app.FlutterApplication` to give deprecation notice, and un-break projects that have not migrated (flutter/flutter#164233)
2025-02-27 [email protected] Revert dart sdks that were causing dartaotruntime issues in g3  (flutter/flutter#164307)
...
FMorschel pushed a commit to FMorschel/packages that referenced this pull request Jun 9, 2025
@engine-flutter-autoroll @FMorschel
Roll Flutter from 2e570ca to 842db35 (59 revisions) (flutter#8771)
427e0c5 
Roll Flutter from 2e570ca to 842db35 (59 revisions)

flutter/flutter@2e570ca...842db35

2025-03-02 [email protected] Roll Skia from ad64415050aa to 101eee8fce59 (1 revision) (flutter/flutter#164449)
2025-03-02 [email protected] Roll Fuchsia Linux SDK from ln3joxJfRN2XGhvCv... to AO1KirSDI7-MVYNPN... (flutter/flutter#164440)
2025-03-02 [email protected] android: Clean up gen_snapshot artifact build (flutter/flutter#164418)
2025-03-02 [email protected] Start using `bin/cache/engine.{stamp|realm}` instead of `bin/internal/engine.{realm|version}`. (flutter/flutter#164352)
2025-03-01 [email protected] Add macos/android_debug_unopt to local_engine.json (flutter/flutter#164410)
2025-03-01 [email protected] Delete unused build archive targets (flutter/flutter#164414)
2025-03-01 [email protected] Roll Fuchsia Linux SDK from QMun2itYrV_zUYrvW... to ln3joxJfRN2XGhvCv... (flutter/flutter#164423)
2025-03-01 [email protected] Roll Skia from ac14158663ea to ad64415050aa (1 revision) (flutter/flutter#164413)
2025-03-01 [email protected] Update linux_host_engine.json ci/host_release description (flutter/flutter#164402)
2025-03-01 [email protected] In update_engine_version_test.dart, do not populate the test environment with the host platform environment (flutter/flutter#164395)
2025-03-01 [email protected] Roll-forward #164317: Use `bin/cache/engine.stamp` (flutter/flutter#164401)
2025-02-28 [email protected] Make pressing and moving on CupertinoButton closer to native behavior. (flutter/flutter#161731)
2025-02-28 [email protected] Roll Skia from 4005ba3ca7b6 to ac14158663ea (7 revisions) (flutter/flutter#164404)
2025-02-28 [email protected] [macOS] Prepare FlutterKeyboardManager for multi-view (flutter/flutter#163962)
2025-02-28 [email protected] Add PlatformDispatcher.engineId (flutter/flutter#163476)
2025-02-28 [email protected] Move `integration_test.FlutterDeviceScreenshotTest` to the framework slow shard (flutter/flutter#164398)
2025-02-28 [email protected] Fix: Update DelegatedTransition animation parameter correctly (flutter/flutter#163853)
2025-02-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Write an identical value to `bin/cache/engine.stamp` to prepare for migration (#164317)" (flutter/flutter#164396)
2025-02-28 [email protected] Add action for configuring default action of EditableText.onTapUpOutside (flutter/flutter#162575)
2025-02-28 [email protected] Align jvmTarget usages across codebase, while editing build.gradle files align them with android version documentation (flutter/flutter#164200)
2025-02-28 [email protected] Roll Packages from 01d3d5c to 70b41e1 (5 revisions) (flutter/flutter#164380)
2025-02-28 [email protected] [tool] Allow using archiveName in android bundle build (flutter/flutter#162390)
2025-02-28 [email protected] Fix incorrectly checking for invalid environment variables in the tool (flutter/flutter#164101)
2025-02-28 [email protected] Support forward and back buttons (flutter/flutter#164356)
2025-02-28 [email protected] Fix window creation callback for multi-window (flutter/flutter#164353)
2025-02-28 [email protected] Fix flutter doctor usage of eglinfo in failure cases. (flutter/flutter#164334)
2025-02-28 [email protected]  Refactor writing of package config in tests (flutter/flutter#163734)
2025-02-28 [email protected] Fixed the issue that Slider's secondaryTrackValue is not updated. (flutter/flutter#163996)
2025-02-28 [email protected] Roll pub packages (flutter/flutter#164357)
2025-02-28 [email protected] Remove `Mac mac_unopt` presubmit retry count (flutter/flutter#164350)
2025-02-28 [email protected] Roll Fuchsia Linux SDK from 1elkOxihZuTEiTXzY... to QMun2itYrV_zUYrvW... (flutter/flutter#164351)
2025-02-28 [email protected] Drag handles only need to be tested on mobile platforms. (flutter/flutter#163723)
2025-02-28 [email protected] [Android] Use java for looking up Android API level. (flutter/flutter#163558)
2025-02-28 [email protected] Write an identical value to `bin/cache/engine.stamp` to prepare for migration (flutter/flutter#164317)
2025-02-27 [email protected] Wires up expanded state in web engine (flutter/flutter#164048)
2025-02-27 [email protected] Remove Cheserton's File (flutter/flutter#164340)
2025-02-27 [email protected] Replace update semantics handler with signal. (flutter/flutter#163583)
2025-02-27 [email protected] Split up the conical gradient fragment shader (flutter/flutter#164058)
2025-02-27 [email protected] Roll Skia from c16b145749d4 to 4005ba3ca7b6 (3 revisions) (flutter/flutter#164339)
2025-02-27 [email protected] Delete and update stale documentation regarding engine/engine hash. (flutter/flutter#164324)
2025-02-27 [email protected] Document how `engine.version` (is/will be) computed (flutter/flutter#164335)
2025-02-27 [email protected] Update conductor to write engine.version file (flutter/flutter#163350)
2025-02-27 [email protected] remove last usages of min/compile/target SdkVersion, align sourceCompatibility across repo and update android version documentation (flutter/flutter#164198)
2025-02-27 [email protected] Update links to the `flutter/engine` repository for the monorepo. (flutter/flutter#164328)
2025-02-27 [email protected] Add empty `io.flutter.app.FlutterApplication` to give deprecation notice, and un-break projects that have not migrated (flutter/flutter#164233)
2025-02-27 [email protected] Revert dart sdks that were causing dartaotruntime issues in g3  (flutter/flutter#164307)
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dkwingsmt dkwingsmt dkwingsmt approved these changes

@gmackall gmackall gmackall approved these changes

@Piinks Piinks Awaiting requested review from Piinks

Assignees

No one assigned

Labels

emergency Jump the queue; land PR in front of all others; only use for emergencies

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Windows framework_tests_misc is 2.17% flaky

3 participants

@matanlurey @gmackall @dkwingsmt