Skip to content

Conversation

@matanlurey
Copy link
Contributor

@matanlurey matanlurey commented Dec 30, 2024

This PR eliminates (1) scenario_app/android, and all references to an Android scenario_app, including documentation, CI configuration, build rules, test runner, Android-side test application, and (2), to verify and disambiguate the remains, renames the folder ios_scenario_app, which is now accurate.

It also eliminates elements that were only used in the Android-side scenario_app, such as Firebase Test Lab uploading.

I would be open to doing this in phases if we thought it was better to do so, but given its mostly a mechanical change (and by renaming the directory, references can be checked merely by looking for /\bscenario_app/, I believe this is safe to iterate on and eventually merge after the holidays.

Background

As of the merged mono-repo, there is no longer a requirement for the engine to be testable as a standalone unit.

As an example, %ENGINE%/testing/scenario_app, which was intended to emulate the Flutter framework (and some of flutter_tools tooling), load the iOS and Android embedder, and run various "scenarios" (which ran a combination of dart:ui code and Android Java/iOS Obj-C) verifying golden-file screenshots.

Instead, it is now possible to write and run real (full) Flutter apps the same way that an end-user (or our own tests) would. One such example is dev/native_driver_test, which is a full-fledged Flutter app, which uses standard tooling (i.e. flutter), to test most of the same elements that previously were only tested in the Android version of scenario_app.

@github-actions github-actions bot added a: text input Entering text in a text field or keyboard related problems engine flutter/engine related. See also e: labels. f: routes Navigator, Router, and related APIs. d: docs/ flutter/flutter/docs, for contributors labels Dec 30, 2024
@jmagman
Copy link
Member

jmagman commented Dec 30, 2024

Instead, it is now possible to write and run real (full) Flutter apps the same way that an end-user (or our own tests) would.

#160800

Copy link
Member

@zanderso zanderso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing these on Android makes sense if we already have similar test coverage that pre-exists on emulators in the framework tests.

I see this is a draft, but still want to note that removing the entries from the ci.yaml file will cause all the builds/tests in the json file to not run, including the non-scenario-app tests. I suspect this might not be your intention.

@matanlurey
Copy link
Contributor Author

I see this is a draft, but still want to note that removing the entries from the ci.yaml file will cause all the builds/tests in the json file to not run, including the non-scenario-app tests. I suspect this might not be your intention.

Yeah that's not the intent, thanks for the catch. Will clean that up before a formal review.

@matanlurey matanlurey marked this pull request as ready for review January 6, 2025 21:09
@matanlurey matanlurey requested a review from zanderso January 6, 2025 21:09
@zanderso
Copy link
Member

zanderso commented Jan 7, 2025

Is there any diff between the test coverage of scenario_app and native_driver_test? If so, do any new tests need to be added to native_driver_test?

@matanlurey
Copy link
Contributor Author

Is there any diff between the test coverage of scenario_app and native_driver_test? If so, do any new tests need to be added to native_driver_test?

@jonahwilliams and I looked briefly.

There is a different configuration of platform views Jonah would like covered, but everything else is either covered or the scenario_app tests were superfluous (not covering something we find important). Filed #161261 and will do that first.

@matanlurey matanlurey added the blocked Issue is blocked by another issue label Jan 7, 2025
@chinmaygarde
Copy link
Member

Triage: The presub check is hung and it seems like the only way to fix it is to push an empty commit.

@matanlurey
Copy link
Contributor Author

Moving back to draft. When android_engine_test feels good (~1 more week), I'll send this out for final review.

See #161534.

@matanlurey matanlurey marked this pull request as draft January 13, 2025 19:16
@matanlurey matanlurey marked this pull request as ready for review January 27, 2025 19:49
Copy link
Member

@zanderso zanderso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matanlurey matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 27, 2025
@auto-submit
Copy link
Contributor

auto-submit bot commented Jan 27, 2025

autosubmit label was removed for flutter/flutter/160992, because - The status or check suite Mac mac_unopt has failed. Please fix the issues identified (or deflake) before re-applying this label.

@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 27, 2025
@matanlurey matanlurey added autosubmit Merge PR when tree becomes green via auto submit App and removed blocked Issue is blocked by another issue labels Jan 27, 2025
@auto-submit
Copy link
Contributor

auto-submit bot commented Jan 28, 2025

autosubmit label was removed for flutter/flutter/160992, because - The status or check suite Mac_arm64 build_tests_4_4 has failed. Please fix the issues identified (or deflake) before re-applying this label.

@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 28, 2025
@matanlurey matanlurey added this pull request to the merge queue Jan 28, 2025
Merged via the queue into flutter:master with commit 9e273d5 Jan 28, 2025
177 checks passed
@matanlurey matanlurey deleted the remove-android-scenario-app branch January 28, 2025 04:18
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 28, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 28, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 28, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 29, 2025
auto-submit bot pushed a commit to flutter/packages that referenced this pull request Jan 30, 2025
Roll Flutter from c1ffaa9 to b007899 (43 revisions)

flutter/flutter@c1ffaa9...b007899

2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315)
2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309)
2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582)
2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903)
2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880)
2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345)
2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272)
2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336)
2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330)
2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337)
2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332)
2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289)
2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335)
2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327)
2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123)
2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049)
2025-01-28 [email protected] Refactor code inside flutter.groovy  (flutter/flutter#160250)
2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282)
2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911)
2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279)
2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313)
2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992)
2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294)
2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270)
2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170)
2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205)
2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277)
2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271)
2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037)
2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996)
2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423)
2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258)
2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198)
2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254)
2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048)
2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118)
2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518)
2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164)
2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089)
2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171)
2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038)
2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113)
2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
...
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
androidseb pushed a commit to androidseb/packages that referenced this pull request Jun 8, 2025
Roll Flutter from c1ffaa9 to b007899 (43 revisions)

flutter/flutter@c1ffaa9...b007899

2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315)
2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309)
2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582)
2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903)
2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880)
2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345)
2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272)
2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336)
2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330)
2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337)
2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332)
2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289)
2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335)
2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327)
2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123)
2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049)
2025-01-28 [email protected] Refactor code inside flutter.groovy  (flutter/flutter#160250)
2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282)
2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911)
2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279)
2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313)
2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992)
2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294)
2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270)
2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170)
2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205)
2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277)
2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271)
2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037)
2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996)
2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423)
2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258)
2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198)
2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254)
2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048)
2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118)
2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518)
2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164)
2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089)
2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171)
2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038)
2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113)
2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
...
FMorschel pushed a commit to FMorschel/packages that referenced this pull request Jun 9, 2025
Roll Flutter from c1ffaa9 to b007899 (43 revisions)

flutter/flutter@c1ffaa9...b007899

2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315)
2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309)
2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582)
2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903)
2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880)
2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345)
2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272)
2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336)
2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330)
2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337)
2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332)
2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289)
2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335)
2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327)
2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123)
2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049)
2025-01-28 [email protected] Refactor code inside flutter.groovy  (flutter/flutter#160250)
2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282)
2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911)
2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279)
2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313)
2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992)
2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294)
2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270)
2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170)
2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205)
2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277)
2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271)
2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037)
2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996)
2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423)
2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258)
2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198)
2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254)
2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048)
2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118)
2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518)
2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164)
2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089)
2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171)
2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038)
2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113)
2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: text input Entering text in a text field or keyboard related problems d: docs/ flutter/flutter/docs, for contributors engine flutter/engine related. See also e: labels. f: routes Navigator, Router, and related APIs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants