Run iOS scenario apps on iPhone 11 and iOS 14 #30104

jmagman · 2021-12-03T07:05:35Z

To facilitate upgrading to Xcode 13, run the tests on an iPhone 11 simulator, which is available on both Xcode 12 and 13.

Succeeded: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8828886065297463201/+/u/test:_Scenario_App_Integration_Tests/stdout

Part of flutter/flutter#85555.

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
I listed at least one issue that this PR fixes in the description above.
I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on
writing and running engine tests.
I updated/added relevant documentation (doc comments with ///).
I signed the CLA.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

jmagman · 2021-12-03T07:09:37Z

testing/scenario_app/ios/Scenarios/ScenariosUITests/GoldenImage.m

-  CGImageRef imageRefA = [self.image CGImage];
-  CGImageRef imageRefB = [image CGImage];
+
+  // The home bar at the bottom of the screen can be different shades of gray


I was hoping to use prefersHomeIndicatorAutoHidden similar to

engine/testing/scenario_app/ios/Scenarios/Scenarios/AppDelegate.m

Line 17 in 9e49ffb

- (BOOL)prefersStatusBarHidden {

But that makes it worse, the home indicator bar animates in and then animates out.

This is ok to me.

Side node:
Is it possible to wait for the home indicator to be hidden with a XCTestExpectation?

Is it possible to wait for the home indicator to be hidden with a XCTestExpectation?

Good idea! I checked springboard and there's a "Home Grabber" element.

jmagman · 2021-12-03T07:10:15Z

testing/scenario_app/ios/Scenarios/ScenariosUITests/iPadGestureTests.m

                    @"PointerChange.hover event did not occur for a hover");
  NSString* removeMessage = [NSString
-      stringWithFormat:@"%d,PointerChange.remove,device=0,buttons=0", lastHoverSequenceNumber + 1];
+      stringWithFormat:@"%ld,PointerChange.remove,device=0,buttons=0", lastHoverSequenceNumber + 1];


Fix a warning.

cyanglaz

LGTM

cyanglaz · 2021-12-03T17:19:37Z

testing/scenario_app/ios/Scenarios/ScenariosUITests/GoldenImage.m

-  CGImageRef imageRefA = [self.image CGImage];
-  CGImageRef imageRefB = [image CGImage];
+
+  // The home bar at the bottom of the screen can be different shades of gray


This is ok to me.

Side node:
Is it possible to wait for the home indicator to be hidden with a XCTestExpectation?

jmagman · 2021-12-04T02:58:12Z

testing/scenario_app/ios/Scenarios/ScenariosUITests/SpawnEngineTest.m

+  XCUIElement* addTextField = application.textFields[@"ready"];
+  XCTAssertTrue([addTextField waitForExistenceWithTimeout:30]);
+
+  GoldenTestManager* manager =


Make this test use GoldenTestManager so it isn't doing one-off screenshots.

jmagman · 2021-12-04T03:17:10Z

Grr, it failed and we don't upload the failing xc results bundle
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8828811169885771905/+/u/test:_Scenario_App_Integration_Tests/stdout

jmagman · 2021-12-04T03:42:19Z

Grr, it failed and we don't upload the failing xc results bundle https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8828811169885771905/+/u/test:_Scenario_App_Integration_Tests/stdout

I'm not sure how to diagnose this in LUCI. I'm going to revert to the unfortunate pixel cropping check, and revisit this change after upgrading to Xcode 13 on the bots.

cyanglaz

LGTM when CI passes

cyanglaz · 2021-12-06T17:41:40Z

testing/scenario_app/ios/Scenarios/ScenariosUITests/GoldenTestManager.m

+
+  // Sometimes even when the home bar indicator element no longer exists
+  // there's still a phantom animation. Wait another second to let it finish animating.
+  XCTWaiterResult waitResult =


Is this actually a thing? https://riptutorial.com/ios/example/24835/disable-animations-during-ui-testing

The animation in question is in springboard, not in our app.

This reverts commit 9b74c21.

…#30104)

* 9b74c21 Run iOS scenario apps on iPhone 11 and iOS 14 (flutter/engine#30104) * 4108898 Add error handler for os.rmtree calls (flutter/engine#30169) * 55c5e56 Don't run curl in verbose mode when --verbose set (flutter/engine#30170) * a1fe531 Win32: Implement DispatchAccessibilityAction (flutter/engine#30187) * a7a8024 Update buildroot to db6d037 (ToT). (flutter/engine#30171) * 7ccf290 Revert "Run iOS scenario apps on iPhone 11 and iOS 14 (#30104)" (flutter/engine#30193) * 1313e73 Add unconditional waits on fml::Semaphore. (flutter/engine#30165) * 094ff9e Roll Fuchsia Mac SDK from EAlr46NQ8... to zMg5gNi2E... (flutter/engine#30190) * a572ab5 Roll Fuchsia Linux SDK from WGMjaVH60... to s03VQc7lX... (flutter/engine#30191) * 07c6f41 Roll Skia from f333f5614a9b to 06f3d68627c2 (24 revisions) (flutter/engine#30192) * 8796f0f Update Swiftshader to dc0f131930 (flutter/engine#30188) * c1e235d Add length guard to fix Windows build with VS2019 16.11.7 headers (flutter/engine#30189) * 7831529 Roll Skia from 06f3d68627c2 to 1c4cf27965bd (2 revisions) (flutter/engine#30194) * 5672f9e Document rationale for Dart VM flag prefix match (flutter/engine#30195) * 1c10e68 Roll buildroot to 430b57c. (flutter/engine#30197) * e32d04a Roll Skia from 1c4cf27965bd to 543b8681c7f2 (1 revision) (flutter/engine#30198) * 6981c04 [ios platform view] fix overlay zPosition (flutter/engine#29930) * efaccf0 renew cirrus key (flutter/engine#30186) * 9b80c08 Roll Skia from 543b8681c7f2 to 00edeefab7f4 (1 revision) (flutter/engine#30200) * 164045d Roll Fuchsia Mac SDK from zMg5gNi2E... to QeaP059wu... (flutter/engine#30203) * 83b84d3 Roll Skia from 00edeefab7f4 to 21b8ccb7393c (3 revisions) (flutter/engine#30206)

…#30104)

google-cla bot added the cla: yes label Dec 3, 2021

jmagman marked this pull request as draft December 3, 2021 07:05

jmagman self-assigned this Dec 3, 2021

jmagman commented Dec 3, 2021

View reviewed changes

jmagman requested a review from cyanglaz December 3, 2021 07:34

jmagman marked this pull request as ready for review December 3, 2021 07:34

jmagman mentioned this pull request Dec 3, 2021

Make iOS 15 SDK and runtime available to engine builders flutter/flutter#85555

Closed

cyanglaz approved these changes Dec 3, 2021

View reviewed changes

Run iOS scenario apps on iPhone 11 and iOS 14

5a95b4e

jmagman force-pushed the scenario-sim-14 branch from 6d75e3e to 5a95b4e Compare December 3, 2021 18:52

Home indicator handling

2768901

jmagman commented Dec 4, 2021

View reviewed changes

cyanglaz approved these changes Dec 6, 2021

View reviewed changes

Revert home indicator change

d2ffefd

jmagman added the waiting for tree to go green This PR is approved and tested, but waiting for the tree to be green to land. label Dec 7, 2021

fluttergithubbot merged commit 9b74c21 into flutter:main Dec 7, 2021

jmagman deleted the scenario-sim-14 branch December 7, 2021 18:59

jmagman added a commit that referenced this pull request Dec 7, 2021

Revert "Run iOS scenario apps on iPhone 11 and iOS 14 (#30104)"

3634daf

This reverts commit 9b74c21.

jmagman mentioned this pull request Dec 7, 2021

Revert "Run iOS scenario apps on iPhone 11 and iOS 14" #30193

Merged

jason-simmons pushed a commit that referenced this pull request Dec 7, 2021

Revert "Run iOS scenario apps on iPhone 11 and iOS 14 (#30104)" (#30193)

7ccf290

This reverts commit 9b74c21.

engine-flutter-autoroll mentioned this pull request Dec 8, 2021

Roll Engine from 6551934b9c37 to 83b84d3336ec (21 revisions) flutter/flutter#94862

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Dec 8, 2021

9b74c21 Run iOS scenario apps on iPhone 11 and iOS 14 (flutter/engine…

48afbb4

…#30104)

engine-flutter-autoroll mentioned this pull request Dec 8, 2021

Roll Engine from 6551934b9c37 to 83b84d3336ec (21 revisions) flutter/flutter#94868

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Dec 8, 2021

9b74c21 Run iOS scenario apps on iPhone 11 and iOS 14 (flutter/engine…

65a0239

…#30104)

jmagman mentioned this pull request Dec 10, 2021

Add iOS version to scenario golden images #30263

Merged

8 tasks

jmagman added the platform-ios label Feb 18, 2022

Run iOS scenario apps on iPhone 11 and iOS 14 #30104

Run iOS scenario apps on iPhone 11 and iOS 14 #30104

Uh oh!

Conversation

jmagman commented Dec 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-launch Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cyanglaz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmagman commented Dec 4, 2021

Uh oh!

jmagman commented Dec 4, 2021

Uh oh!

cyanglaz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jmagman commented Dec 3, 2021 •

edited

Loading

cyanglaz left a comment •

edited

Loading