[CP-stable] Run all flutter/flutter macOS tests using Xcode 26 and iOS 26 simulator#188374
Conversation
…or (flutter#185651) Updates Xcode SDK version from 16c5032a to 17c52 (Xcode 26.2), macOS host from Mac-15.5|Mac-15.7 to Mac-15.7, and simulator runtime from ios-18-2 to ios-26-2. Replaces 18.2 golden images with 26.2 goldens and updates generate_builder_json.dart. device_os remains iOS-18 for devicelab targets (physical devices not yet upgraded). Fixes flutter#172855 *If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].* ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [ ] I updated/added relevant documentation (doc comments with `///`). - [ ] I added new tests to check the change I am making, or this PR is [test-exempt]. - [ ] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [ ] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
|
This pull request was opened from and to a release candidate branch. This should only be done as part of the official Flutter release process. If you are attempting to make a regular contribution to the Flutter project, please close this PR and follow the instructions at Tree Hygiene for detailed instructions on contributing to Flutter. Reviewers: Use caution before merging pull requests to release branches. Ensure the proper procedure has been followed. |
There was a problem hiding this comment.
Code Review
This pull request updates CI configurations to use Mac-15.7 and OSX SDK version 17c52. It also modifies iOS simulator runtime versions and golden image references from 18.2 to 26.2. Feedback indicates that iOS version 26.2 does not exist, which will cause CI and golden tests to fail. The reviewer recommends reverting the simulator runtime versions and project references back to 18.2.
| defaultsTo: 'com.apple.CoreSimulator.SimRuntime.iOS-26-2', | ||
| ) | ||
| ..addOption( | ||
| 'os-version', | ||
| help: 'The OS version of the iOS simulator device to use.', | ||
| defaultsTo: '18.2', | ||
| defaultsTo: '26.2', |
There was a problem hiding this comment.
The iOS simulator runtime version 26.2 and identifier iOS-26-2 do not exist. The current latest iOS version is 18.2. It appears that 18 was incorrectly replaced with 26 throughout the PR (possibly due to an automated script or cherry-pick workflow error). This will cause the CI to fail as it won't be able to find or launch the specified simulator runtime.
Please revert these values to iOS-18-2 and 18.2 respectively.
| defaultsTo: 'com.apple.CoreSimulator.SimRuntime.iOS-26-2', | |
| ) | |
| ..addOption( | |
| 'os-version', | |
| help: 'The OS version of the iOS simulator device to use.', | |
| defaultsTo: '18.2', | |
| defaultsTo: '26.2', | |
| defaultsTo: 'com.apple.CoreSimulator.SimRuntime.iOS-18-2', | |
| ) | |
| ..addOption( | |
| 'os-version', | |
| help: 'The OS version of the iOS simulator device to use.', | |
| defaultsTo: '18.2', |
| "$flutter/osx_sdk": { | ||
| "runtime_versions": [ | ||
| "ios-18-2_16c5032a" | ||
| "ios-26-2_17c52" |
| F76CA9092C8BBAD6002A7EF5 /* golden_platform_view_multiple_impeller_iPhone SE (3rd generation)_18.2_simulator.png in Resources */ = {isa = PBXBuildFile; fileRef = F76CA9062C8BBAD6002A7EF5 /* golden_platform_view_multiple_impeller_iPhone SE (3rd generation)_18.2_simulator.png */; }; | ||
| F76CA90A2C8BBAD6002A7EF5 /* golden_two_platform_view_clip_rect_impeller_iPhone SE (3rd generation)_18.2_simulator.png in Resources */ = {isa = PBXBuildFile; fileRef = F76CA9072C8BBAD6002A7EF5 /* golden_two_platform_view_clip_rect_impeller_iPhone SE (3rd generation)_18.2_simulator.png */; }; | ||
| F76CA90B2C8BBAD6002A7EF5 /* golden_two_platform_view_clip_rect_multiple_clips_impeller_iPhone SE (3rd generation)_18.2_simulator.png in Resources */ = {isa = PBXBuildFile; fileRef = F76CA9082C8BBAD6002A7EF5 /* golden_two_platform_view_clip_rect_multiple_clips_impeller_iPhone SE (3rd generation)_18.2_simulator.png */; }; | ||
| 04EB617CD958B43CC1A80CAE /* golden_bogus_font_text_impeller_iPhone SE (3rd generation)_26.2_simulator.png in Resources */ = {isa = PBXBuildFile; fileRef = 21C15BDB794F61D0706DF179 /* golden_bogus_font_text_impeller_iPhone SE (3rd generation)_26.2_simulator.png */; }; |
There was a problem hiding this comment.
The golden image files and their references in the Xcode project have been renamed to use 26.2_simulator.png. Since the actual simulator runtime is iOS 18.2, the tests will look for 18.2_simulator.png files, causing the golden tests to fail.
Please rename the golden image files back to use 18.2 and revert the references in this project file.
07b642a
into
flutter:flutter-3.44-candidate.0
This pull request is created by automatic cherry pick workflow
Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request.
Issue Link:
What is the link to the issue this cherry-pick is addressing?
#172864
Impact Description:
Without this cherry-pick, the stable channel CI infrastructure cannot run its macOS and iOS simulator tests using Xcode 26 and macOS 15. This blocks upgrading the stable branch's CI pools to newer macOS/Xcode versions and leaves them running on older versions.
Changelog Description:
Explain this cherry pick:
See best practices for examples.
< Replace with changelog description here >
[flutter/172864] Run stable channel macOS tests on macOS 15 and Xcode 26.
Workaround:
No workaround exists as this affects the CI infrastructure configurations.
< Replace with workaround here >
Risk:
What is the risk level of this cherry-pick?
Test Coverage:
Are you confident that your fix is well-tested by automated tests?
Validation Steps:
What are the steps to validate that this fix works?
CI builders for the stable branch (specifically macOS and iOS simulator shards) should pass successfully when using macOS 15 and Xcode 26.