Repository navigation
Run the mac_unopt sub-builds as standalone builders - #195868
Conversation
| "--no-goma", | ||
| "--xcode-symlinks" | ||
| ], | ||
| "name": "ci/host_debug_arm64_tests", |
There was a problem hiding this comment.
| "--xcode-symlinks", | ||
| "--use-glfw-swiftshader" | ||
| ], | ||
| "name": "ci/host_debug_unopt_arm64", |
There was a problem hiding this comment.
| "name": "ci/host_profile_arm64_tests", | ||
| "description": "Produces profile mode arm64 macOS host-side tooling and builds host-side unit tests for arm64 macOS.", | ||
| "ninja": { | ||
| "config": "ci/host_profile_arm64_tests", |
There was a problem hiding this comment.
| "--no-goma", | ||
| "--xcode-symlinks" | ||
| ], | ||
| "name": "ci/host_release_arm64_licenses", |
There was a problem hiding this comment.
| "name": "ci/host_release_arm64_tests", | ||
| "description": "Produces release mode arm64 macOS host-side tooling and builds host-side unit tests for arm64 macOS.", | ||
| "ninja": { | ||
| "config": "ci/host_release_arm64_tests", |
There was a problem hiding this comment.
| "--target-dir", | ||
| "ci/ios_debug_unopt_sim_arm64" | ||
| ], | ||
| "name": "ci/ios_debug_unopt_sim_arm64", |
There was a problem hiding this comment.
| "--target-dir", | ||
| "ci/ios_debug_unopt_sim_arm64_extension_safe" | ||
| ], | ||
| "name": "ci/ios_debug_unopt_sim_arm64_extension_safe", |
There was a problem hiding this comment.
| "--xcode-symlinks", | ||
| "--use-glfw-swiftshader" | ||
| ], | ||
| "name": "ci/mac_release_arm64_tests", |
There was a problem hiding this comment.
| bringup: true | ||
| timeout: 120 | ||
| properties: | ||
| add_recipes_cq: "true" |
There was a problem hiding this comment.
Mac unopt is add_recipes_cq:
https://github.com/flutter/cocoon/blob/main/CI_YAML.md#properties
Whether to add this target to flutter/recipes CQ. This ensures changes to flutter/recipes pass on this target before landing.
Instead of making them all add_recipes_cq I figured just adding 1-2 would give enough coverage. I can change them all to add_recipes_cq: "true" though I don't have a preference.
There was a problem hiding this comment.
sgtm, we can tweak it in the future if we want
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request adds several new Mac arm64 standalone builder configurations to .ci.yaml along with their corresponding JSON configuration files. The review feedback highlights typos in the target OS and SDK runtime versions, specifically pointing out that Mac-26.6 should be corrected to a valid macOS version such as Mac-14.6, and ios-26-2_17c52 should be corrected to ios-17-2_17c52 to prevent builder failures.
There was a problem hiding this comment.
Code Review
This pull request introduces several new macOS and iOS arm64 builder configurations to .ci.yaml along with their corresponding standalone JSON configuration files. The review feedback highlights that the $flutter/osx_sdk property is duplicated in .ci.yaml for the mac_ios_debug_unopt_sim_arm64 and mac_ios_debug_unopt_sim_arm64_extension_safe targets, as these properties are already defined in their respective JSON files. This duplication violates the style guide rule to avoid duplicating state and maintain a single source of truth.
gaaclarke
left a comment
There was a problem hiding this comment.
lgtm. and just to be sure, these macs are not vms, right? so a mostly idle computer waiting on subtasks is in fact sitting on resources?
| bringup: true | ||
| timeout: 120 | ||
| properties: | ||
| add_recipes_cq: "true" |
There was a problem hiding this comment.
sgtm, we can tweak it in the future if we want
These Macs are not VMs. However even if they were these are the bigger wins:
|
…migration/phase0 The branch was 61 commits behind master. Merged rather than rebased: the branch's commits are pushed and under review (PR flutter#196005). Conflicts, resolved by hand: - dev/bots/test.dart: master's flutter#195863 balances tool_tests_commands and web_tool_tests by duration through _selectWeightedToolTests (one durations file, dev/bots/tool_test_durations.json, for the three tool shards); the Bazel branches of the three shard runners now cover the paths it selects, as before over selectIndexOfTotalSubshard's. Regenerated for master's rolls and renames: - build_defs/repositories/deps.lock.bzl (gen_deps.dart): the Dart SDK roll to c9153304136a and its package revisions, the Fuchsia SDK and test scripts. - build_defs/dart/pub.lock.bzl (gen_pub_lock.dart after `dart pub get` at the root and in lib/web_ui): the pubspec changes of master. - engine/src/flutter/lib/ui/BUILD.bazel (gn_to_bazel): the GLES compositing unit test renamed to impeller_gles_without_surface_unittests.cc. - engine/src/flutter/impeller/BUILD.bazel (gn_to_bazel): run_tests.py's Vulkan synchronization-validation list grew by the tests flutter#194989 fixed. `gn_to_bazel --check` is current. Adjusted by hand for master's renames: - dev/bots/BUILD.bazel and packages/flutter_tools/BUILD.bazel: the durations test is tool_test_durations_test over tool_test_durations.json and the test files of the commands, integration and web shards (`durations_shard_test_files`, was `integration_shard_test_files`). - dev/bots/test/test_test.dart: the `_bazel` subshard test expects master's "Selecting subshard 1 of 3 (N of M tests, estimated weight W)". Checked: gclient sync clean (Skia f96be51af95d, Dart c9153304136a); gn_to_bazel 282/282 and --check current; bazel_tools 46/46; bazel_ci_test.py 23; pylint clean; Bazel analysis of //:framework_tests //:tool_tests //:dart_tests //dev/bots:tests and of ci_host_debug_unopt (+tests), android_debug, wasm_release, ci_android_release_arm64; `bazel test //dev/bots:tests` green; dev/bots test_test.dart's Bazel tests green (its `--dry-run prints every test` test depends on SHARD being set in the environment, on master as here). Note: master's flutter#195868 adds the mac_unopt builds as standalone bringup builders under ci/builders/standalone/ (GN; the converter does not read that directory). mac_unopt.json itself still carries --bazel.
The standalone builders added in flutter#195868 are passing in post-submit. Remove bringup from them and delete the old mac_unopt orchestrator builder and its config. Update docs that referenced mac_unopt.json. (cherry picked from commit 84d0eb2)
The standalone builders added in flutter#195868 are passing in post-submit. Remove bringup from them and delete the old mac_unopt orchestrator builder and its config. Update docs that referenced mac_unopt.json. (cherry picked from commit 84d0eb2)
The standalone builders added in flutter#195868 are passing, remove `bringup: true` and delete the `Mac mac_unopt` builder. The new builds ran on this PR, for example: https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20mac_host_release_arm64_licenses/1/overview A few docs linked to mac_unopt.json or used it as an example, point them instead to the `standalone/` configs. Part of flutter#155041 Fixes flutter#186462 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
This pull request is created by [automatic cherry pick workflow](https://github.com/flutter/flutter/blob/main/docs/releases/Flutter-Cherrypick-Process.md#automatically-creates-a-cherry-pick-request) Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request. ### Issue Link: #186462 2 PRs: #195868 and #196027 ### Impact Description: Release builders should match the master branch configs. Without this change, the beta release branch will try to run the old builder names and fail because the infra config is missing. ### Changelog Description: N/A - CI test-only change ### Workaround: N/A ### Risk: What is the risk level of this cherry-pick? ### Test Coverage: Are you confident that your fix is well-tested by automated tests? ### Validation Steps: Release PRs should not complain about missing builders.
…'s builds were; the converter reads ci/builders/standalone/*.json Master's flutter#195868 and flutter#196027 moved mac_unopt.json's eight builds into one-build files under ci/builders/standalone/ (the engine_v2/builder recipe's form: name, gn, ninja and tests at the top of the file) and removed the mac_unopt builder, so 523's merge took the deletion and the Mac test drones -- host_{debug,profile,release}_arm64_tests, host_release_arm64_licenses, mac_release_arm64_tests, host_debug_unopt_arm64, ios_debug_unopt_sim_arm64 and its extension-safe twin -- ran GN again in 523's presubmit (and ios_debug_unopt_sim_arm64 failed a timing test of the GN-built embedder unit tests, testViewWillTransitionToSize_DoesDelayEngineCallIfNonZeroDuration, three times; it passes on master's postsubmit and failed another PR the same way this morning). - tools/gn_to_bazel: withStandaloneBuilds (builders.dart) reads a standalone file's build out of it -- BuildConfigLoader reads such a file as a builder file with no builds -- as a builder file of that one build under the loader's key, so the host's prefix, the configurations, the installs, the suites, builds.json and the --bazel checks cover them as they cover linux_*.json and mac_*.json; gn_to_bazel.dart applies it. Test added. The GN-only standalone builds (linux_benchmarks, linux_clangd, mac_clangd) get configurations like any other GN build's; linux_license's --enable-minimal-linux has none. - ci/builders/standalone/mac_*.json (the eight): --no-rbe for --rbe, --bazel appended, download_bazelisk among the gclient variables (use_rbe stays true: the Mac's reclient and the Linux clang for the wraps), and the licenses build's unit-test step as run_tests.py --type binaries, as 517 had it in mac_unopt.json (the converter refuses the executable step). mac_clangd.json stays GN. - ci/builders/BUILD.bazel, configs.bazelrc, build/bazel/builds.json regenerated: the eight Mac configurations and installs are back, plus the GN-only standalone builds' configurations. ci.md's builder list. Converter 282+1 tests, --check current; engine_build_configs 84; bazel_ci_test 23; buildifier clean.
Swift Testing runs a suite's tests in parallel. `testStaticLogInfoDoesNotEvaluateMessageBelowLogLevel` sets the global `Logger.logLevel` to `.warning` and expects no output, while `testConcurrentLoggingAndLevelMutation` flips it to `.info` and logs. When they overlap the first one fails. Put both in a nested `@Suite(.serialized)`. This is the top cause of test-step retries on the standalone Mac mac_host_*_arm64_tests builders from flutter#195868, and each retry reruns the whole 7–27 minute step. Fixes flutter#194776 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
…'s builds were; the converter reads ci/builders/standalone/*.json Master's flutter#195868 and flutter#196027 moved mac_unopt.json's eight builds into one-build files under ci/builders/standalone/ (the engine_v2/builder recipe's form: name, gn, ninja and tests at the top of the file) and removed the mac_unopt builder, so 523's merge took the deletion and the Mac test drones -- host_{debug,profile,release}_arm64_tests, host_release_arm64_licenses, mac_release_arm64_tests, host_debug_unopt_arm64, ios_debug_unopt_sim_arm64 and its extension-safe twin -- ran GN again in 523's presubmit (and ios_debug_unopt_sim_arm64 failed a timing test of the GN-built embedder unit tests, testViewWillTransitionToSize_DoesDelayEngineCallIfNonZeroDuration, three times; it passes on master's postsubmit and failed another PR the same way this morning). - tools/gn_to_bazel: withStandaloneBuilds (builders.dart) reads a standalone file's build out of it -- BuildConfigLoader reads such a file as a builder file with no builds -- as a builder file of that one build under the loader's key, so the host's prefix, the configurations, the installs, the suites, builds.json and the --bazel checks cover them as they cover linux_*.json and mac_*.json; gn_to_bazel.dart applies it. Test added. The GN-only standalone builds (linux_benchmarks, linux_clangd, mac_clangd) get configurations like any other GN build's; linux_license's --enable-minimal-linux has none. - ci/builders/standalone/mac_*.json (the eight): --no-rbe for --rbe, --bazel appended, download_bazelisk among the gclient variables (use_rbe stays true: the Mac's reclient and the Linux clang for the wraps), and the licenses build's unit-test step as run_tests.py --type binaries, as 517 had it in mac_unopt.json (the converter refuses the executable step). mac_clangd.json stays GN. - ci/builders/BUILD.bazel, configs.bazelrc, build/bazel/builds.json regenerated: the eight Mac configurations and installs are back, plus the GN-only standalone builds' configurations. ci.md's builder list. Converter 282+1 tests, --check current; engine_build_configs 84; bazel_ci_test 23; buildifier clean.
Mac mac_unoptis an orchestrator. It schedules 8 Mac sub-builds, and holds a Mac bot for the 43 minutes they take.Instead, this PR moves each sub-build to a top-level engine_v2/builder target, which is how
windows_unopthas run since flutter-team-archive/engine#41785. The build definitions are copied unchanged into ci/builders/standalone/.The new targets start in bringup and a follow-up PR will delete Mac mac_unopt and mac_unopt.json.
With this change each builder will have its own history on the dashboard, so a failure shows which build broke rather than "mac_unopt". When one flakes, only that one reruns, ditto suppressions. When there's a flake it will be much easier to spelunk into the logs to figure out what's happening (see #185908).
The orchestrator's bot goes away (around ~300 Mac bot-hours / wk). Unfortunately wall time doesn't change because the sub-builds were already running in parallel (though it should be easier to make changes in the future that improve wall time). This will also add 8 new squares to the dashboard. Will also need release branch cherry-picks to get the infra configs to match.
This shouldn't impact building/uploading engine binaries or anything in the merge queue (afaik).
Additional history:
mac_unoptorchestrator builder to Linux flutter-team-archive/engine#41184 and Remove 'Mac mac_unopt' in favor of Linux flutter-team-archive/engine#41226Part of #155041
Part of #186462
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.