Skip to content

Commit 07510ad

Browse files
authored
Move iOS compile benchmarks to Mac_benchmark and pin Mac_benchmark to M1 (flutter#192472)
Part of flutter#186414 This batch moves the 4 iOS compile benchmarks off physical-device bots and onto the pinned benchmark pool: * `basic_material_app_ios__compile` * `flutter_gallery_ios__compile` * `hello_world_ios__compile` * `imitation_game_flutter` (task: `imitation_game_flutter__compile`) These tasks only run `flutter build ios` and never launch an app on the attached iPhone (details in [flutter#186414 (comment)](flutter#186414 (comment))). On `Mac_arm64_ios` they tie up physical devices and, because that pool mixes M1 and M4 Pro hosts, every metric is split into two disconnected Skia Perf series (`device_type=iPhone_11` vs `iPhone_16_Pro`) with a ~40–60% timing gap. Per [flutter#186414 (comment)](flutter#186414 (comment)), instead of moving them to the unpinned `Mac_arm64` pool: **Changes** - `platform_properties.mac_benchmark`: `mac_model` `"Macmini8,1"` (Intel, being turned down per [flutter.dev/go/macos-intel-deprecation](https://flutter.dev/go/macos-intel-deprecation)) → `"Macmini9,1"` (Apple Silicon M1 Mac mini), plus `cpu: arm64` and an `arm64` tag. - Moved the 4 targets from `Mac_arm64_ios` to `Mac_benchmark` with `bringup: true`. They inherit `device_type: none` and `["devicelab", "hostonly", "mac", "arm64"]` from the platform; dropping the `ios` tag also stops the devicelab recipe from running `ideviceinfo` (which would fail on a host-only bot). `flutter_gallery_ios__compile` keeps the `ruby` dependency for CocoaPods, matching `flutter_gallery_macos__compile`. - `imitation_game_swiftui` stays on `Mac_arm64_ios` — it runs `xcodebuild test` against a physical device. **Heads-up for the metrics sheriff** - All 13 existing `Mac_benchmark` macOS targets (`*_macos__compile`, `*_macos__start_up`, `complex_layout_scroll_perf_macos*`, `new_gallery_macos_impeller__transition_perf`, `flutter_tool_startup__macos`, …) move from Intel to M1 hardware with this change, so their baselines will shift (and the `arch` benchmark tag changes from `intel` to `arm`). Expect one-time step changes on Skia Perf rather than regressions. - Host-only `Macmini9,1` capacity in `luci.flutter.prod` is ~56 bots (vs ~37 Intel `Macmini8,1`), shared with `Mac_arm64`; today that pool is all M1, so the pin mostly future-proofs against M4 Pros being added.
1 parent ed2392e commit 07510ad

1 file changed

Lines changed: 19 additions & 14 deletions

File tree

‎.ci.yaml‎

Lines changed: 19 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -211,10 +211,15 @@ platform_properties:
211211
{"dependency": "apple_signing", "version": "version:to_2027"}
212212
]
213213
device_type: none
214-
mac_model: "Macmini8,1"
214+
# Benchmarks are pinned to a single Mac model so that timing metrics are
215+
# comparable across runs. Macmini9,1 is the Apple Silicon (M1) Mac mini;
216+
# the Intel Macmini8,1 bots are being turned down
217+
# (https://flutter.dev/go/macos-intel-deprecation).
218+
mac_model: "Macmini9,1"
219+
cpu: arm64
215220
os: Mac-15.7
216221
tags: >
217-
["devicelab", "hostonly", "mac"]
222+
["devicelab", "hostonly", "mac", "arm64"]
218223
$flutter/osx_sdk : >-
219224
{
220225
"sdk_version": "17c52"
@@ -5362,13 +5367,12 @@ targets:
53625367
["devicelab", "ios", "mac"]
53635368
task_name: backdrop_filter_perf_ios__timeline_summary
53645369

5365-
- name: Mac_arm64_ios basic_material_app_ios__compile
5370+
- name: Mac_benchmark basic_material_app_ios__compile
53665371
recipe: devicelab/devicelab_drone
53675372
presubmit: false
5373+
bringup: true
53685374
timeout: 60
53695375
properties:
5370-
tags: >
5371-
["devicelab", "ios", "mac"]
53725376
task_name: basic_material_app_ios__compile
53735377

53745378
- name: Mac channels_integration_test_ios
@@ -5472,13 +5476,16 @@ targets:
54725476
["devicelab", "hostonly", "mac"]
54735477
task_name: flavors_test_ios
54745478

5475-
- name: Mac_arm64_ios flutter_gallery_ios__compile
5479+
- name: Mac_benchmark flutter_gallery_ios__compile
54765480
recipe: devicelab/devicelab_drone
54775481
presubmit: false
5482+
bringup: true
54785483
timeout: 60
54795484
properties:
5480-
tags: >
5481-
["devicelab", "ios", "mac", "arm64"]
5485+
dependencies: >-
5486+
[
5487+
{"dependency": "ruby", "version": "ruby_3.1-pod_1.13"}
5488+
]
54825489
task_name: flutter_gallery_ios__compile
54835490

54845491
- name: Mac_ios flutter_gallery_ios__start_up
@@ -5499,22 +5506,20 @@ targets:
54995506
["devicelab", "ios", "mac"]
55005507
task_name: flutter_view_ios__start_up
55015508

5502-
- name: Mac_arm64_ios hello_world_ios__compile
5509+
- name: Mac_benchmark hello_world_ios__compile
55035510
recipe: devicelab/devicelab_drone
55045511
presubmit: false
5512+
bringup: true
55055513
timeout: 60
55065514
properties:
5507-
tags: >
5508-
["devicelab", "ios", "mac", "arm64"]
55095515
task_name: hello_world_ios__compile
55105516

5511-
- name: Mac_arm64_ios imitation_game_flutter
5517+
- name: Mac_benchmark imitation_game_flutter
55125518
recipe: devicelab/devicelab_drone
55135519
presubmit: false
5520+
bringup: true
55145521
timeout: 60
55155522
properties:
5516-
tags: >
5517-
["devicelab", "ios", "mac", "arm64"]
55185523
task_name: imitation_game_flutter__compile
55195524

55205525
- name: Mac_arm64_ios imitation_game_swiftui

0 commit comments

Comments
 (0)