With looming updates on the horizon, we will need to perform various upgrades to our devicelab machines. This issue is meant to track the overall process of performing the necessary updates.

• Move flutter-devicelab-mac-45 to try pool #134558 (@yusuf-goog)
• Move mac-346-g140 to try pool #134559 (@yusuf-goog)
• Upgrade one chromium x86 bot to macOS 13.5+ from try pool #134742 (@yusuf-goog)
• Use unaltered Xcode in CI on macOS 13 #134735 (@vashworth)
• Upgrade 1 x86 devicelab bot with iOS device in try pool to macOS 13.5+ #136256 (@yusuf-goog)
• Upgrade 1 arm devicelab bot in staging pool with msm8952 device to macOS 13.5+ #136396 (@yusuf-goog)
• Upgrade another chromium x86 bot to macOS 13.5+ from try pool  #136257 (@yusuf-goog)
• Test Xcode 14.3.1 on macOS 13.5+ (@vashworth)
• Update .ci.yaml to use either macOS 12 or 13 #134736 (@vashworth)
• Upgrade Phase 1 of Upgrade fleet to macOS 13 #134737 (@yusuf-goog)
  • Blocking Issues discovered:
    • Xcode install on macOS 13 sometimes fails and then continuously fails on following runs #138238 (workaround added)
    • Corrupted Xcode causes flutter doctor to fail repeatedly #139152
    • CIPD auth not working on macOS 13 flutter-devicelab-mac-x #138043
    • Recipes don't wait for simulator runtimes to be unmounted #137634
    • -[UnobstructedPlatformViewTests testMultiplePlatformViewsWithOverlays] test flaky on macOS 13.6.1 #138193 (temporarily fixed in Temporarily disable test UnobstructedPlatformViewTests testMultiplePlatformViewsWithOverlays engine#47883)
  • Other Issues discovered:
    • Xcode cache mysteriously gets cleared on macOS #138246
    • Some tests that use iOS simulators get stuck and timeout #138224
    • Reinstall certs on macOS 13 bots #138194
    • Newer versions of mac_toolchain slow down Xcode install by ~2 minutes #138109
• Test Xcode 15 / iOS 17 simulator / macOS 13 via PR and led (@vashworth)
• Upgrade Phase 2 of Upgrade fleet to macOS 13 #134737 (@yusuf-goog)
• Update 50% of macOS tests to Xcode 15 #140975
  • Devicelab is done
  • Updating chromium tests is blocked by:
    • Upgrade release bots in pool luci.dart-internal.flutter to macOS 13 #140835
• Upgrade Phase 3 of Upgrade fleet to macOS 13 #134737 (@yusuf-goog)
• Upgrade iOS devices in staging pool to iOS 17 #140908 (@yusuf-goog)
• Allow iOS devicelab tests to run on either iOS 16 or iOS 17 #141177 (@vashworth)
• Test Xcode 15 / iOS 17 device / macOS 13 on devicelab tests via led (@vashworth)
• Update all tests to use Xcode 15 / iOS 17 simulator / macOS 13: ☂️ Upgrade CI to Xcode 15 and iOS 17 simulator #132237
  • Blocking Issues:
    • Upgrade release bots in pool luci.dart-internal.flutter to macOS 13 #140835
• Upgrade Phase 1 for iOS 17 (~25%) #132252
  • Blocking Issues:
    • "Run app to dismiss dialogs" flakey with iOS 17 devices #141383 - workaround added
    • Xcode automation permission sometimes fails with "Error: AppleEvent timed out." #141624 - was specific to one bot and has not reoccured since
• Upgrade Phase 2 for iOS 17 (more x86 prod bots) #142326
• Upgrade Phase 4 of Upgrade fleet to macOS 13 #134737 (@yusuf-goog)
  • Blocking Issues:
    • Run flutter/flutter tests on 3.16 stable release on either macOS 12 or macOS 13 #140895
    • Upgrade release bots in pool luci.dart-internal.flutter to macOS 13 #140835
    • Safari Web Flutter Engine Goldens differ on macOS 13 from macOS 12 #139924
    • Failure to connect to webdriver process on macOS 13 in CI #136972
    • test: run suite safari-dart2js-canvaskit-canvaskit fails on macOS 13 in CI #136274
    • test: run suite safari-dart2js-html-engine fails on macOS 13 in CI #136279
    • ☂️ Upgrade CI to Xcode 15 and iOS 17 simulator #132237 (@vashworth)
• Upgrade Phase 3 for iOS 17 (all but bare minimum) #142314
• Run devicelab tests on iOS 17 exclusively #144020
• Allow stable and beta branches to run tests on iOS 17 #143003
• Upgrade Phase 4 for iOS 17 #144021
• ☔ automate ruby self contained packages #136377 (@godofredoc)
  • Cannot install CocoaPods 1.12 in CI, failed installing dependency json 2.6.3 with native extensions #122057 (@godofredoc)
    • [devicelab/infra] update cocoapods on devicelab machines to 1.13.0 #133096 (@godofredoc)