[snapshot] use simulator UDIDs for destinations#30054
Conversation
Generate modern snapshot xcodebuild destinations with the resolved simulator UDID so test execution targets the same simulator that preparation steps modified. Document the iOS 26.4 runtime mismatch investigation and reproduction notes.
|
thanks! Is this accurate to say this fixes mismatches across these shifts that occurred between lookup or the dreaded error 70?
|
|
@iBotPeaches Yes, as far as I can tell from looking at the issue descriptions this fix should solve all of them. |
|
Thanks. Just trying to comb some history to see why we moved away from the UDID in first place, since to your point it seems in way older code it was that way already. |
There was a problem hiding this comment.
Hey @philprime 👋
Thank you for your contribution to fastlane and congrats on getting this pull request merged 🎉
The code change now lives in the master branch, however it wasn't released to RubyGems yet.
We usually ship about once a month, and your PR will be included in the next one.
Please let us know if this change requires an immediate release by adding a comment here 👍
We'll notify you once we shipped a new release with your changes 🚀"
|
@philprime Thank you! This approach is airtight and fixes all different kinds of problems that we have seen for looking up by name. |
|
Happy to help! 😃 |
There was a problem hiding this comment.
Congratulations! 🎉 This was released as part of fastlane 2.236.0 🚀
Checklist
bundle exec rspecfrom the root directory to see all new and existing tests passbundle exec rubocop -ato ensure the code style is validci/circlecibuilds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)Motivation and Context
snapshotalready prepares simulators by resolving a concrete simulator UDID. For example, it opens Simulator and runssimctlcommands against that UDID:But the generated
xcodebuildcommand did not use that same UDID. It emitted a separateplatform,name,OSdestination:That can fail when CoreSimulator and
xcodebuilddisagree about the string OS version for the same simulator runtime. In the observed CI failure, snapshot prepared aniPhone 17 Prosimulator by UDID, butxcodebuildfailed to resolveOS=26.4:The available destinations in that job included
iPhone 17 Prodevices forOS:26.4.1andOS:26.5, but notOS:26.4.CI references:
Description
This changes the modern snapshot test command generator to pass the resolved simulator UDID to
xcodebuild:The important part is that
xcodebuildnow receives the same simulator identity that snapshot uses during simulator preparation. This avoids a second destination lookup bynameandOS, which can select no device or a different device when runtime metadata is inconsistent.The local reproduction environment had both iOS 26.4 and iOS 26.4.1 runtimes installed.
simctl list runtimesreported both under the same display runtime name and identifier:simctl list devicesthen printed duplicate-- iOS 26.4 --sections, whilexcodebuild -showdestinationsreported the same simulator UDID asOS:26.4.1:Using a UDID destination avoids depending on the mismatched OS string.
This also keeps explicit
ios_versionbehavior: if a user configuresios_version, snapshot still uses that version when resolving the simulator. The change removes the implicitLatestOsVersion.version(os)lookup from the defaultdestinationpath so the generatedxcodebuildcommand uses the same default simulator lookup contract as the preparation steps. Without that, snapshot could still prepare one UDID and run tests on another one.The older Xcode 8 generator already used a UDID destination. This aligns the modern generator with that approach.
Regression coverage was added for:
id=<udid>instead ofname=<name>,OS=<version>;xcodebuilddestination matchingTestCommandGenerator.device_udid;Testing Steps
Focused destination regression specs:
bundle exec rspec \ snapshot/spec/test_command_generator_spec.rb:31 \ snapshot/spec/test_command_generator_spec.rb:42 \ snapshot/spec/test_command_generator_spec.rb:52Broader command-generator specs:
bundle exec rspec \ snapshot/spec/test_command_generator_spec.rb \ snapshot/spec/test_command_generator_xcode_8_spec.rbManual destination reproduction:
xcodebuild -showBuildSettings \ -scheme ScreenshotUITests \ -project ./Flinky.xcodeproj \ -configuration Debug \ -destination 'platform=iOS Simulator,name=iPhone 17 Pro,OS=26.4'Observed result:
The same request succeeds when addressed by concrete UDID:
xcodebuild -showBuildSettings \ -scheme ScreenshotUITests \ -project ./Flinky.xcodeproj \ -configuration Debug \ -destination 'platform=iOS Simulator,id=FC42BA48-38ED-4A94-8E96-5512C25BB0F9'Observed result includes:
fixes: #29553
fixes: #29481
fixes: #30040
fixes: #22284