Run Xcode command lines tools in native ARM #68050

jmagman · 2020-10-13T21:01:07Z

Description

On macOS ARM machines run Xcode outside Rosetta without x86 translation by prepending commands with arch -arm64.

Only update the iOS xcodebuild code paths. More research needed to see what changes flutter build macos needs.

Related Issues

Fixes #65133

Tests

xcrun runs natively on arm64 test, updated xcrun tests.

Breaking Change

No, no existing tests failed, so this is not a breaking change.
Yes, this is a breaking change. If not, delete the remainder of this section.

jmagman · 2020-10-13T23:31:25Z

packages/flutter_tools/lib/src/base/os.dart

I checked again, it exits 1 on x86 macOS 11, too.

jmagman · 2020-10-13T23:34:49Z

packages/flutter_tools/lib/src/macos/xcode.dart

This check is overkill. If Xcode and xcrun is installed, xcdevice is installed.

jmagman · 2020-10-13T23:35:58Z

packages/flutter_tools/test/general.shard/macos/xcode_test.dart

This test is new.

jmagman · 2020-10-13T23:38:29Z

packages/flutter_tools/test/general.shard/macos/xcode_test.dart

This section looks like a big change, but it's mostly whitespace due to the the new group indentation. The changes in this file:

Remove xcrun --find xcdevice commands.

Use FakePlatform instead of MockPlatform.

Add the sysctl ARM check command where needed.

christopherfujino

LGTM!

jonahwilliams

LGTM

jonahwilliams · 2020-10-14T18:49:37Z

packages/flutter_tools/lib/src/macos/xcode.dart

nit: doc comment please

jmagman · 2020-10-14T22:46:53Z

@alexmarkley I know you've been offering to help out with ARM support. If you are still willing, it would be awesome if you could to take this change for a test drive and see where else the flutter command line tool falls down when running against a physical iOS device...

alexmarkley · 2020-10-14T22:49:58Z

@jmagman thanks for tagging me! I will take a look later tonight.

alexmarkley · 2020-10-15T00:23:13Z

@jmagman first impressions. Ran flutter upgrade in flutter channel master. Here's the output of flutter doctor -v:

DTK:~ alex$ flutter doctor -v
[✓] Flutter (Channel master, 1.23.0-19.0.pre.41, on macOS 11.0 20A5395g darwin-arm, locale en-US)
    • Flutter version 1.23.0-19.0.pre.41 at /Users/alex/Development/Flutter/flutter
    • Framework revision ed2a4809da (17 minutes ago), 2020-10-14 20:02:04 -0400
    • Engine revision ce75dda492
    • Dart version 2.11.0 (build 2.11.0-218.0.dev)

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, please use
      `flutter config --android-sdk` to update to that location.


⣽Error executing simctl: 1
arch: posix_spawnp: xcrun: Bad CPU type in executable


[!] Xcode - develop for iOS and macOS (Xcode 12.2)
    • Xcode at /Applications/Xcode-beta.app/Contents/Developer
    • Xcode 12.2, Build version 12B5035g
    ✗ Xcode requires additional components to be installed in order to run.
      Launch Xcode and install additional required components when prompted or run:
        sudo xcodebuild -runFirstLaunch
    • CocoaPods version 1.9.3

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).

[✓] Connected device (1 available)
    • macOS (desktop) • macos • darwin-x64 • macOS 11.0 20A5395g darwin-arm

! Doctor found issues in 3 categories.
DTK:~ alex$

Not sure what's going on with simctl. Running xcrun simctl --help yields:

DTK:~ alex$ xcrun simctl --help
usage: simctl [--set <path>] [--profiles <path>] <subcommand> ...
       simctl help [subcommand]
Command line utility to control the Simulator

For subcommands that require a <device> argument, you may specify a device UDID
or the special "booted" string which will cause simctl to pick a booted device.
If multiple devices are booted when the "booted" device is selected, simctl
will choose one of them.

Subcommands:
	create              Create a new device.
	clone               Clone an existing device.
	upgrade             Upgrade a device to a newer runtime.
	delete              Delete spcified devices, unavailable devices, or all devices.
	pair                Create a new watch and phone pair.
	unpair              Unpair a watch and phone pair.
	pair_activate       Set a given pair as active.
	erase               Erase a device's contents and settings.
	boot                Boot a device.
	shutdown            Shutdown a device.
	rename              Rename a device.
	getenv              Print an environment variable from a running device.
	openurl             Open a URL in a device.
	addmedia            Add photos, live photos, videos, or contacts to the library of a device.
	install             Install an app on a device.
	uninstall           Uninstall an app from a device.
	get_app_container   Print the path of the installed app's container
	install_app_data    Install an xcappdata package to a device, replacing the current contents of the container.
	launch              Launch an application by identifier on a device.
	terminate           Terminate an application by identifier on a device.
	spawn               Spawn a process by executing a given executable on a device.
	list                List available devices, device types, runtimes, or device pairs.
	icloud_sync         Trigger iCloud sync on a device.
	pbsync              Sync the pasteboard content from one pasteboard to another.
	pbcopy              Copy standard input onto the device pasteboard.
	pbpaste             Print the contents of the device's pasteboard to standard output.
	help                Prints the usage for a given subcommand.
	io                  Set up a device IO operation.
	diagnose            Collect diagnostic information and logs.
	logverbose          enable or disable verbose logging for a device
	status_bar          Set or clear status bar overrides
	ui                  Get or Set UI options
	push                Send a simulated push notification
	privacy             Grant, revoke, or reset privacy and permissions
	keychain            Manipulate a device's keychain
DTK:~ alex$

alexmarkley · 2020-10-15T00:29:10Z

Seems like the simctl issue is preventing flutter from detecting physical devices plugged into the machine. Flutter doctor reports "No devices available" and flutter run in a hello world project yields:

DTK:helloWorldThree alex$ flutter run
Error executing simctl: 1
arch: posix_spawnp: xcrun: Bad CPU type in executable


No supported devices connected.
DTK:helloWorldThree alex$

alexmarkley · 2020-10-15T00:33:22Z

@jmagman uh oh, this is tricky:

DTK:~ alex$ xcrun simctl --version
@(#)PROGRAM:simctl  PROJECT:CoreSimulator-732.18.0.2
DTK:~ alex$ /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl --version
@(#)PROGRAM:simctl  PROJECT:CoreSimulator-732.18.0.2
DTK:~ alex$ arch -arm64 /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl
arch: posix_spawnp: /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl: Bad CPU type in executable
DTK:~ alex$ lipo -detailed_info /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl
fatal error: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: can't figure out the architecture type of: /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl
DTK:~ alex$ file /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl
/Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl: Bourne-Again shell script text executable, ASCII text
DTK:~ alex$ 
DTK:~ alex$ 
DTK:~ alex$ head -n 5 /Applications/Xcode-beta.app/Contents/Developer/usr/bin/simctl
#!/bin/bash
DEVELOPER_USR_BIN_DIR=${0%/*}
DEVELOPER_USR_BIN_DIR=${DEVELOPER_USR_BIN_DIR%/local/bin}
DEVELOPER_USR_BIN_DIR=${DEVELOPER_USR_BIN_DIR%/bin}
DEVELOPER_USR_BIN_DIR=${DEVELOPER_USR_BIN_DIR}/bin
DTK:~ alex$

alexmarkley · 2020-10-15T00:41:19Z

Oh! @jmagman I think you want this:

DTK:~ alex$ arch -arm64e xcrun simctl --version
@(#)PROGRAM:simctl  PROJECT:CoreSimulator-732.18.0.2
DTK:~ alex$

As per this:

DTK:~ alex$ lipo -info /usr/bin/xcrun 
Architectures in the fat file: /usr/bin/xcrun are: x86_64 arm64e 
DTK:~ alex$

This leads to some interesting behavior like this (running inside of iTerm, which is running under Rosetta):

(And no, I have no idea why arch reports the running architecture as arm64 but expects you to pass arm64e when specifying a runtime architecture. 🤷‍♂️ )

jmagman · 2020-10-15T01:37:08Z

Hm, I was trying to figure out where arm64e was needed... on my dtk it only worked with -arm64 not -arm64e

On this commit flutter run works:

and flutter devices:

$ lipo -info /usr/bin/xcrun
Architectures in the fat file: /usr/bin/xcrun are: x86_64 arm64e
$ arch -arm64 xcrun simctl --version
@(#)PROGRAM:simctl  PROJECT:CoreSimulator-732.18
$ arch -arm64e xcrun simctl --version
arch: Unknown architecture: arm64e
$ arch -arm64 /Applications/Xcode-12.2.beta2.app/Contents/Developer/usr/bin/simctl --version
@(#)PROGRAM:simctl  PROJECT:CoreSimulator-732.18
$ arch -arm64e /Applications/Xcode-12.2.beta2.app/Contents/Developer/usr/bin/simctl --version
arch: Unknown architecture: arm64e

alexmarkley · 2020-10-15T01:54:48Z

@jmagman that is somewhat baffling. I am on Beta 10 as of this afternoon, I wonder if that could factor in somehow?

I assume Apple's intention in the long term is for the xcrun to spawn ARM processes because the caller is also an ARM process. I guess that's not going to be possible with Dart in the short term?

alexmarkley · 2020-10-15T02:19:32Z

@jmagman are you still running the image that it shipped with? https://betawiki.net/wiki/MacOS_Big_Sur_build_20A5299w

I noticed some pretty significant changes around Beta 7 and following, so maybe an update is in order. 😉

flutter-dashboard bot added tool Affects the "flutter" command-line tool. See also t: labels. work in progress; do not review labels Oct 13, 2020

google-cla bot added the cla: yes label Oct 13, 2020

jmagman force-pushed the xcodebuild-arch branch from 09d9954 to 99f7317 Compare October 13, 2020 21:29

jmagman commented Oct 13, 2020

View reviewed changes

packages/flutter_tools/test/general.shard/macos/xcode_test.dart Outdated

Copy link

Member Author

jmagman Oct 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is new.

jmagman commented Oct 13, 2020

View reviewed changes

jmagman force-pushed the xcodebuild-arch branch 4 times, most recently from 7964014 to 52203ef Compare October 14, 2020 01:50

jmagman removed the work in progress; do not review label Oct 14, 2020

jmagman requested review from christopherfujino and jonahwilliams October 14, 2020 18:44

jmagman marked this pull request as ready for review October 14, 2020 18:44

jmagman self-assigned this Oct 14, 2020

jmagman added the platform-mac Building on or for macOS specifically label Oct 14, 2020

christopherfujino approved these changes Oct 14, 2020

View reviewed changes

jonahwilliams approved these changes Oct 14, 2020

View reviewed changes

jmagman added 3 commits October 14, 2020 13:19

Run Xcode command lines tools in native ARM

2e79af1

Fix tests

693e637

Add doc

097cd4c

jmagman force-pushed the xcodebuild-arch branch from fefd7ae to 097cd4c Compare October 14, 2020 20:19

jmagman merged commit a82807d into flutter:master Oct 14, 2020

jmagman deleted the xcodebuild-arch branch October 14, 2020 22:44

This was referenced Oct 15, 2020

Upgrade flutter-devicelab-mac-15 to latest Big Sur beta #68167

Closed

"flutter build ios" fails on ARM macOS, ibtoold crashes compiling xcassets/storyboards #65133

Closed

jmagman mentioned this pull request Oct 23, 2020

Apple silicon arch -arm64 to -arm64e #68855

Merged

jmagman added a commit to jmagman/flutter that referenced this pull request Oct 28, 2020

Run Xcode command lines tools in native ARM (flutter#68050)

6ee3b98

jmagman mentioned this pull request Oct 28, 2020

Support targeting physical iOS devices on Apple Silicon #69212

Closed

jmagman mentioned this pull request Nov 5, 2020

Run more xcodebuild commands in native arm on Apple Silicon #69837

Merged

jmagman added the platform-host-arm Building on an ARM-based platform label Aug 20, 2021

Run Xcode command lines tools in native ARM #68050

Run Xcode command lines tools in native ARM #68050

Uh oh!

Conversation

jmagman commented Oct 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Tests

Breaking Change

Uh oh!

jmagman Oct 13, 2020

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 13, 2020

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 13, 2020

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 13, 2020

Choose a reason for hiding this comment

Uh oh!

christopherfujino left a comment

Choose a reason for hiding this comment

Uh oh!

jonahwilliams left a comment

Choose a reason for hiding this comment

Uh oh!

jonahwilliams Oct 14, 2020

Choose a reason for hiding this comment

Uh oh!

jmagman Oct 14, 2020

Choose a reason for hiding this comment

Uh oh!

jmagman commented Oct 14, 2020

Uh oh!

alexmarkley commented Oct 14, 2020

Uh oh!

alexmarkley commented Oct 15, 2020

Uh oh!

alexmarkley commented Oct 15, 2020

Uh oh!

alexmarkley commented Oct 15, 2020

Uh oh!

alexmarkley commented Oct 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jmagman commented Oct 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexmarkley commented Oct 15, 2020

Uh oh!

alexmarkley commented Oct 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jmagman commented Oct 13, 2020 •

edited

Loading

alexmarkley commented Oct 15, 2020 •

edited

Loading

jmagman commented Oct 15, 2020 •

edited

Loading