Move embedding and linking Flutter frameworks into the tool #51453

jmagman · 2020-02-26T06:08:00Z

Description

Xcode 11.4 changed its behavior to require linked and embedded frameworks (App.framework and Flutter.framework) to have the right architecture BEFORE the Flutter script can run. So building against the simulator would work, then the next time building against a real device it would just fail:

Linked and embedded framework 'App.framework' was built for iOS Simulator

and vice versa from real device to simulator:

Linked and embedded framework 'App.framework' was built for iOS

Observations

App.framework doesn't need to be linked since there are no symbols.
It would be better to push the linker and embedding logic down in the tool so we have more control. For example, we could distribute a Flutter.xcframework and link on that without a breaking change. Or name App.framework uniquely to support different add-to-app modules. Or we could move more things from xcode_backend into the tool. Or we could update search paths to point directly to the right build mode engine artifacts directory without copying Flutter.framework into the project.
We keep needing to change the Xcode project file template (Disable CocoaPods input and output paths in Xcode build phase and adopt new Xcode build system #33684, so many more examples).
Ideally this migration could be versioned, see Flutter migrate #40460. However this migration looks safe: there are unique identifiers!
Since it's not versioned, the migrator will run on every flutter build ios/flutter run! However these migrations are really fast when there's no changes to make.

Changes

Remove App.framework and Flutter.framework link step in template build phase.
Add OTHER_LDFLAGS=$(inherited) -framework Flutter to Generated.xcconfig and remove Flutter.framework from linking build phase (no need add for App since there are no symbols).
Remove App.framework and Flutter.framework from the framework embedding build phase.
Add migrators to do this automatically based on the Xcode identifiers. I confirmed these are the original template identifiers (first Flutter.framework was Finish integrating HelloServices model on iOS #4820 and App.framework was Build Flutter app as a framework on iOS #8971).
Add error message and temporary workaround instructions in case the migration failed:

 Your Xcode project requires migration. See https://github.com/flutter/flutter/issues/50568 for details.
 
 You can temporarily work around this issue by running:
  rm -rf ios/Flutter/App.framework

Run the SHARD=build_tests and let all the examples migrate.

Related Issues

Fixes #50568
Fixes #32454

Tests

ios_project_migration_test, updated xcodeproj_test

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
I signed the [CLA].
I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
I read the [Tree Hygiene] wiki page, which explains my responsibilities.
I updated/added relevant documentation (doc comments with ///).
All existing and new tests are passing.
The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
I am willing to follow-up on review comments in a timely manner.

Breaking Change

No, no existing tests failed, so this is not a breaking change.
Yes, this is a breaking change. If not, delete the remainder of this section.

jmagman · 2020-02-26T06:08:41Z

packages/flutter_tools/bin/xcode_backend.sh

EXPANDED_CODE_SIGN_IDENTITY is more correct, it's the resolved underlying ID as opposed to a name (on my machine the name was ambiguous)

jmagman · 2020-02-26T22:03:45Z

packages/flutter_tools/lib/src/ios/mac.dart

This link is temporary. I will write wiki or website instructions and update this link: flutter/website#3741

jonahwilliams

Follow up questions:

Did you use the flutter tool to migrate the examples in repo?
Do we need to make a similar change to the macOS project structure? If so, that doesn't need to be done now but it would be nice to have an issue for it.
Is there an issue filled for adding this information to the web site?

Re: migration time - I think this is okay provided that we're not overwriting the file if it isn't necessary. It doesn't look like this is the case in your code.

jonahwilliams · 2020-02-26T23:06:32Z

packages/flutter_tools/lib/src/ios/ios_project_migration.dart

Since this is a private member with a private typedef, I would instead do the function type inline:

void _processFileLines(File file, String Function(String) processLine)

jonahwilliams · 2020-02-26T23:14:50Z

packages/flutter_tools/lib/src/ios/ios_project_migration.dart

consider a better name, like migrateProject et cetera

jonahwilliams · 2020-02-26T23:15:29Z

packages/flutter_tools/lib/src/ios/ios_project_migration.dart

I would add some contextual top level information about why this class exists

jmagman · 2020-02-26T23:38:31Z

Follow up questions:

Did you use the flutter tool to migrate the examples in repo?

Yes, all examples and tests were migrated by running flutter build ios in the right directory (or letting integration tests do it for me).

Do we need to make a similar change to the macOS project structure? If so, that doesn't need to be done now but it would be nice to have an issue for it.

I think it may be a good idea to do the same pattern for macOS for the future-proofing xcframework / App.framework rename reasons I mentioned in the "Observations" section, and to remove the disable_input_output_paths hack in the macOS Podfile. However there's no immediate need to do this since Mac won't hit #50568 (there is no simulator, frameworks always have the x86 architectures, no thinning happens). I would defer to @stuartmorgan on the necessity and/or timing of that change.

Is there an issue filled for adding this information to the web site?

flutter/website#3741

Re: migration time - I think this is okay provided that we're not overwriting the file if it isn't necessary. It doesn't look like this is the case in your code.

No, it doesn't overwrite the file unless there's a change. See the "skipped if nothing to upgrade" ios_project_migration_test unit test.

jonahwilliams · 2020-02-27T00:02:12Z

Even if its not necessary, it would be nice to keep the macOS and iOS projects as similar as possible.

jmagman · 2020-02-27T01:17:52Z

Oh another thing: this migration will prevent users from downgrading Flutter versions to before this change since OTHER_LDFLAGS won't exist in past Generated.xcconfigs... I'm not quite sure how to help with that...

I could change the script build phase to first run embed, then run thin, since that's all the new embed_and_thin is doing. embed_and_thin doesn't exist on past Flutter versions, but embed and thin do, respectively.

jonahwilliams · 2020-02-27T02:01:09Z

I would recommend that if you modify the project files, to take the old sources and save them as "*.old". Then the documentation on the website/wiki can explain how to restore the project by renaming them

stuartmorgan-g

It terms of the general approach to managing the framework, this looks good to me! And capturing from offline discussion, it has the added bonus that it should make it pretty easy to solve the debug/profile/release problem with assemble that @jonahwilliams had encountered previously, since we won't be bound by Xcode's limitation of needing a single known path to the Flutter framework.

My comments are actually about the specifics of we do the migration, rather than the end result.

stuartmorgan-g · 2020-02-28T00:30:45Z

examples/platform_view/ios/Runner.xcodeproj/project.pbxproj

Why do only some of the projects have these input/output path changes?

CocoaPods adds this in their scripting, so it's present in apps that have iOS platform code in their plugins.

(so projects that have Podfiles)

stuartmorgan-g · 2020-02-28T00:38:32Z

packages/flutter_tools/lib/src/ios/ios_project_migration.dart

Breadcrumb for later reference: I think we can use this to fix #39659

I would love to move most of that logic into a Ruby helper script in the project (like xcode_backend) so it (almost) never has to be upgraded again #45197

stuartmorgan-g · 2020-02-28T00:44:33Z

packages/flutter_tools/lib/src/ios/ios_project_migration.dart

I'm looking at this class and thinking about what happens when we want to add a second migration for something else later; it seems like it'll become very hard to maintain/understand pretty quickly.

I think it would be helpful to design this up front so that instead of a single processor, it uses an array of processors. That way we can encapsulate each conceptual migration and keep them all from bleeding together over time. It would also mean we could run individual ones selectively (see later comment). I don't think it would add too much complexity, and it would make the next migration much easier to add in a maintainable way (vs us always deciding that it's not bad enough to refactor this time, but maybe next time...)

stuartmorgan-g · 2020-02-28T00:46:47Z

packages/flutter_tools/lib/src/ios/mac.dart

Couldn't we pretty easily use the .metadata file to keep track of migrations that we've done (say, with a UUID for each completed migration)?

In addition to efficiency over time, it also means that we won't accidentally fight users. E.g., if someone had their own unrelated reasons that they needed disable_input_output_paths in their podspec, they'd have a really bad time if we re-run on every build.

I really wish we could wait until https://docs.google.com/document/d/1fPnpUI6gXE9PkNWuFoAW20m_sQ87ZSC1rrn_D7fxBvs/edit?ts=5e58625e#heading=h.pub7jnop54q0 was in place and use that instead of having a bunch of one-off migrators fighting each other... but we can't wait in this case.

Since I removed the disable_input_output_paths change, I didn't implement migration versioning, but it really is a good idea. I'd like to split it out into it's own change as part of the wider flutter migrate effort. For now it's running on every run like #13011.

jmagman added platform-ios iOS applications specifically tool Affects the "flutter" command-line tool. See also t: labels. work in progress; do not review t: xcode "xcodebuild" on iOS and general Xcode project management labels Feb 26, 2020

jmagman self-assigned this Feb 26, 2020

fluttergithubbot added d: examples Sample code and demos c: contributor-productivity Team-specific productivity, code health, technical debt. customer: gallery Relating to flutter/gallery repository. Please transfer non-framework issues there. labels Feb 26, 2020

googlebot added the cla: yes label Feb 26, 2020

jmagman commented Feb 26, 2020

View reviewed changes

jmagman force-pushed the linker branch from 29eb82e to 5a65634 Compare February 26, 2020 06:09

This comment has been minimized.

Sign in to view

jmagman force-pushed the linker branch from 1c29cf0 to ad0a214 Compare February 26, 2020 21:30

jmagman commented Feb 26, 2020

View reviewed changes

jmagman requested review from cbracken and jonahwilliams February 26, 2020 22:04

jmagman removed the work in progress; do not review label Feb 26, 2020

jonahwilliams reviewed Feb 26, 2020

View reviewed changes

jmagman requested a review from stuartmorgan-g February 26, 2020 23:30

jmagman mentioned this pull request Feb 26, 2020

Document migration steps for migrating Xcode project to not explicitly link or embed on App.framework and Flutter.framework flutter/website#3741

Closed

jmagman mentioned this pull request Feb 27, 2020

Clean App.framework, Generated.xcconfig, flutter_export_environment.sh #51532

Merged

10 tasks

stuartmorgan-g reviewed Feb 28, 2020

View reviewed changes

jmagman added 3 commits February 27, 2020 17:17

Move embedding and linking Flutter frameworks into the tool

e70ed52

Unused import

f56619d

Migrate

f9bf57f

jmagman mentioned this pull request Mar 9, 2020

Failed to find assets path for "Frameworks/App.framework/flutter_assets", App.framework missing from host app Frameworks directory #33121

Closed

dpedrinha mentioned this pull request Mar 10, 2020

Failed to find assets path for "Frameworks/App.framework/flutter_assets" #52084

Closed

This was referenced Mar 11, 2020

Flutter migrate #40460

Closed

[cloud_firestore] Xcode build extremely slow firebase/flutterfire#349

Closed

[App.framework] Linked and embedded framework 'App.framework' was built for iOS/iOS Simulator #50568

Closed

This was referenced Mar 30, 2020

flutter build ios failing: ld: framework not found App #32454

Closed

Add option to avoid pod install during "flutter build ios" #53781

Open

mattetti mentioned this pull request Apr 12, 2020

App.framework bundle nil from plugin #54586

Closed

This was referenced Apr 13, 2020

Migrate Runner project base configuration #54691

Merged

Flutter framework linker flag should only be set for iOS SDKs #54769

Closed

hpoul mentioned this pull request Apr 30, 2020

[package_info] add support for macos to package_info plugin flutter/plugins#2618

Merged

13 tasks

jonahwilliams mentioned this pull request May 7, 2020

macOS xcode project should move Framework linking to tool #56581

Closed

This was referenced May 8, 2020

iOS build fails "No code signature found" when multiple code signing identities have the same name #53891

Closed

"Path does not exist" building for iOS [Crash report] #56507

Closed

jmagman mentioned this pull request May 20, 2020

Allow FLUTTER_APPLICATION_PATH to be null for misconfigured Xcode projects #57701

Merged

10 tasks

jmagman mentioned this pull request Jun 1, 2020

Remove outdated disable_input_output_paths from example project Podfiles #58444

Merged

10 tasks

jmagman mentioned this pull request Jun 23, 2020

Distribute engine as Flutter.xcframework #60109

Closed

3 tasks

jmagman mentioned this pull request Jul 8, 2020

iOS build error: The domain/default pair of (/Runner.app/Frameworks/App.framework/flutter_assets/ios/Flutter/App.framework/Info.plist, CFBundleExecutable) does not exist #57210

Closed

jmagman mentioned this pull request Aug 10, 2020

Fix App.framework path in Podfile #63412

Merged

10 tasks

This was referenced Nov 2, 2020

Build App.framework directly to build directory #69612

Merged

Build App.framework directly to build directory #69699

Merged

jmagman mentioned this pull request Nov 14, 2020

The following build commands failed: app/build/ios/Debug-iphonesimulator/Runner.app/Runner normal x86_64 #43421

Closed

jmagman mentioned this pull request Dec 15, 2020

Reland Move embedding and linking macOS Flutter frameworks into the tool #72372

Merged

jmagman mentioned this pull request Apr 20, 2021

ld: can't map file, errno=22 file 'Flutter' for architecture arm64 #52736

Closed

jmagman mentioned this pull request Jun 24, 2021

Build all iOS example apps on current Flutter stable flutter/plugins#4101

Merged

11 tasks

github-actions bot locked as resolved and limited conversation to collaborators Aug 1, 2021

Move embedding and linking Flutter frameworks into the tool #51453

Move embedding and linking Flutter frameworks into the tool #51453

Uh oh!

Conversation

jmagman commented Feb 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Observations

Changes

Related Issues

Tests

Checklist

Breaking Change

Uh oh!

jmagman Feb 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

jmagman Feb 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jonahwilliams left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmagman commented Feb 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonahwilliams commented Feb 27, 2020

Uh oh!

jmagman commented Feb 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonahwilliams commented Feb 27, 2020

Uh oh!

stuartmorgan-g left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmagman Feb 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

jmagman commented Feb 26, 2020 •

edited

Loading

jmagman Feb 26, 2020 •

edited

Loading

jmagman Feb 26, 2020 •

edited

Loading

jmagman commented Feb 26, 2020 •

edited

Loading

jmagman commented Feb 27, 2020 •

edited

Loading

jmagman Feb 28, 2020 •

edited

Loading