-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Improve build time when using SwiftPM #150052
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
auto-submit
merged 3 commits into
flutter:master
from
vashworth:improve_swiftpm_flutter_framework_caching
Jun 12, 2024
Merged
Improve build time when using SwiftPM #150052
auto-submit
merged 3 commits into
flutter:master
from
vashworth:improve_swiftpm_flutter_framework_caching
Jun 12, 2024
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jmagman
approved these changes
Jun 12, 2024
| void prepare() { | ||
| // The "prepare" command runs in a pre-action script, which also runs when | ||
| // using the Xcode/xcodebuild clean command. Skip if cleaning. | ||
| if (environment['ACTION'] == 'clean') { |
Member
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh good call...
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 13, 2024
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 13, 2024
auto-submit bot
pushed a commit
to flutter/packages
that referenced
this pull request
Jun 13, 2024
flutter/flutter@b1f9d71...01db23b 2024-06-13 [email protected] Roll Flutter Engine from c7fcbfce608f to 4cb3025d3abf (28 revisions) (flutter/flutter#150199) 2024-06-13 [email protected] Revert "[CupertinoActionSheet] Add sliding tap gesture" (flutter/flutter#150147) 2024-06-13 [email protected] RawScrollbar: don't listen for drag gestures when scrolling is not possible (flutter/flutter#149925) 2024-06-13 [email protected] Update testowners (flutter/flutter#150141) 2024-06-12 [email protected] Revert "Reland 2: [CupertinoActionSheet] Match colors to native" (flutter/flutter#150142) 2024-06-12 [email protected] Reland 2: [CupertinoActionSheet] Match colors to native (flutter/flutter#150129) 2024-06-12 49699333+dependabot[bot]@users.noreply.github.com Bump actions/checkout from 4.1.6 to 4.1.7 (flutter/flutter#150132) 2024-06-12 49699333+dependabot[bot]@users.noreply.github.com Bump github/codeql-action from 3.25.8 to 3.25.9 (flutter/flutter#150133) 2024-06-12 [email protected] Improve build time when using SwiftPM (flutter/flutter#150052) 2024-06-12 [email protected] Reland: Request focus if accessibility focus is given to a Focus widget (#142942) (flutter/flutter#149840) 2024-06-12 [email protected] Update WidgetStatesController docs (flutter/flutter#150081) 2024-06-12 [email protected] [Reland] Fix `SegmentedButton` clipping when drawing segments (#149739) (flutter/flutter#150090) 2024-06-12 [email protected] Fix markdown hyperlinks in the style guide (flutter/flutter#150071) 2024-06-12 [email protected] Update packages desktop PR triage link lables (flutter/flutter#150124) 2024-06-12 [email protected] Add mouse cursor property to `CupertinoRadio` (flutter/flutter#149681) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
victorsanni
pushed a commit
to victorsanni/flutter
that referenced
this pull request
Jun 14, 2024
When using SwiftPM, we use `flutter assemble` in an Xcode Pre-action to run the `debug_unpack_macos` (or profile/release) target. This target is also later used in a Run Script build phase. Depending on `ARCHS` build setting, the Flutter/FlutterMacOS binary is thinned. In the Run Script build phase, `ARCHS` is filtered to the active arch. However, in the Pre-action it doesn't always filter to the active arch. As a workaround, assume arm64 if the [`NATIVE_ARCH`](https://developer.apple.com/documentation/xcode/build-settings-reference/#NATIVEARCH) is arm, otherwise assume x86_64. Also, this PR adds a define flag `PreBuildAction`, which gives the Pre-action a [unique configuration of defines](https://github.com/flutter/flutter/blob/fdb74fd3e7e08d99d8e19a0d7c548cf6cab56b31/packages/flutter_tools/lib/src/build_system/build_system.dart#L355-L372) and therefore a separate filecache from the Run Script build phase filecache. This improves caching so the Run Script build phase action doesn't find missing targets in the filecache. It also uses this flag to skip cleaning up the previous build files. Lastly, skip the Pre-action if the build command is `clean`. Note: For iOS, if [`CodesignIdentity`](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/bin/xcode_backend.dart#L470-L473) is set, the Pre-action and Run Script build phase will not match because the Pre-action does not send the `EXPANDED_CODE_SIGN_IDENTITY` and therefore will codesign it with [`-`](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/lib/src/build_system/targets/ios.dart#L695) instead. This will cause `debug_unpack_macos` to invalidate and rerun every time. A potential solution would be to move [codesigning out of the Run Script build phase](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/lib/src/build_system/targets/ios.dart#L299) to the [Thin Binary build phase](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/bin/xcode_backend.dart#L204-L257) after it's copied into the TARGET_BUILD_DIR, like we do with [macOS](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/bin/macos_assemble.sh#L179-L183).
victorsanni
pushed a commit
to victorsanni/flutter
that referenced
this pull request
Jun 14, 2024
When using SwiftPM, we use `flutter assemble` in an Xcode Pre-action to run the `debug_unpack_macos` (or profile/release) target. This target is also later used in a Run Script build phase. Depending on `ARCHS` build setting, the Flutter/FlutterMacOS binary is thinned. In the Run Script build phase, `ARCHS` is filtered to the active arch. However, in the Pre-action it doesn't always filter to the active arch. As a workaround, assume arm64 if the [`NATIVE_ARCH`](https://developer.apple.com/documentation/xcode/build-settings-reference/#NATIVEARCH) is arm, otherwise assume x86_64. Also, this PR adds a define flag `PreBuildAction`, which gives the Pre-action a [unique configuration of defines](https://github.com/flutter/flutter/blob/fdb74fd3e7e08d99d8e19a0d7c548cf6cab56b31/packages/flutter_tools/lib/src/build_system/build_system.dart#L355-L372) and therefore a separate filecache from the Run Script build phase filecache. This improves caching so the Run Script build phase action doesn't find missing targets in the filecache. It also uses this flag to skip cleaning up the previous build files. Lastly, skip the Pre-action if the build command is `clean`. Note: For iOS, if [`CodesignIdentity`](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/bin/xcode_backend.dart#L470-L473) is set, the Pre-action and Run Script build phase will not match because the Pre-action does not send the `EXPANDED_CODE_SIGN_IDENTITY` and therefore will codesign it with [`-`](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/lib/src/build_system/targets/ios.dart#L695) instead. This will cause `debug_unpack_macos` to invalidate and rerun every time. A potential solution would be to move [codesigning out of the Run Script build phase](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/lib/src/build_system/targets/ios.dart#L299) to the [Thin Binary build phase](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/bin/xcode_backend.dart#L204-L257) after it's copied into the TARGET_BUILD_DIR, like we do with [macOS](https://github.com/flutter/flutter/blob/14df7be3f9471a97f34e4601fb7710850373ac3b/packages/flutter_tools/bin/macos_assemble.sh#L179-L183).
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 6, 2024
This was referenced Sep 3, 2024
auto-submit bot
pushed a commit
that referenced
this pull request
Sep 11, 2024
#154645) ### Problem Enabling the Swift Package Manager feature caused post-submit tests to fail on Mac x64 hosts: <details> <summary>Example error...</summary> https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_ios%20rrect_blur_perf_ios__timeline_summary/575/overview ``` â�¦ ... flutter --verbose assemble ... -dIosArchs=x86_64 ... profile_unpack_ios Target profile_unpack_ios failed: Exception: Binary ... build/ios/Profile-iphoneos/Flutter.framework/Flutter does not contain x86_64. Running lipo -info: Non-fat file: ... build/ios/Profile-iphoneos/Flutter.framework/Flutter is architecture: arm64 #0 UnpackIOS._thinFramework (package:flutter_tools/src/build_system/targets/ios.dart:351:7) <asynchronous suspension> #1 UnpackIOS.build (package:flutter_tools/src/build_system/targets/ios.dart:298:5) <asynchronous suspension> ... ``` </details> ### Reproduction On a mac x64 host: 1. Switch to the latest master channel: `flutter channel master ; flutter upgrade` 1. Disable the Swift Package Manager feature: `flutter config --no-enable-swift-package-manager` 2. Create a Flutter project 2. [Edit the Xcode project manually to add the prepare pre-action](https://docs.flutter.dev/packages-and-plugins/swift-package-manager/for-app-developers#step-2-add-run-prepare-flutter-framework-script-pre-action) 3. Run `flutter run` (`flutter build ios` does not reproduce this issue). ### Background Previously, the Flutter framework was unpacked in the Xcode target's build. Unfortunately, this happens after Swift packages are built; this prevented Swift packages from using the Flutter framework. To fix this, we added an Xcode pre-action that unpacks the Flutter framework _before_ Swift Package Manager builds packages. The Xcode target still runs the Flutter framework unpack step, but this step no-ops if the unpack step has the exact same inputs. ```mermaid flowchart LR A[flutter run -d iphone] --> B(Build Xcode project) B --> C(Xcode 'prepare framework' pre-action) B --> G[Build Swift packages] B --> D(Build 'Runner' target) C --> E[Unpack Flutter framework #1] D --> F[" Unpack Flutter framework #2 (No-ops if inputs are same as #1) "] ``` #150052 added an optimization that made it more likely the second unpack step no-ops by fixing a case where the target architecture input could be different: > When using SwiftPM, we use `flutter assemble` in an Xcode Pre-action to run the `debug_unpack_macos` (or profile/release) target. This target is also later used in a Run Script build phase. Depending on `ARCHS` build setting, the Flutter/FlutterMacOS binary is thinned. In the Run Script build phase, `ARCHS` is filtered to the active arch. However, in the Pre-action it doesn't always filter to the active arch. As a workaround, assume arm64 if the [`NATIVE_ARCH`](https://developer.apple.com/documentation/xcode/build-settings-reference/#NATIVEARCH) is arm, otherwise assume x86_64. This optimization is only applied if [`ONLY_ACTIVE_ARCH`](https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/XcodeBuildSettingRef/1-Build_Setting_Reference/build_setting_ref.html#//apple_ref/doc/uid/TP40003931-CH3-SW157) is `YES`. > [!IMPORTANT] > [`ONLY_ACTIVE_ARCH`](https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/XcodeBuildSettingRef/1-Build_Setting_Reference/build_setting_ref.html#//apple_ref/doc/uid/TP40003931-CH3-SW157)'s name is misleading. It specifies whether the product includes only object code for the native architecture. > > A value of `YES` means the product includes only code for the native architecture ([NATIVE_ARCH](https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/XcodeBuildSettingRef/1-Build_Setting_Reference/build_setting_ref.html#//apple_ref/doc/uid/TP40003931-CH3-SW59)). > > A value of `NO` means the product includes code for the architectures specified in [ARCHS (Architectures)](https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/XcodeBuildSettingRef/1-Build_Setting_Reference/build_setting_ref.html#//apple_ref/doc/uid/TP40003931-CH3-SW62). ### Problem `buildXcodeProject` incorrectly always sets `ONLY_ACTIVE_ARCH` to `YES` if the Xcode built is for a single architecture: https://github.com/flutter/flutter/blob/6abef222514e8039bde5c47ab7864abbc4caf7e8/packages/flutter_tools/lib/src/ios/mac.dart#L353-L361 This is incorrect! If the host architecture is `x64` but the target architecture is `arm64`, [`ONLY_ACTIVE_ARCH`](https://developer.apple.com/library/archive/documentation/DeveloperTools/Reference/XcodeBuildSettingRef/1-Build_Setting_Reference/build_setting_ref.html#//apple_ref/doc/uid/TP40003931-CH3-SW157) should be `NO`. This caused the prepare pre-action to incorrectly use x64 as the target architecture for arm64 devices on an x64 host, which in turn caused builds to fail if Swift Package Manager was enabled. ### Solution This change updates `buildXcodeProject` to set `ONLY_ACTIVE_ARCH` correctly. This change also updates the prepare pre-action's to be more conservative in applying the optimization that filters the target architecture. This ensures that the build still works (but without the optimization) if `ONLY_ACTIVE_ARCH` is incorrectly set. Follow-up PR: #154649 This unblocks: #151567 ### DeviceLab test This problem reproduces if you `flutter run` to an iPhone Arm64 device from an x64 mac host with the Swift Package Manager feature enabled. I ran an affected DeviceLab test to verify the fix works as expected: Description | CI test | Result -- | -- | -- SwiftPM enabled without this fix: #154750 | [Link](https://ci.chromium.org/ui/p/flutter/builders/try.shadow/Mac_ios%20rrect_blur_perf_ios__timeline_summary/7/overview) | â�� SwiftPM enabled with this fix: #154749 | [Link](https://ci.chromium.org/ui/p/flutter/builders/try.shadow/Mac_ios%20rrect_blur_perf_ios__timeline_summary/8/overview) | â�
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
autosubmit
Merge PR when tree becomes green via auto submit App
tool
Affects the "flutter" command-line tool. See also t: labels.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using SwiftPM, we use
flutter assemblein an Xcode Pre-action to run thedebug_unpack_macos(or profile/release) target. This target is also later used in a Run Script build phase. Depending onARCHSbuild setting, the Flutter/FlutterMacOS binary is thinned. In the Run Script build phase,ARCHSis filtered to the active arch. However, in the Pre-action it doesn't always filter to the active arch. As a workaround, assume arm64 if theNATIVE_ARCHis arm, otherwise assume x86_64.Also, this PR adds a define flag
PreBuildAction, which gives the Pre-action a unique configuration of defines and therefore a separate filecache from the Run Script build phase filecache. This improves caching so the Run Script build phase action doesn't find missing targets in the filecache. It also uses this flag to skip cleaning up the previous build files.Lastly, skip the Pre-action if the build command is
clean.Note: For iOS, if
CodesignIdentityis set, the Pre-action and Run Script build phase will not match because the Pre-action does not send theEXPANDED_CODE_SIGN_IDENTITYand therefore will codesign it with-instead. This will causedebug_unpack_macosto invalidate and rerun every time. A potential solution would be to move codesigning out of the Run Script build phase to the Thin Binary build phase after it's copied into the TARGET_BUILD_DIR, like we do with macOS.Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.