-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Support conditional bundling of assets based on --flavor
#132985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support conditional bundling of assets based on --flavor
#132985
Conversation
9b4080b to
2537ea3
Compare
8854b10 to
589ef1b
Compare
04d8c5f to
5eb34d1
Compare
dc2a9f8 to
d1194c1
Compare
9e7441d to
30769b1
Compare
862fb99 to
793eab6
Compare
|
|
||
| /// The platform-independent URL provided by the user in the pubspec that this | ||
| /// asset was found from. | ||
| final Uri originUri; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this being used for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See #132985 (comment). In short, it's to help disambiguate in scenarios where multiple declarations affect the same asset:
assets:
- path: assets/paid/
flavor: paid
- path: assets/paid/pip.mp3 # For pip.mp3, this declaration overrides the previous one.
flavor: freeThis could probably just be a ToolExit instead though.
| if (flavor != null && flavor != preExistingAsset.flavor ) { | ||
| if (assetUri.path.endsWith('/') == preExistingAsset.originUri.path.endsWith('/')) { | ||
| throwToolExit('Asset "${assetUri.path}" belongs to multiple flavors.\n' | ||
| 'An entry with the path "${preExistingAsset.originUri.path}" gives it a flavor of "${preExistingAsset.flavor}".\n' | ||
| 'An entry with the path "${assetUri.path}" gives it a flavor of "$flavor".' | ||
| ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you ellaborate a bit on what this error state is? Its an error in your implementation to have a flavor specific asset with more than one flavor by listing it twice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've tried to capture the intended behavior in the tests. In short, the answer is, yes, this implementation doesn't support multiple flavors for one asset. However, we could add support for that by, for example, changing the manifest syntax to accept a flavors list rather than a flavor string.
The error logic you pointed out is certainly complicated. To better understand where it came from, start by considering this case:
assets:
- assets/
- path: assets/free.mp3
flavor: freeTo me, it is intuitive to bundle free.mp3 if and only if --flavor is set to free. Following that logic, my first intuition was that any asset covered by an entry with a flavor is globally limited to that flavor. Therefore, two declarations that list the same asset but with different flavors results in an ambiguous manifest1:
assets:
- path: assets/pip.mp3
flavor: free
- path: assets/pip.mp3
flavor: paid
# asset/pip.mp3 can't be limited *only* to "free" and also *only* to "paid"To finally circle back to the weird error logic you highlighted, consider this kind of case (which is hopefully extremely rare):
assets:
- path: assets/paid/
flavor: paid
- path: assets/paid/pip.mp3
flavor: freeHere, for pip.mp3, the second entry takes precedence over the first since it is more specific. However, in hindsight, I think it's okay to make the code simpler by having this also result in a ToolExit, because I don't expect many—if any—users would specify something like this.
One alternative to all of this is reconsidering one of my old ideas where flavor-specific assets aren't listed in the asset manifest but rather within separate flavor manifests:
assets:
- assets/
# ...
flavors:
- name: free
additional-assets: # Assets can only be added, not removed
- assets/free/Still, if we add other conditional logic to assets in the future (e.g. platform-specificity), we'll still run into the same kind of problem. What if I have an asset that is specific to a combination of platform and flavor?
Footnotes
-
Now that you point it out though, it's possible some users may intuit that it's possible to give an asset multiple flavors by listing it multiple times with different flavors (I'm guessing this would also result in a simpler implementation). ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting! As a side note, these sort of comments are absolute 🥇 so consider adding these as documentation in the code.
I don't remember, but I suspect what happens today is that when you list the same asset multiple times, or list a directory and then a specific asset, that it gets ignored? And I suppose this probably works fine since we only need a signal as to whether or not the asset is included - there is no additional metadata.
But now with flavor specific assets, we do have additional data. I think for the first version I like this approach - lets keep it simple and define this as an error case, we can always weaken it later. The alternative is having rules for resolving confilicts which sounds like it would only be more complicated.
Consider pulling this code into its own method with your comment as a docstring.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't remember, but I suspect what happens today is that when you list the same asset multiple times, or list a directory and then a specific asset, that it gets ignored?
IIRC, redundant entries will get processed, but (as you alluded to) it's essentially a no-op (we just overwrite entries in the output manifest with identical data).
But now with flavor specific assets, we do have additional data. I think for the first version I like this approach - lets keep it simple and define this as an error case, we can always weaken it later.
I assume you are referring to this case (and implicitly the one just before it):
assets:
- path: assets/paid/
flavor: paid
- path: assets/paid/pip.mp3
flavor: freeIf so, yeah, I think it makes sense to just have this be an error case for now. Initially, I thought this case might be more common if/when we support glob/regex declarations, but even then I don't think any significant number of users would need conflict-resolving logic.
Consider pulling this code into its own method with your comment as a docstring.
Sure, I'll work on that. I'm glad you enjoyed my writing :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything here SGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did some more pondering on this (cheers to @jonahwilliams for chatting with me on this). To recap, the central question is "what should we do when multiple asset entries without identical1 conditions affect the same file?" Should conditions from separate entries be considered disjunctively (with "or" logic) or conjunctively (with "and" logic)? Should it depend on the specificity of those entries (i.e. file vs directory)? I don't believe there are obvious answers to these questions, but we don't need answers right now—if at all. We can instead throw a ToolExit when such ambiguities arise, and if users later overwhelmingly agree that such conditions should be combined in a specific way, we can change the behavior later without breaking anyone. In the meantime, the users can avoid these ToolExits by writing more explicit assets manifests and/or organizing assets into individual directories2.
To be clear, all the example assets manifests I provided previously in this thread (including the first one) would result in a ToolExit being thrown. Here's a specific example to focus on:
assets:
- assets/
- path: assets/free.mp3
flavor: freeAssuming the file assets/free.mp3 exists, this would result in a ToolExit. The message would explain that the tool could not determine whether or not to bundle assets/free.mp3 and where the ambiguity the came from, along with a vague—but still useful—hint as to how to remove the ambiguity.
Footnotes
| bool operator ==(Object other) { | ||
| if (other is! AssetsEntry) { | ||
| return false; | ||
| } | ||
|
|
||
| if (uri != other.uri) { | ||
| return false; | ||
| } | ||
|
|
||
| return true; | ||
| } | ||
|
|
||
| @override | ||
| int get hashCode => uri.hashCode; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it correct that this ignores flavor?
This comment was marked as off-topic.
This comment was marked as off-topic.
e0a530b to
0fbad0e
Compare
jonahwilliams
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change seems ready overall, what else needs to be done?
| return; | ||
| } | ||
|
|
||
| String errorMessage = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: use a StringBuffer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked with #hackers-dart, and I learned that the Dart compiler does not automatically optimize this in. I've replaced errorMessage with a StringBuffer in c494656.
(IIRC OpenJDK and maybe Rosyln have such optimizations, but that's years old memory and I could be very wrong)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
interesting discussion on this: dart-lang/sdk#50304 (comment)
| return uri == null ? (null, error) : (AssetsEntry(uri: uri), null); | ||
| } else if (yaml is Map) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you return you don't need to do an else:
if (yaml is String) {
...
return ..;
}
if (yaml is Map) {
...
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in nits
|
|
||
| if (uri != other.uri || flavor != other.flavor) { | ||
| return false; | ||
| } | ||
|
|
||
| return true; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if (uri != other.uri || flavor != other.flavor) { | |
| return false; | |
| } | |
| return true; | |
| } | |
| return uri == other.uri && flavor == other.flavor; | |
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in nits
| ReassembleHelper reassembleHelper = _defaultReassembleHelper, | ||
| NativeAssetsBuildRunner? buildRunner, | ||
| required Analytics analytics, | ||
| this.flavor, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider putting flavor on debugging options instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reminded me that the flavor is already available in debuggingOptions.buildInfo, so we can just reference that. Addressed in remove redundant flavor arg from HotRunner
I'm changing |
|
@andrewkolos Is there any example app to build and try? |
Sure, you can use this project with If I recall correctly, this app is configured to work with flavors on iOS, macOS, and Android. Keep in mind this feature is not available on the |
Oh, I confused the master branch with the stable one. I was using stable. That's why it wasn't working. Is there any prediction for when this feature will be available in the stable version? |
|
It's finally on the stable branch! Great work, congratulations! |
|
Can I handle translated strings the same way? My app uses translated strings this way: https://docs.flutter.dev/ui/accessibility-and-internationalization/internationalization and I need to have separate .arb files for each flavor. But .arb are not resources, is there any way to do that? |
|
I think it's crazy to create translations by flavor; for example, 2 flavors and 3 languages results in 6 generally large files to maintain. This way if you don't use translations, the benefit will be because there will only be 1 file, regardless of the number of flavors |
|
Hi! Great work!! Is this feature present at the moment? In the Flutter 3.19.0 changelog I see that this commit has been reverted with #139787 but I am not sure why 🤔 EDIT: @christopherfujino according to that PR you reverted the commit, but I couldn't find a reason why in that PR. Do you know why that decision was made? |
|
Indeed.. it's reverted, why? @andrewkolos |
|
I think the feature was relanded a few days after the revert. See #139834. I just tested the feature locally using the flutter stable branch and it works. Is this not working in 3.19.0? |
|
I haven't tried yet, my Flutter version is 3.19.2 stable. Regarding to revert, I'm asking to confirm if this feature is not removed from some reason |
|
Works for me with Flutter 3.19.4. My .ipa is 46.5MB, .aab is 91.2MB. Android should have additional 41.2MB of assets. Seems about right. |
|
hello, I want to know if this feature supports platform specific? |
|
I think your case it not related to flavors. Flavors only exists on mobile platforms. If you don't want to pack files into apk or ipa, don't use them as resources, put them to some external directory instead |
I saw @andrewkolos wrote something about platform. And the comment above just said he used this feature and solved his problem like platform-specific assets. and can you share more information about the |
|
|
|
I see. I think I know how to solve my problem. thanks. |
… cached asset bundle (#150461) Fixes #150296 **Context.** `flutter test` has its own code path for writing flutter app [assets](https://docs.flutter.dev/ui/assets/assets-and-images). #132985 introduced [flavor-conditional asset bundling ](https://docs.flutter.dev/deployment/flavors#conditionally-bundling-assets-based-on-flavor), which lets users control which assets get bundled based on `--flavor`. `--flavor` is supported in `flutter test`. **Bug and fix.** `--flavor` isn't considered when deciding whether we need to rebuild this asset bundle: https://github.com/flutter/flutter/blob/5e448f4ce57723ac0792ae822ebac69df3188ba1/packages/flutter_tools/lib/src/commands/test.dart#L709 This PR address this by writing the value of `--flavor` to a file in the build directory and checking that when validating the cached asset bundle.
… cached asset bundle (flutter#150461) Fixes flutter#150296 **Context.** `flutter test` has its own code path for writing flutter app [assets](https://docs.flutter.dev/ui/assets/assets-and-images). flutter#132985 introduced [flavor-conditional asset bundling ](https://docs.flutter.dev/deployment/flavors#conditionally-bundling-assets-based-on-flavor), which lets users control which assets get bundled based on `--flavor`. `--flavor` is supported in `flutter test`. **Bug and fix.** `--flavor` isn't considered when deciding whether we need to rebuild this asset bundle: https://github.com/flutter/flutter/blob/5e448f4ce57723ac0792ae822ebac69df3188ba1/packages/flutter_tools/lib/src/commands/test.dart#L709 This PR address this by writing the value of `--flavor` to a file in the build directory and checking that when validating the cached asset bundle.
… cached asset bundle (flutter#150461) Fixes flutter#150296 **Context.** `flutter test` has its own code path for writing flutter app [assets](https://docs.flutter.dev/ui/assets/assets-and-images). flutter#132985 introduced [flavor-conditional asset bundling ](https://docs.flutter.dev/deployment/flavors#conditionally-bundling-assets-based-on-flavor), which lets users control which assets get bundled based on `--flavor`. `--flavor` is supported in `flutter test`. **Bug and fix.** `--flavor` isn't considered when deciding whether we need to rebuild this asset bundle: https://github.com/flutter/flutter/blob/5e448f4ce57723ac0792ae822ebac69df3188ba1/packages/flutter_tools/lib/src/commands/test.dart#L709 This PR address this by writing the value of `--flavor` to a file in the build directory and checking that when validating the cached asset bundle.
|
I have different environments even on Web and overcome the missing flavor issue by just providing different main.dart entrypoints. Still an issue is that all my assets are loaded in Flutter Web even I don't need them in my build for that specific environment. Are there any solutions to this? |
|
@LoadJulz That would be a good question to ask as a new issue. Generally that's a more effective venue for discussion than on the thread of an old merged PR. (You can always link to the old PR when it's relevant context.) |
Provides support for conditional bundling of assets through the existing
--flavoroption forflutter buildandflutter run. Closes #21682. Resolves #136092Change
Within the
assetssection pubspec.yaml, the user can now specify one or moreflavorsthat an asset belongs to. Consider this example:With this pubspec,
flutter run --flavor vanillawill not includeassets/strawberry/ice-cream.pngin the build output.flutter run --flavor strawberrywill not includeassets/vanilla/ice-cream.png.flutter runwill only includeassets/normal-asset.png.Open questions
--flavorsupport (Android, iOS, and (implicitly) MacOS)? This PR currently only enables this feature for officially supported platforms.Design thoughts, what this PR does not do, etc.
This does not provide an automatic mapping/resolution of asset keys/paths to others based on flavor at runtime.
The implementation in this PR represents a simplest approach. Notably, it does not give Flutter the ability to dynamically choose an asset based on flavor using a single asset key. For example, one can't use
Image.asset('config.json')to dynamically choose between different "flavors" ofconfig.json(such asdev-flavor/config.jsonorprod-flavor/config.json). However, a user could always implement such a mechanism in their project or in a library by examining the flavor at runtime.When multiple entries affect the same file and 1) at least one of these entries have a
flavorslist provided and 2) these lists are not equivalent, we always consider the manifest to be ambiguous and will throw aToolExit.Details
For example, these manifests would all be considered ambiguous:See this review comment thread for the full story on how I arrived at this decision.
This does not support Android's multidimensional flavors feature (in an intuitive way)
Details
Conder this excerpt from a Flutter project's android/app/build.gradle file:
android { // ... flavorDimensions "mode", "api" productFlavors { free { dimension "mode" applicationIdSuffix ".free" } premium { dimension "mode" applicationIdSuffix ".premium" } minApi23 { dimension "api" versionNameSuffix "-minApi23" } minApi21 { dimension "api" versionNameSuffix "-minApi21" } } }In this setup, the following values are valid
--flavorare validfreeMinApi21,freeMinApi23,premiumMinApi21, andpremiumMinApi23. We call these values "flavor combinations". Consider the following from the Android documentation1:This feature will not behave in this way. If a user utilizes this feature and also Android's multidimensional flavors feature, they will have to list out all flavor combinations that contain the flavor they want to limit an asset to:
This is mostly due to a technical limitation in the hot-reload feature of
flutter run. During a hot reload, the tool will try to update the asset bundle on the device, but the tool does not know the flavors contained within the flavor combination (that the user passes to--flavor). Gradle is the source of truth of what flavors were involved in the build, andflutter runcurrently does not access to that information since it's an implementation detail of the build process. We could bubble up this information, but it would require a nontrivial amount of engineering work, and it's unclear how desired this functionality is. It might not be worth implementing.See https://flutter.dev/go/flavor-specific-assets for the (outdated) design document.
///).Footnotes
https://developer.android.com/build/build-variants#flavor-dimensions ↩