[flutter_tools] Track asset transformer dependencies for hot reload#187947
Conversation
Allow asset transformers to register additional input dependencies for hot reload by passing a `--depfile` flag to the transformer subprocess and parsing the resulting Ninja-format depfile. Key changes: - `AssetTransformer` now passes `--depfile` and parses it using `DepfileService`, returning dependencies in a new `AssetTransformationResult`. - `DevelopmentAssetTransformer` tracks these dependencies and provides them to `DevFS`. - `DevFS.updateBundle` does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web). - `DevFS.updateBundle` checks if any tracked dependencies are invalidated and triggers re-transformation. - `DevFS.update` and `WebDevFS.update` append collected dependencies to the watched `sources` set. - `copyAssets` build target collects and adds these dependencies to its inputs to ensure correct rebuilds. - Added unit and integration tests to verify the behavior. Fixes flutter#187051
There was a problem hiding this comment.
Code Review
This pull request updates the asset transformation process to track and monitor dependencies. The AssetTransformer now parses a generated depfile and returns an AssetTransformationResult containing these dependencies. DevFS and WebDevFS use this information to trigger asset re-transformations when dependencies change. Feedback was provided to ensure relative paths in the depfile are correctly resolved relative to the project directory by passing the working directory to depfileService.parse.
| if (depfile.existsSync()) { | ||
| try { | ||
| final depfileService = DepfileService(logger: logger, fileSystem: _fileSystem); | ||
| final Depfile parsedDepfile = depfileService.parse(depfile); |
There was a problem hiding this comment.
When parsing the depfile, any relative paths written by the transformer will be resolved relative to the current working directory of the flutter_tools process (or the depfile's temp directory) rather than the project's working directory where the transformer was executed. To ensure relative paths in the depfile are correctly resolved relative to the project directory, pass _fileSystem.directory(workingDirectory) as the second argument to depfileService.parse.
| final Depfile parsedDepfile = depfileService.parse(depfile); | |
| final Depfile parsedDepfile = depfileService.parse(depfile, _fileSystem.directory(workingDirectory)); |
Update existing tests in `assets_test.dart` to expect the `--depfile` argument in `FakeCommand`s and handle it in `ArgParser`s. Also fix the `Uses processors~/2 to transform assets` test to expect 5 transformed assets instead of 4, which is the correct number of assets (1 primary + 4 variants) created and processed in the test.
Omit obvious local variable types for `processManager` and `environment` in `Uses processors~/2 to transform assets` test.
Update `FakeCommand` and `onRun` in `bundle_builder_test.dart` to support the new `--depfile` argument.
…transformers Redesign `AssetTransformer` to pass the depfile path via `FLUTTER_ASSET_TRANSFORMER_DEPFILE` environment variable instead of `--depfile` command-line argument. This ensures 100% backwards-compatibility with existing or user-defined transformers that do not support the `--depfile` argument and would otherwise crash. Make `invalidatedFiles` optional in `DevFS.updateBundle` to maintain compatibility with existing tests and callers that do not pass it. Update `asset_transformer_test.dart` to verify the new environment variable behavior, using a robust temp directory filtering approach in the depfile parsing test. Fix a latent bug in `assets_test.dart` where the `Uses processors~/2` test expected 4 transformed assets instead of 5 (1 primary + 4 variants). Revert other test files (`devfs_test.dart`, `bundle_builder_test.dart`, and other tests in `assets_test.dart`) to their `origin/main` state since they no longer need to mock the `--depfile` argument.
Update `DepfileService.parse` to accept an optional `baseDirectory` parameter, which is used to resolve relative paths found in the depfile. Pass the `workingDirectory` (project root) as `baseDirectory` to `depfileService.parse` in `AssetTransformer` to ensure relative paths written by transformers are resolved correctly. Add a unit test to `depfile_test.dart` to verify relative path resolution against a base directory.
Rewrite `AssetTransformer` to use an implicit naming convention for depfiles instead of an environment variable. Now, if a transformer wants to register dependencies, it should write its depfile to the output path with `.d` appended (i.e., `<output>.d`). `AssetTransformer` checks for the existence of this file after running the transformer, parses it, and registers the dependencies. This approach: - Is 100% backwards-compatible with existing transformers (no new arguments or env vars are passed). - Eliminates the need for environment variables. - Simplifies the code by removing the second temporary directory. Update `asset_transformer_test.dart` to match this new convention.
In the `Uses processors~/2 to transform assets` test, assert the length of `inputFilePaths` and `outputFilePaths` lists directly instead of calling `toSet()`. `MemoryFileSystem` can reuse deleted temporary directory names sequentially. Since the test runs multiple transformations, some directories are deleted before others start, leading to name reuse (e.g., two runs using `/.tmp_rand0/rand1/`). Calling `toSet()` deduplicated these paths and caused the assertion to fail with a length of 4 instead of 5 in some environments (like CI). Asserting the list length directly is more robust and correctly verifies that all 5 transformations were executed.
| final String message; | ||
| } | ||
|
|
||
| final class AssetTransformationResult { |
There was a problem hiding this comment.
Nit: using the new "Primary Constructor" would make this slightly cleaner :)
There was a problem hiding this comment.
I would if we could :( primary constructors require Dart 3.13, which isn't available for use in flutter/flutter yet.
|
Reason for revert: possibly causing flakiness https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/29840/overview |
…reload" (flutter#188751) Reverts: [[flutter_tools] Track asset transformer dependencies for hot reload](flutter#187947) Initiated by: @bkonyi Reason for reverting: possibly causing flakiness https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/29840/overview Original PR Author: @bkonyi Reviewed By: @chingjun The original PR description is provided below: [flutter_tools] Track asset transformer dependencies for hot reload Allow asset transformers to register additional input dependencies for hot reload by passing a `--depfile` flag to the transformer subprocess and parsing the resulting Ninja-format depfile. Key changes: - `AssetTransformer` now passes `--depfile` and parses it using `DepfileService`, returning dependencies in a new `AssetTransformationResult`. - `DevelopmentAssetTransformer` tracks these dependencies and provides them to `DevFS`. - `DevFS.updateBundle` does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web). - `DevFS.updateBundle` checks if any tracked dependencies are invalidated and triggers re-transformation. - `DevFS.update` and `WebDevFS.update` append collected dependencies to the watched `sources` set. - `copyAssets` build target collects and adds these dependencies to its inputs to ensure correct rebuilds. - Added unit and integration tests to verify the behavior. Fixes flutter#187051
…Reland flutter#187947) (flutter#188808) **Reland of flutter#187947** (reverted in flutter#188751) This is a reland of the previously reverted PR flutter#187947. The failure in Mac `web_tool_tests` that triggered the revert was determined to be an unrelated flake. Specifically, the test failed in build [29841](https://ci.chromium.org/ui/b/8725838089301019649) (which also failed in 29846 without the PR). Original description: [flutter_tools] Track asset transformer dependencies for hot reload Allow asset transformers to register additional input dependencies for hot reload by passing a `--depfile` flag to the transformer subprocess and parsing the resulting Ninja-format depfile. Key changes: - `AssetTransformer` now passes `--depfile` and parses it using `DepfileService`, returning dependencies in a new `AssetTransformationResult`. - `DevelopmentAssetTransformer` tracks these dependencies and provides them to `DevFS`. - `DevFS.updateBundle` does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web). - `DevFS.updateBundle` checks if any tracked dependencies are invalidated and triggers re-transformation. - `DevFS.update` and `WebDevFS.update` append collected dependencies to the watched `sources` set. - `copyAssets` build target collects and adds these dependencies to its inputs to ensure correct rebuilds. - Added unit and integration tests to verify the behavior. Fixes flutter#187051
…#12081) Manual roll Flutter from 0c80830e465b to ca9f874f5284 (119 revisions) Manual roll requested by [email protected] flutter/flutter@0c80830...ca9f874 2026-06-30 [email protected] [tool] Don't require a Flutter compile task when staging jniLibs (flutter/flutter#188805) 2026-06-30 [email protected] [Impeller] Fix potential overflow when allocating buffers at the next power of two size (flutter/flutter#188742) 2026-06-30 [email protected] [Framework/Tool] Decouple preview theme imports (flutter/flutter#188176) 2026-06-30 [email protected] Add import of `dart:_js_interop_wasm` in sdk rewriter tool (flutter/flutter#188620) 2026-06-30 [email protected] Detach LLDB and print stack trace on process stop (flutter/flutter#188576) 2026-06-30 [email protected] [Linux] Fix FlCompositorOpenGL.pixels comment (flutter/flutter#188754) 2026-06-30 [email protected] [flutter_tools] Fix crash in flutter create when pubspec.yaml is empty (flutter/flutter#188385) 2026-06-30 [email protected] Roll Packages from 656ccaa to 274ed3e (23 revisions) (flutter/flutter#188792) 2026-06-30 [email protected] In AndroidImageGenerator, check that the destination pixel buffer has sufficient capacity for the decoded data (flutter/flutter#188752) 2026-06-30 [email protected] Roll pub packages (flutter/flutter#188773) 2026-06-30 [email protected] [Impeller] Add a flat VertexAttributeFormat for vertex inputs (flutter/flutter#188684) 2026-06-30 [email protected] Roll pub packages (flutter/flutter#188764) 2026-06-30 [email protected] Roll Dart SDK from 0cb483880b6b to e1bdb9ce3327 (2 revisions) (flutter/flutter#188763) 2026-06-29 [email protected] Handle 'no permissions' adb device state (flutter/flutter#187248) 2026-06-29 [email protected] [windows]: adjusts uniform buffers to hit hlsl optimization (flutter/flutter#188538) 2026-06-29 [email protected] Roll Skia from bfb7860cb9c7 to 71947c4110b0 (9 revisions) (flutter/flutter#188747) 2026-06-29 [email protected] ci: extract wait-for-engine-build logic into a reusable composite action (flutter/flutter#188748) 2026-06-29 [email protected] Provide guided migration logs when iOS app crashes on simulator (flutter/flutter#188736) 2026-06-29 [email protected] Revert "[flutter_tools] Track asset transformer dependencies for hot reload" (flutter/flutter#188751) 2026-06-29 [email protected] Migrate ABI splits to new AGP dsl (flutter/flutter#188369) 2026-06-29 [email protected] Add Impeller+OpenGLES startup benchmark for mokey (flutter/flutter#188495) 2026-06-29 [email protected] Increase macOS minimum supported version from 10.15 to 12 to support Xcode 27 (flutter/flutter#188520) 2026-06-29 [email protected] Moves test ownership validation to flutter/flutter (flutter/flutter#188655) 2026-06-29 [email protected] Add --flavor support for Windows desktop builds (flutter/flutter#187034) 2026-06-29 [email protected] Android_hardware_smoke_test: Enable pixel exact local file comparator to read goldens from flutter asset URI (flutter/flutter#188587) 2026-06-29 [email protected] [flutter_tools] Use DeviceHub.app for iOS simulator path on Xcode 27+ (flutter/flutter#187910) 2026-06-29 [email protected] Roll Skia from 111e7582d081 to bfb7860cb9c7 (2 revisions) (flutter/flutter#188731) 2026-06-29 [email protected] Use `revert` label instead of `revert_wf` (flutter/flutter#188639) 2026-06-29 [email protected] Properly await Dart Development Service shutdown with timeout (flutter/flutter#188387) 2026-06-29 [email protected] [flutter_tools] Track asset transformer dependencies for hot reload (flutter/flutter#187947) 2026-06-29 [email protected] [Tool] Tolerate malformed UTF-8 in process streaming decoders (flutter/flutter#188453) 2026-06-29 [email protected] [flutter_tools] Use new ddc modules in test (flutter/flutter#188240) 2026-06-29 [email protected] Roll Packages from c1f7d92 to 656ccaa (12 revisions) (flutter/flutter#188728) 2026-06-29 [email protected] Remove unused fields (flutter/flutter#188705) 2026-06-29 [email protected] Clear text input handler widget on view dispose (flutter/flutter#188701) 2026-06-29 [email protected] Free compositor in view renderer finalize to avoid use-after-free (flutter/flutter#188702) 2026-06-29 [email protected] [linux] Use GWeakRef in mock signal handler test helper (flutter/flutter#188700) 2026-06-29 [email protected] Fixing few related editing issues with LTR/RTL text (flutter/flutter#188503) 2026-06-29 [email protected] [Flutter GPU] Add Texture.fromImage to wrap a ui.Image texture (flutter/flutter#188605) 2026-06-29 [email protected] [Flutter GPU] Honor the enable argument in RenderPass.setDepthWriteEnable (flutter/flutter#188715) 2026-06-29 [email protected] Roll Skia from ba1942d8c3e1 to 111e7582d081 (1 revision) (flutter/flutter#188721) 2026-06-29 [email protected] Roll Skia from 587d8befe1ee to ba1942d8c3e1 (6 revisions) (flutter/flutter#188717) 2026-06-29 [email protected] Remove some refs to package:intl (flutter/flutter#188504) 2026-06-29 [email protected] [VPAT] Update a11y assessment app FAB example to announce value change when it's updated. (flutter/flutter#188466) ...
…flutter#12081) Manual roll Flutter from 0c80830e465b to ca9f874f5284 (119 revisions) Manual roll requested by [email protected] flutter/flutter@0c80830...ca9f874 2026-06-30 [email protected] [tool] Don't require a Flutter compile task when staging jniLibs (flutter/flutter#188805) 2026-06-30 [email protected] [Impeller] Fix potential overflow when allocating buffers at the next power of two size (flutter/flutter#188742) 2026-06-30 [email protected] [Framework/Tool] Decouple preview theme imports (flutter/flutter#188176) 2026-06-30 [email protected] Add import of `dart:_js_interop_wasm` in sdk rewriter tool (flutter/flutter#188620) 2026-06-30 [email protected] Detach LLDB and print stack trace on process stop (flutter/flutter#188576) 2026-06-30 [email protected] [Linux] Fix FlCompositorOpenGL.pixels comment (flutter/flutter#188754) 2026-06-30 [email protected] [flutter_tools] Fix crash in flutter create when pubspec.yaml is empty (flutter/flutter#188385) 2026-06-30 [email protected] Roll Packages from 656ccaa to 274ed3e (23 revisions) (flutter/flutter#188792) 2026-06-30 [email protected] In AndroidImageGenerator, check that the destination pixel buffer has sufficient capacity for the decoded data (flutter/flutter#188752) 2026-06-30 [email protected] Roll pub packages (flutter/flutter#188773) 2026-06-30 [email protected] [Impeller] Add a flat VertexAttributeFormat for vertex inputs (flutter/flutter#188684) 2026-06-30 [email protected] Roll pub packages (flutter/flutter#188764) 2026-06-30 [email protected] Roll Dart SDK from 0cb483880b6b to e1bdb9ce3327 (2 revisions) (flutter/flutter#188763) 2026-06-29 [email protected] Handle 'no permissions' adb device state (flutter/flutter#187248) 2026-06-29 [email protected] [windows]: adjusts uniform buffers to hit hlsl optimization (flutter/flutter#188538) 2026-06-29 [email protected] Roll Skia from bfb7860cb9c7 to 71947c4110b0 (9 revisions) (flutter/flutter#188747) 2026-06-29 [email protected] ci: extract wait-for-engine-build logic into a reusable composite action (flutter/flutter#188748) 2026-06-29 [email protected] Provide guided migration logs when iOS app crashes on simulator (flutter/flutter#188736) 2026-06-29 [email protected] Revert "[flutter_tools] Track asset transformer dependencies for hot reload" (flutter/flutter#188751) 2026-06-29 [email protected] Migrate ABI splits to new AGP dsl (flutter/flutter#188369) 2026-06-29 [email protected] Add Impeller+OpenGLES startup benchmark for mokey (flutter/flutter#188495) 2026-06-29 [email protected] Increase macOS minimum supported version from 10.15 to 12 to support Xcode 27 (flutter/flutter#188520) 2026-06-29 [email protected] Moves test ownership validation to flutter/flutter (flutter/flutter#188655) 2026-06-29 [email protected] Add --flavor support for Windows desktop builds (flutter/flutter#187034) 2026-06-29 [email protected] Android_hardware_smoke_test: Enable pixel exact local file comparator to read goldens from flutter asset URI (flutter/flutter#188587) 2026-06-29 [email protected] [flutter_tools] Use DeviceHub.app for iOS simulator path on Xcode 27+ (flutter/flutter#187910) 2026-06-29 [email protected] Roll Skia from 111e7582d081 to bfb7860cb9c7 (2 revisions) (flutter/flutter#188731) 2026-06-29 [email protected] Use `revert` label instead of `revert_wf` (flutter/flutter#188639) 2026-06-29 [email protected] Properly await Dart Development Service shutdown with timeout (flutter/flutter#188387) 2026-06-29 [email protected] [flutter_tools] Track asset transformer dependencies for hot reload (flutter/flutter#187947) 2026-06-29 [email protected] [Tool] Tolerate malformed UTF-8 in process streaming decoders (flutter/flutter#188453) 2026-06-29 [email protected] [flutter_tools] Use new ddc modules in test (flutter/flutter#188240) 2026-06-29 [email protected] Roll Packages from c1f7d92 to 656ccaa (12 revisions) (flutter/flutter#188728) 2026-06-29 [email protected] Remove unused fields (flutter/flutter#188705) 2026-06-29 [email protected] Clear text input handler widget on view dispose (flutter/flutter#188701) 2026-06-29 [email protected] Free compositor in view renderer finalize to avoid use-after-free (flutter/flutter#188702) 2026-06-29 [email protected] [linux] Use GWeakRef in mock signal handler test helper (flutter/flutter#188700) 2026-06-29 [email protected] Fixing few related editing issues with LTR/RTL text (flutter/flutter#188503) 2026-06-29 [email protected] [Flutter GPU] Add Texture.fromImage to wrap a ui.Image texture (flutter/flutter#188605) 2026-06-29 [email protected] [Flutter GPU] Honor the enable argument in RenderPass.setDepthWriteEnable (flutter/flutter#188715) 2026-06-29 [email protected] Roll Skia from ba1942d8c3e1 to 111e7582d081 (1 revision) (flutter/flutter#188721) 2026-06-29 [email protected] Roll Skia from 587d8befe1ee to ba1942d8c3e1 (6 revisions) (flutter/flutter#188717) 2026-06-29 [email protected] Remove some refs to package:intl (flutter/flutter#188504) 2026-06-29 [email protected] [VPAT] Update a11y assessment app FAB example to announce value change when it's updated. (flutter/flutter#188466) ...
[flutter_tools] Track asset transformer dependencies for hot reload
Allow asset transformers to register additional input dependencies for hot reload by passing a
--depfileflag to the transformer subprocess and parsing the resulting Ninja-format depfile.Key changes:
AssetTransformernow passes--depfileand parses it usingDepfileService, returning dependencies in a newAssetTransformationResult.DevelopmentAssetTransformertracks these dependencies and provides them toDevFS.DevFS.updateBundledoes not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web).DevFS.updateBundlechecks if any tracked dependencies are invalidated and triggers re-transformation.DevFS.updateandWebDevFS.updateappend collected dependencies to the watchedsourcesset.copyAssetsbuild target collects and adds these dependencies to its inputs to ensure correct rebuilds.Fixes #187051