Skip to content

[flutter_tools] Track asset transformer dependencies for hot reload#187947

Merged
auto-submit[bot] merged 9 commits into
flutter:masterfrom
bkonyi:track-asset-transformer-dependencies
Jun 29, 2026
Merged

[flutter_tools] Track asset transformer dependencies for hot reload#187947
auto-submit[bot] merged 9 commits into
flutter:masterfrom
bkonyi:track-asset-transformer-dependencies

Conversation

@bkonyi

@bkonyi bkonyi commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

[flutter_tools] Track asset transformer dependencies for hot reload

Allow asset transformers to register additional input dependencies for hot reload by passing a --depfile flag to the transformer subprocess and parsing the resulting Ninja-format depfile.

Key changes:

  • AssetTransformer now passes --depfile and parses it using DepfileService, returning dependencies in a new AssetTransformationResult.
  • DevelopmentAssetTransformer tracks these dependencies and provides them to DevFS.
  • DevFS.updateBundle does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web).
  • DevFS.updateBundle checks if any tracked dependencies are invalidated and triggers re-transformation.
  • DevFS.update and WebDevFS.update append collected dependencies to the watched sources set.
  • copyAssets build target collects and adds these dependencies to its inputs to ensure correct rebuilds.
  • Added unit and integration tests to verify the behavior.

Fixes #187051

Allow asset transformers to register additional input dependencies for hot reload by passing a `--depfile` flag to the transformer subprocess and parsing the resulting Ninja-format depfile.

Key changes:
- `AssetTransformer` now passes `--depfile` and parses it using `DepfileService`, returning dependencies in a new `AssetTransformationResult`.
- `DevelopmentAssetTransformer` tracks these dependencies and provides them to `DevFS`.
- `DevFS.updateBundle` does not skip transformed assets on startup, running the transformer to collect dependencies but optimizing by not uploading them to the device unless requested (e.g., for Web).
- `DevFS.updateBundle` checks if any tracked dependencies are invalidated and triggers re-transformation.
- `DevFS.update` and `WebDevFS.update` append collected dependencies to the watched `sources` set.
- `copyAssets` build target collects and adds these dependencies to its inputs to ensure correct rebuilds.
- Added unit and integration tests to verify the behavior.

Fixes flutter#187051
@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Jun 12, 2026
@github-actions github-actions Bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Jun 12, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the asset transformation process to track and monitor dependencies. The AssetTransformer now parses a generated depfile and returns an AssetTransformationResult containing these dependencies. DevFS and WebDevFS use this information to trigger asset re-transformations when dependencies change. Feedback was provided to ensure relative paths in the depfile are correctly resolved relative to the project directory by passing the working directory to depfileService.parse.

if (depfile.existsSync()) {
try {
final depfileService = DepfileService(logger: logger, fileSystem: _fileSystem);
final Depfile parsedDepfile = depfileService.parse(depfile);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When parsing the depfile, any relative paths written by the transformer will be resolved relative to the current working directory of the flutter_tools process (or the depfile's temp directory) rather than the project's working directory where the transformer was executed. To ensure relative paths in the depfile are correctly resolved relative to the project directory, pass _fileSystem.directory(workingDirectory) as the second argument to depfileService.parse.

Suggested change
final Depfile parsedDepfile = depfileService.parse(depfile);
final Depfile parsedDepfile = depfileService.parse(depfile, _fileSystem.directory(workingDirectory));

Update existing tests in `assets_test.dart` to expect the `--depfile` argument in `FakeCommand`s and handle it in `ArgParser`s.

Also fix the `Uses processors~/2 to transform assets` test to expect 5 transformed assets instead of 4, which is the correct number of assets (1 primary + 4 variants) created and processed in the test.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
Omit obvious local variable types for `processManager` and `environment` in `Uses processors~/2 to transform assets` test.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
Update `FakeCommand` and `onRun` in `bundle_builder_test.dart` to support the new `--depfile` argument.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
…transformers

Redesign `AssetTransformer` to pass the depfile path via `FLUTTER_ASSET_TRANSFORMER_DEPFILE` environment variable instead of `--depfile` command-line argument. This ensures 100% backwards-compatibility with existing or user-defined transformers that do not support the `--depfile` argument and would otherwise crash.

Make `invalidatedFiles` optional in `DevFS.updateBundle` to maintain compatibility with existing tests and callers that do not pass it.

Update `asset_transformer_test.dart` to verify the new environment variable behavior, using a robust temp directory filtering approach in the depfile parsing test.

Fix a latent bug in `assets_test.dart` where the `Uses processors~/2` test expected 4 transformed assets instead of 5 (1 primary + 4 variants).

Revert other test files (`devfs_test.dart`, `bundle_builder_test.dart`, and other tests in `assets_test.dart`) to their `origin/main` state since they no longer need to mock the `--depfile` argument.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 12, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 12, 2026
Update `DepfileService.parse` to accept an optional `baseDirectory` parameter, which is used to resolve relative paths found in the depfile.

Pass the `workingDirectory` (project root) as `baseDirectory` to `depfileService.parse` in `AssetTransformer` to ensure relative paths written by transformers are resolved correctly.

Add a unit test to `depfile_test.dart` to verify relative path resolution against a base directory.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 19, 2026
Rewrite `AssetTransformer` to use an implicit naming convention for depfiles instead of an environment variable.

Now, if a transformer wants to register dependencies, it should write its depfile to the output path with `.d` appended (i.e., `<output>.d`). `AssetTransformer` checks for the existence of this file after running the transformer, parses it, and registers the dependencies.

This approach:
- Is 100% backwards-compatible with existing transformers (no new arguments or env vars are passed).
- Eliminates the need for environment variables.
- Simplifies the code by removing the second temporary directory.

Update `asset_transformer_test.dart` to match this new convention.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 19, 2026
In the `Uses processors~/2 to transform assets` test, assert the length of `inputFilePaths` and `outputFilePaths` lists directly instead of calling `toSet()`.

`MemoryFileSystem` can reuse deleted temporary directory names sequentially. Since the test runs multiple transformations, some directories are deleted before others start, leading to name reuse (e.g., two runs using `/.tmp_rand0/rand1/`). Calling `toSet()` deduplicated these paths and caused the assertion to fail with a length of 4 instead of 5 in some environments (like CI).

Asserting the list length directly is more robust and correctly verifies that all 5 transformations were executed.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 19, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 19, 2026
@bkonyi
bkonyi requested a review from chingjun June 23, 2026 17:14
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 25, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 25, 2026
final String message;
}

final class AssetTransformationResult {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: using the new "Primary Constructor" would make this slightly cleaner :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would if we could :( primary constructors require Dart 3.13, which isn't available for use in flutter/flutter yet.

@bkonyi bkonyi added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 29, 2026
@auto-submit
auto-submit Bot added this pull request to the merge queue Jun 29, 2026
Merged via the queue into flutter:master with commit fd73141 Jun 29, 2026
167 of 168 checks passed
@flutter-dashboard flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jun 29, 2026
@bkonyi

bkonyi commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@bkonyi bkonyi added the revert Autorevert PR (with "Reason for revert:" comment) label Jun 29, 2026
pull Bot pushed a commit to edisplay/flutter that referenced this pull request Jun 30, 2026
…reload" (flutter#188751)

Reverts: [[flutter_tools] Track asset transformer dependencies for hot
reload](flutter#187947)

Initiated by: @bkonyi

Reason for reverting: possibly causing flakiness
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/29840/overview

Original PR Author: @bkonyi

Reviewed By: @chingjun

The original PR description is provided below:

[flutter_tools] Track asset transformer dependencies for hot reload

Allow asset transformers to register additional input dependencies for
hot reload by passing a `--depfile` flag to the transformer subprocess
and parsing the resulting Ninja-format depfile.

Key changes:
- `AssetTransformer` now passes `--depfile` and parses it using
`DepfileService`, returning dependencies in a new
`AssetTransformationResult`.
- `DevelopmentAssetTransformer` tracks these dependencies and provides
them to `DevFS`.
- `DevFS.updateBundle` does not skip transformed assets on startup,
running the transformer to collect dependencies but optimizing by not
uploading them to the device unless requested (e.g., for Web).
- `DevFS.updateBundle` checks if any tracked dependencies are
invalidated and triggers re-transformation.
- `DevFS.update` and `WebDevFS.update` append collected dependencies to
the watched `sources` set.
- `copyAssets` build target collects and adds these dependencies to its
inputs to ensure correct rebuilds.
- Added unit and integration tests to verify the behavior.

Fixes flutter#187051
pull Bot pushed a commit to jinzaizhichi/flutter that referenced this pull request Jul 1, 2026
…Reland flutter#187947) (flutter#188808)

**Reland of flutter#187947** (reverted in flutter#188751)

This is a reland of the previously reverted PR flutter#187947. The failure in
Mac `web_tool_tests` that triggered the revert was determined to be an
unrelated flake. Specifically, the test failed in build
[29841](https://ci.chromium.org/ui/b/8725838089301019649) (which also
failed in 29846 without the PR).

Original description:
[flutter_tools] Track asset transformer dependencies for hot reload

Allow asset transformers to register additional input dependencies for
hot reload by passing a `--depfile` flag to the transformer subprocess
and parsing the resulting Ninja-format depfile.

Key changes:
- `AssetTransformer` now passes `--depfile` and parses it using
`DepfileService`, returning dependencies in a new
`AssetTransformationResult`.
- `DevelopmentAssetTransformer` tracks these dependencies and provides
them to `DevFS`.
- `DevFS.updateBundle` does not skip transformed assets on startup,
running the transformer to collect dependencies but optimizing by not
uploading them to the device unless requested (e.g., for Web).
- `DevFS.updateBundle` checks if any tracked dependencies are
invalidated and triggers re-transformation.
- `DevFS.update` and `WebDevFS.update` append collected dependencies to
the watched `sources` set.
- `copyAssets` build target collects and adds these dependencies to its
inputs to ensure correct rebuilds.
- Added unit and integration tests to verify the behavior.

Fixes flutter#187051
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request Jul 7, 2026
…#12081)

Manual roll Flutter from 0c80830e465b to ca9f874f5284 (119 revisions)

Manual roll requested by [email protected]

flutter/flutter@0c80830...ca9f874

2026-06-30 [email protected] [tool] Don't require a Flutter compile task when staging jniLibs (flutter/flutter#188805)
2026-06-30 [email protected] [Impeller] Fix potential overflow when allocating buffers at the next power of two size (flutter/flutter#188742)
2026-06-30 [email protected] [Framework/Tool] Decouple preview theme imports (flutter/flutter#188176)
2026-06-30 [email protected] Add import of `dart:_js_interop_wasm` in sdk rewriter tool (flutter/flutter#188620)
2026-06-30 [email protected] Detach LLDB and print stack trace on process stop (flutter/flutter#188576)
2026-06-30 [email protected] [Linux] Fix FlCompositorOpenGL.pixels comment (flutter/flutter#188754)
2026-06-30 [email protected] [flutter_tools] Fix crash in flutter create when pubspec.yaml is empty (flutter/flutter#188385)
2026-06-30 [email protected] Roll Packages from 656ccaa to 274ed3e (23 revisions) (flutter/flutter#188792)
2026-06-30 [email protected] In AndroidImageGenerator, check that the destination pixel buffer has sufficient capacity for the decoded data (flutter/flutter#188752)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188773)
2026-06-30 [email protected] [Impeller] Add a flat VertexAttributeFormat for vertex inputs (flutter/flutter#188684)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188764)
2026-06-30 [email protected] Roll Dart SDK from 0cb483880b6b to e1bdb9ce3327 (2 revisions) (flutter/flutter#188763)
2026-06-29 [email protected] Handle 'no permissions' adb device state (flutter/flutter#187248)
2026-06-29 [email protected] [windows]: adjusts uniform buffers to hit hlsl optimization (flutter/flutter#188538)
2026-06-29 [email protected] Roll Skia from bfb7860cb9c7 to 71947c4110b0 (9 revisions) (flutter/flutter#188747)
2026-06-29 [email protected] ci: extract wait-for-engine-build logic into a reusable composite action (flutter/flutter#188748)
2026-06-29 [email protected] Provide guided migration logs when iOS app crashes on simulator (flutter/flutter#188736)
2026-06-29 [email protected] Revert "[flutter_tools] Track asset transformer dependencies for hot reload" (flutter/flutter#188751)
2026-06-29 [email protected] Migrate ABI splits to new AGP dsl (flutter/flutter#188369)
2026-06-29 [email protected] Add Impeller+OpenGLES startup benchmark for mokey (flutter/flutter#188495)
2026-06-29 [email protected] Increase macOS minimum supported version from 10.15 to 12 to support Xcode 27  (flutter/flutter#188520)
2026-06-29 [email protected] Moves test ownership validation to flutter/flutter (flutter/flutter#188655)
2026-06-29 [email protected] Add --flavor support for Windows desktop builds (flutter/flutter#187034)
2026-06-29 [email protected] Android_hardware_smoke_test: Enable pixel exact local file comparator to read goldens from flutter asset URI (flutter/flutter#188587)
2026-06-29 [email protected] [flutter_tools] Use DeviceHub.app for iOS simulator path on Xcode 27+ (flutter/flutter#187910)
2026-06-29 [email protected] Roll Skia from 111e7582d081 to bfb7860cb9c7 (2 revisions) (flutter/flutter#188731)
2026-06-29 [email protected] Use `revert` label instead of `revert_wf` (flutter/flutter#188639)
2026-06-29 [email protected] Properly await Dart Development Service shutdown with timeout (flutter/flutter#188387)
2026-06-29 [email protected] [flutter_tools] Track asset transformer dependencies for hot reload (flutter/flutter#187947)
2026-06-29 [email protected] [Tool] Tolerate malformed UTF-8 in process streaming decoders (flutter/flutter#188453)
2026-06-29 [email protected] [flutter_tools] Use new ddc modules in test (flutter/flutter#188240)
2026-06-29 [email protected] Roll Packages from c1f7d92 to 656ccaa (12 revisions) (flutter/flutter#188728)
2026-06-29 [email protected] Remove unused fields (flutter/flutter#188705)
2026-06-29 [email protected] Clear text input handler widget on view dispose (flutter/flutter#188701)
2026-06-29 [email protected] Free compositor in view renderer finalize to avoid use-after-free (flutter/flutter#188702)
2026-06-29 [email protected] [linux] Use GWeakRef in mock signal handler test helper (flutter/flutter#188700)
2026-06-29 [email protected] Fixing few related editing issues with LTR/RTL text (flutter/flutter#188503)
2026-06-29 [email protected] [Flutter GPU] Add Texture.fromImage to wrap a ui.Image texture (flutter/flutter#188605)
2026-06-29 [email protected] [Flutter GPU] Honor the enable argument in RenderPass.setDepthWriteEnable (flutter/flutter#188715)
2026-06-29 [email protected] Roll Skia from ba1942d8c3e1 to 111e7582d081 (1 revision) (flutter/flutter#188721)
2026-06-29 [email protected] Roll Skia from 587d8befe1ee to ba1942d8c3e1 (6 revisions) (flutter/flutter#188717)
2026-06-29 [email protected] Remove some refs to package:intl (flutter/flutter#188504)
2026-06-29 [email protected] [VPAT] Update a11y assessment app FAB example to announce value change when it's updated. (flutter/flutter#188466)
...
kalyujniy pushed a commit to brickit-app/camera that referenced this pull request Jul 8, 2026
…flutter#12081)

Manual roll Flutter from 0c80830e465b to ca9f874f5284 (119 revisions)

Manual roll requested by [email protected]

flutter/flutter@0c80830...ca9f874

2026-06-30 [email protected] [tool] Don't require a Flutter compile task when staging jniLibs (flutter/flutter#188805)
2026-06-30 [email protected] [Impeller] Fix potential overflow when allocating buffers at the next power of two size (flutter/flutter#188742)
2026-06-30 [email protected] [Framework/Tool] Decouple preview theme imports (flutter/flutter#188176)
2026-06-30 [email protected] Add import of `dart:_js_interop_wasm` in sdk rewriter tool (flutter/flutter#188620)
2026-06-30 [email protected] Detach LLDB and print stack trace on process stop (flutter/flutter#188576)
2026-06-30 [email protected] [Linux] Fix FlCompositorOpenGL.pixels comment (flutter/flutter#188754)
2026-06-30 [email protected] [flutter_tools] Fix crash in flutter create when pubspec.yaml is empty (flutter/flutter#188385)
2026-06-30 [email protected] Roll Packages from 656ccaa to 274ed3e (23 revisions) (flutter/flutter#188792)
2026-06-30 [email protected] In AndroidImageGenerator, check that the destination pixel buffer has sufficient capacity for the decoded data (flutter/flutter#188752)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188773)
2026-06-30 [email protected] [Impeller] Add a flat VertexAttributeFormat for vertex inputs (flutter/flutter#188684)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188764)
2026-06-30 [email protected] Roll Dart SDK from 0cb483880b6b to e1bdb9ce3327 (2 revisions) (flutter/flutter#188763)
2026-06-29 [email protected] Handle 'no permissions' adb device state (flutter/flutter#187248)
2026-06-29 [email protected] [windows]: adjusts uniform buffers to hit hlsl optimization (flutter/flutter#188538)
2026-06-29 [email protected] Roll Skia from bfb7860cb9c7 to 71947c4110b0 (9 revisions) (flutter/flutter#188747)
2026-06-29 [email protected] ci: extract wait-for-engine-build logic into a reusable composite action (flutter/flutter#188748)
2026-06-29 [email protected] Provide guided migration logs when iOS app crashes on simulator (flutter/flutter#188736)
2026-06-29 [email protected] Revert "[flutter_tools] Track asset transformer dependencies for hot reload" (flutter/flutter#188751)
2026-06-29 [email protected] Migrate ABI splits to new AGP dsl (flutter/flutter#188369)
2026-06-29 [email protected] Add Impeller+OpenGLES startup benchmark for mokey (flutter/flutter#188495)
2026-06-29 [email protected] Increase macOS minimum supported version from 10.15 to 12 to support Xcode 27  (flutter/flutter#188520)
2026-06-29 [email protected] Moves test ownership validation to flutter/flutter (flutter/flutter#188655)
2026-06-29 [email protected] Add --flavor support for Windows desktop builds (flutter/flutter#187034)
2026-06-29 [email protected] Android_hardware_smoke_test: Enable pixel exact local file comparator to read goldens from flutter asset URI (flutter/flutter#188587)
2026-06-29 [email protected] [flutter_tools] Use DeviceHub.app for iOS simulator path on Xcode 27+ (flutter/flutter#187910)
2026-06-29 [email protected] Roll Skia from 111e7582d081 to bfb7860cb9c7 (2 revisions) (flutter/flutter#188731)
2026-06-29 [email protected] Use `revert` label instead of `revert_wf` (flutter/flutter#188639)
2026-06-29 [email protected] Properly await Dart Development Service shutdown with timeout (flutter/flutter#188387)
2026-06-29 [email protected] [flutter_tools] Track asset transformer dependencies for hot reload (flutter/flutter#187947)
2026-06-29 [email protected] [Tool] Tolerate malformed UTF-8 in process streaming decoders (flutter/flutter#188453)
2026-06-29 [email protected] [flutter_tools] Use new ddc modules in test (flutter/flutter#188240)
2026-06-29 [email protected] Roll Packages from c1f7d92 to 656ccaa (12 revisions) (flutter/flutter#188728)
2026-06-29 [email protected] Remove unused fields (flutter/flutter#188705)
2026-06-29 [email protected] Clear text input handler widget on view dispose (flutter/flutter#188701)
2026-06-29 [email protected] Free compositor in view renderer finalize to avoid use-after-free (flutter/flutter#188702)
2026-06-29 [email protected] [linux] Use GWeakRef in mock signal handler test helper (flutter/flutter#188700)
2026-06-29 [email protected] Fixing few related editing issues with LTR/RTL text (flutter/flutter#188503)
2026-06-29 [email protected] [Flutter GPU] Add Texture.fromImage to wrap a ui.Image texture (flutter/flutter#188605)
2026-06-29 [email protected] [Flutter GPU] Honor the enable argument in RenderPass.setDepthWriteEnable (flutter/flutter#188715)
2026-06-29 [email protected] Roll Skia from ba1942d8c3e1 to 111e7582d081 (1 revision) (flutter/flutter#188721)
2026-06-29 [email protected] Roll Skia from 587d8befe1ee to ba1942d8c3e1 (6 revisions) (flutter/flutter#188717)
2026-06-29 [email protected] Remove some refs to package:intl (flutter/flutter#188504)
2026-06-29 [email protected] [VPAT] Update a11y assessment app FAB example to announce value change when it's updated. (flutter/flutter#188466)
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flutter_tools] Allow asset transformers to register additional input dependencies for hot reload

2 participants