Repository navigation
[flutter_tools] Avoid unlinking native_assets.json and handle EXDEV - #196134
Conversation
Previously, `installCodeAssets` invoked `_copyNativeCodeAssetsForOS` on every `flutter test` run, which iterated `build/native_assets/<os>/` and called `ErrorHandlingFileSystem.deleteIfExists(entity, recursive: true)` on `native_assets.json` right before `_writeNativeAssetsJson` truncated and overwrote that same file in place. On macOS external volumes (for example `/Volumes/...` with endpoint-security or recycle-bin hooks intercepting `unlink(2)` and attempting a cross-volume `rename(2)`), unlinking `native_assets.json` failed with `FileSystemException (OS Error: Cross-device link, errno = 18)` (`EXDEV`). In addition, `ErrorHandlingFileSystem.deleteIfExists` passed `recursive: true` to `File.deleteSync` (routing non-directory deletions through `Directory.deleteSync`), and `_handlePosixException` did not map `EXDEV` (`errno = 18`) to a `ToolExit`. This change: 1. Skips deleting `native_assets.json` in `_copyNativeCodeAssetsForOS` when it already exists on disk as a regular file, since `_writeNativeAssetsJson` immediately truncates and overwrites it in place (while still deleting stale bundled native asset artifacts, directories, and symlinks). 2. Updates `ErrorHandlingFileSystem.deleteIfExists` to resolve `.file`, `.unixDomainSock`, and `.pipe` entities as `File` and pass `recursive: true` only when `type == .directory`. 3. Handles POSIX `EXDEV` (`errno = 18`) in `_handlePosixException` on Linux and macOS by throwing an actionable `ToolExit` instead of crashing with an unhandled `FileSystemException`. Previous output: ``` Unhandled exception: FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) #0 _File.throwIfError (dart:io/file_impl.dart:779:7) #1 _File.renameSync (dart:io/file_impl.dart:396:5) #2 ForwardingFileSystemEntity.renameSync (package:file/src/forwarding/forwarding_file_system_entity.dart:50:49) #3 ErrorHandlingFile.renameSync.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:430:31) #4 _runSync (package:flutter_tools/src/base/error_handling_io.dart:1116:16) #5 ErrorHandlingFile.renameSync (package:flutter_tools/src/base/error_handling_io.dart:429:12) #6 main (file:///usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_exdev.dart:24:37) flutter#7 _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:313:19) flutter#8 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12) ``` Current output: ``` Flutter failed to rename file at "/tmp/repro_196121_src.tmp" to "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp". A file system operation failed across different volumes or mount points. FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) Ensure that the project, build output, and temporary directories are on the same writable volume, or check external volume and security software settings. ``` Previous output (flutter test overwriting existing native_assets.json): ``` Flutter failed to delete file at "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4 d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json". Please ensure that the SDK and/or project is installed in a location that has read/write permissions for the current user. Try running: sudo chown -R $(whoami) /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc 4d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json ``` Current output (flutter test overwriting existing native_assets.json): ``` 00:00 +0: loading /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_app/test/smoke_test.dart 00:00 +0: smoke test 00:00 +1: All tests passed! ``` Fixes flutter#196121
… overwrite test Updates `native_assets_flutter_test_manifest_overwrite_test.dart` to invoke `flutterBin` (`test_utils.dart`) with `@Timeout(Duration(minutes: 5))` instead of manually compiling `bin/flutter_tools.dart` to `flutter_tools.dill`, and uses `loadPackageConfig`, `PackageConfig`, `Package`, `LanguageVersion`, and `savePackageConfig` from `package:package_config/package_config.dart` to construct the offline test project's `.dart_tool/package_config.json`. Previous output: ``` Unhandled exception: FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) #0 _File.throwIfError (dart:io/file_impl.dart:779:7) #1 _File.renameSync (dart:io/file_impl.dart:396:5) #2 ForwardingFileSystemEntity.renameSync (package:file/src/forwarding/forwarding_file_system_entity.dart:50:49) #3 ErrorHandlingFile.renameSync.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:430:31) #4 _runSync (package:flutter_tools/src/base/error_handling_io.dart:1116:16) #5 ErrorHandlingFile.renameSync (package:flutter_tools/src/base/error_handling_io.dart:429:12) #6 main (file:///usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_exdev.dart:24:37) flutter#7 _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:313:19) flutter#8 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12) ``` Current output: ``` Flutter failed to rename file at "/tmp/repro_196121_src.tmp" to "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp". A file system operation failed across different volumes or mount points. FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) Ensure that the project, build output, and temporary directories are on the same writable volume, or check external volume and security software settings. ``` Previous output (flutter test overwriting existing native_assets.json): ``` Flutter failed to delete file at "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4 d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json". Please ensure that the SDK and/or project is installed in a location that has read/write permissions for the current user. Try running: sudo chown -R $(whoami) /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc 4d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json ``` Current output (flutter test overwriting existing native_assets.json): ``` 00:00 +0: loading /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_app/test/smoke_test.dart 00:00 +0: smoke test 00:00 +1: All tests passed! ``` Fixes flutter#196121
Simplifies the fix and tests for avoiding redundant `native_assets.json` deletion and handling `EXDEV`: - Restores `ErrorHandlingFileSystem.deleteIfExists` to upstream and removes custom `ForwardingFileSystem` test doubles in `error_handling_io_test.dart`, testing `EXDEV` (`errno = 18`) directly with `MemoryFileSystem.test(opHandle: exceptionHandler.opHandle)`. - Streams `targetDir.list(followLinks: false)` in `_copyNativeCodeAssetsForOS` with `entity is File` instead of `.toList()` and `typeSync`, and deduplicates `installCodeAssets` test setup in `native_assets_test.dart`. - Removes the redundant `flutter test` subprocess integration test (`native_assets_flutter_test_manifest_overwrite_test.dart`). Reproduction notes: Both output pairs below were captured on a Linux host, which cannot reproduce the macOS external-volume behavior directly. The first pair comes from a small Dart script (`repro_exdev.dart`) calling `ErrorHandlingFileSystem.file(...).renameSync(...)` from `/tmp` (`tmpfs`) to an `ext4` directory to trigger a real kernel `EXDEV` (`errno = 18`). The second pair comes from running `BOT=true flutter test --no-pub -r expanded` on a minimal app after making `build/native_assets/linux` read-only (`chmod 0555`) while leaving `native_assets.json` writable (`0644`), so `unlink(2)` fails while in-place truncation succeeds; with a writable `build/native_assets/linux` directory, the unfixed tool passes with the same 3-line test output. Local workspace paths inside the fenced output blocks are preserved verbatim to match exact command output. Previous output: ``` Unhandled exception: FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) #0 _File.throwIfError (dart:io/file_impl.dart:779:7) #1 _File.renameSync (dart:io/file_impl.dart:396:5) #2 ForwardingFileSystemEntity.renameSync (package:file/src/forwarding/forwarding_file_system_entity.dart:50:49) #3 ErrorHandlingFile.renameSync.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:430:31) #4 _runSync (package:flutter_tools/src/base/error_handling_io.dart:1116:16) #5 ErrorHandlingFile.renameSync (package:flutter_tools/src/base/error_handling_io.dart:429:12) #6 main (file:///usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_exdev.dart:24:37) flutter#7 _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:313:19) flutter#8 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12) ``` Current output: ``` Flutter failed to rename file at "/tmp/repro_196121_src.tmp" to "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp". A file system operation failed across different volumes or mount points. FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) Ensure that the project, build output, and temporary directories are on the same writable volume, or check external volume and security software settings. ``` Previous output (flutter test overwriting existing native_assets.json): ``` Flutter failed to delete file at "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4 d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json". Please ensure that the SDK and/or project is installed in a location that has read/write permissions for the current user. Try running: sudo chown -R $(whoami) /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc 4d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json ``` Current output (flutter test overwriting existing native_assets.json): ``` 00:00 +0: loading /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_app/test/smoke_test.dart 00:00 +0: smoke test 00:00 +1: All tests passed! ``` Fixes flutter#196121
There was a problem hiding this comment.
Code Review
This pull request introduces handling for cross-device link errors (exdev) within the error-handling file system utilities, providing descriptive error messages when operations cross different volumes. Additionally, it updates the native assets installation process to overwrite the existing manifest file in place rather than deleting it, preventing unnecessary unlinking. The review feedback recommends eagerly collecting directory contents into a list before iterating and deleting files to avoid potential platform-specific issues or flakiness associated with modifying a directory during stream iteration.
…ative assets Restores `.toList()` on `targetDir.list(followLinks: false)` in `_copyNativeCodeAssetsForOS` so directory enumeration completes before deleting stale entries in `targetDir`, avoiding concurrent directory stream mutation issues on Windows and network filesystems. Reproduction notes: Both output pairs below were captured on a Linux host, which cannot reproduce the macOS external-volume behavior directly. The first pair comes from a small Dart script (`repro_exdev.dart`) calling `ErrorHandlingFileSystem.file(...).renameSync(...)` from `/tmp` (`tmpfs`) to an `ext4` directory to trigger a real kernel `EXDEV` (`errno = 18`). The second pair comes from running `BOT=true flutter test --no-pub -r expanded` on a minimal app after making `build/native_assets/linux` read-only (`chmod 0555`) while leaving `native_assets.json` writable (`0644`), so `unlink(2)` fails while in-place truncation succeeds; with a writable `build/native_assets/linux` directory, the unfixed tool passes with the same 3-line test output. Local workspace paths inside the fenced output blocks are preserved verbatim to match exact command output. Previous output: ``` Unhandled exception: FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) #0 _File.throwIfError (dart:io/file_impl.dart:779:7) #1 _File.renameSync (dart:io/file_impl.dart:396:5) #2 ForwardingFileSystemEntity.renameSync (package:file/src/forwarding/forwarding_file_system_entity.dart:50:49) #3 ErrorHandlingFile.renameSync.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:430:31) #4 _runSync (package:flutter_tools/src/base/error_handling_io.dart:1116:16) #5 ErrorHandlingFile.renameSync (package:flutter_tools/src/base/error_handling_io.dart:429:12) #6 main (file:///usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_exdev.dart:24:37) flutter#7 _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:313:19) flutter#8 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12) ``` Current output: ``` Flutter failed to rename file at "/tmp/repro_196121_src.tmp" to "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp". A file system operation failed across different volumes or mount points. FileSystemException: Cannot rename file to '/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_196121_dst.tmp', path = '/tmp/repro_196121_src.tmp' (OS Error: Invalid cross-device link, errno = 18) Ensure that the project, build output, and temporary directories are on the same writable volume, or check external volume and security software settings. ``` Previous output (flutter test overwriting existing native_assets.json): ``` Flutter failed to delete file at "/usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4 d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json". Please ensure that the SDK and/or project is installed in a location that has read/write permissions for the current user. Try running: sudo chown -R $(whoami) /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc 4d2ca98a9/scratch/repro_app/build/native_assets/linux/native_assets.json ``` Current output (flutter test overwriting existing native_assets.json): ``` 00:00 +0: loading /usr/local/google/home/bkonyi/.gemini/jetski/brain/0a669702-1e2d-46a6-9c37-fdc4d2ca98a9/scratch/repro_app/test/smoke_test.dart 00:00 +0: smoke test 00:00 +1: All tests passed! ``` Fixes flutter#196121
|
Failed to create CP due to merge conflicts. |
1 similar comment
|
Failed to create CP due to merge conflicts. |
flutter/flutter@fffd8c2...b98afb5 2026-11-11 [email protected] Roll Dart SDK from 172cf8f121c0 to 8f15c4ca2d41 (1 revision) (flutter/flutter#196367) 2026-11-11 [email protected] Roll Skia from 657283e78115 to cc4c54f752ba (5 revisions) (flutter/flutter#196365) 2026-11-11 [email protected] Roll Skia from 2605c4a9761c to 657283e78115 (1 revision) (flutter/flutter#196358) 2026-11-11 [email protected] Roll Dart SDK from 890c6d15334c to 172cf8f121c0 (1 revision) (flutter/flutter#196352) 2026-11-11 [email protected] [flutter_tools] Migrate general.shard project, runner, and web tests off globals (flutter/flutter#196128) 2026-11-11 [email protected] [flutter_tools] Migrate test helpers and integration tests off globals (flutter/flutter#196124) 2026-11-11 [email protected] [web] Report skwasm WebGL context creation failure instead of hanging (flutter/flutter#193293) 2026-11-11 [email protected] [flutter_tools] Migrate commands.shard tests off globals (flutter/flutter#196139) 2026-11-11 [email protected] [devicelab] Fix memory benchmark tap synchronization and swipe flakes (flutter/flutter#194457) 2026-11-11 [email protected] Roll Skia from 06062e36d938 to 2605c4a9761c (2 revisions) (flutter/flutter#196350) 2026-11-11 [email protected] Do not wait again for a closed mock event stream in tear down (flutter/flutter#196110) 2026-11-11 [email protected] [gen_l10n] Add `use-explicit-plural-values` option (flutter/flutter#194816) 2026-11-11 [email protected] Roll Skia from 7bd33f8a3b70 to 06062e36d938 (2 revisions) (flutter/flutter#196340) 2026-11-11 [email protected] Skip redundant work in plugin_test_* (flutter/flutter#196258) 2026-11-10 [email protected] Drop test_timeout_secs from shards that no longer need it (flutter/flutter#196277) 2026-11-10 [email protected] Roll Dart SDK from 15c1699d3eeb to 890c6d15334c (1 revision) (flutter/flutter#196321) 2026-11-10 [email protected] iOS: Extract FlutterPlatformViewsController platform-thread state to Obj-C (flutter/flutter#196142) 2026-11-10 [email protected] Remove `Mac web_tool_tests` and take new subshard builders out of bringup (flutter/flutter#196276) 2026-11-10 [email protected] Roll Skia from 9c3d43d8a04b to 7bd33f8a3b70 (12 revisions) (flutter/flutter#196317) 2026-11-10 [email protected] Take customer_testing_devtools out of bringup (flutter/flutter#196239) 2026-11-10 [email protected] Roll ICU from a0ea93436c52 to b09986242cfb (1 revision) (flutter/flutter#196316) 2026-11-10 [email protected] [flutter_tools] Avoid unlinking native_assets.json and handle EXDEV (flutter/flutter#196134) 2026-11-10 [email protected] Remove old `Windows tool_integration_tests` builders and take the new shards out of bringup (flutter/flutter#196275) 2026-11-10 [email protected] [SwiftPM] Disable tests for when settings fail to compile (flutter/flutter#196310) 2026-11-10 [email protected] Roll Fuchsia Test Scripts from PXc5D7S_q6sIYpfFJ... to oEU515sDqSTvD8Spx... (flutter/flutter#196289) 2026-11-10 [email protected] Fix ime insets animation jump (flutter/flutter#191453) 2026-11-10 [email protected] Link to original PR in automatic cherry-pick PRs (flutter/flutter#196130) 2026-11-10 [email protected] Roll Packages from 565c657 to 08faf64 (9 revisions) (flutter/flutter#196296) 2026-11-10 [email protected] Roll Skia from b11315d0a096 to 9c3d43d8a04b (45 revisions) (flutter/flutter#196295) 2026-11-10 [email protected] Replace type-specific window widgets with a single Window widget (flutter/flutter#196002) 2026-11-10 [email protected] Roll Dart SDK from a2baae273bb2 to 15c1699d3eeb (2 revisions) (flutter/flutter#196294) If this roll has caused a breakage, revert this CL and set the roller to dry run mode using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Previously,
installCodeAssetsinvoked_copyNativeCodeAssetsForOSon everyflutter testrun, which listedbuild/native_assets/<os>/and deletednative_assets.jsonright before_writeNativeAssetsJsontruncated and overwrote that same file in place. On macOS external volumes (for example/Volumes/...where endpoint-security or recycle-bin interposers interceptunlink(2)and attempt a cross-volumerename(2)), unlinkingnative_assets.jsonfailed withFileSystemException (OS Error: Cross-device link, errno = 18)(EXDEV). In addition,_handlePosixExceptiondid not mapEXDEV(errno = 18) to aToolExit.This change:
native_assets.jsonin_copyNativeCodeAssetsForOSwhen it already exists on disk as a regular file, since_writeNativeAssetsJsonimmediately truncates and overwrites it in place (while still deleting stale bundled native asset artifacts, directories, and symlinks).EXDEV(errno = 18) in_handlePosixExceptionon Linux and macOS by throwing an actionableToolExitinstead of crashing with an unhandledFileSystemException.Reproduction notes: Both output pairs below were captured on a Linux host, which cannot reproduce the macOS external-volume behavior directly. The first pair comes from a small Dart script (
repro_exdev.dart) callingErrorHandlingFileSystem.file(...).renameSync(...)from/tmp(tmpfs) to anext4directory to trigger a real kernelEXDEV(errno = 18). The second pair comes from runningBOT=true flutter test --no-pub -r expandedon a minimal app after makingbuild/native_assets/linuxread-only (chmod 0555) while leavingnative_assets.jsonwritable (0644), sounlink(2)fails while in-place truncation succeeds; with a writablebuild/native_assets/linuxdirectory, the unfixed tool passes with the same 3-line test output. Local workspace paths inside the fenced output blocks are preserved verbatim to match exact command output.Previous output:
Current output:
Previous output (flutter test overwriting existing native_assets.json):
Current output (flutter test overwriting existing native_assets.json):
Fixes #196121